MORPHA STEMMER
http://www.informatics.sussex.ac.uk/research/groups/nlp/carroll/morph.html
A fast and robust morphological analyser for English based on finite-state
techniques that returns the lemma and inflection type of a word, given the word
form and its part of speech. (The latter is optional but accuracy is degraded
if it is not present).
Converted to Java using JFlex. The class is not threadsafe.
README FROM ORIGINAL MORPHA DISTRIBUTION
University of Sussex 8 Sep 2003
This directory contains software for morphological processing of English
as developed by Kevin Humphreys <[email protected]>, John Carroll
<[email protected]> and Guido Minnen.
To be used for research purposes only (see section 4 below). If you make
any changes, the authors would appreciate it if you sent them details of
what you have done.
Covers the English inflectional suffixes:
-s plural of nouns, 3rd person singular present of verbs
-ed past tense
-en past participle
-ing progressive of verbs
1. Usage
--------
morpha [-a] [-c] [-t] [-u] [-f verbstem-file]
morphg [-c] [-t] [-u] [-f verbstem-file]
The commands operate as filters, reading from the standard input and
writing to the standard output.
They may be invoked with the following command-line options:
-a Output affixes (morpha only).
-c Preserve case distinctions wherever possible.
-t Output part-of-speech tags if they are in the input.
-u Indicate that the words in the input are not tagged with
part-of-speech labels. N.B. This mode of use is not recommended
since the resulting ambiguity in the input is likely to lead to
incorrect output.
-f By default, the commands attempt to read a file called
'verbstem.list' in the user's current directory which is expected
to contain a list of stems of verbs that undergo doubling of
their final consonant, as occurs in British English spelling.
This option allows the user to specify a different file of verb
stems (for example if American English behaviour is required).
If this option is specified then it must be the last one on
the command-line.
See the file doc.txt for specifications of input and output formats,
and examples of usage.
2. Files
--------
Makefile makefile for compiling the flex sources; can be
used for compiling both flex descriptions by
the command `make flex-description-file'
README this file
doc.txt specifications of input/output formats, and usage
examples
gpost postamble file used in deriving morphg.lex
gpre preamble file used in deriving morphg.lex
invert.sh unix shell program that derives morphg.lex from
morpha.lex
minnen.pdf pre-final PDF version of the NLE article by Minnen,
Carroll and Pearce (2001)
morpha.{ix86_linux|ppc_darwin|sun4_sunos}
executables for the morphological analyser; for
details of usage see above
morpha.lex flex description constituting the source of the
morphological analyser
morphg.{ix86_linux|ppc_darwin|sun4_sunos}
executables for the morphological generator; for
details of usage see above
morphg.lex flex description constituting the source of the
morphological generator
verbstem.list list of verb stems that allow for consonant doubling
in British English
The file morphg.lex is derived automatically from the file morpha.lex
using invert.sh, as described in the paper by Minnen, Carroll and
Pearce (2001) -- full reference below.
3. Compilation
--------------
To recompile the morph tools, either type the following commands
(making sure that you use the 2.5.4a version of flex recompiled with
larger internal limits -- see below), or (more conveniently) use the
Makefile in this directory by typing `make morpha' or `make morphg'.
flex -i -Cfe -8 -omorpha.yy.c morpha.lex
gcc -o morpha morpha.yy.c
or
flex -i -Cfe -8 -omorphg.yy.c morphg.lex
gcc -o morphg morphg.yy.c
The executables included in this release were built omitting the
Flex options -Cfe -8, resulting in a reduction in binary file size
of two thirds (and a reduction in processing speed of around 20%).
These options also have to be left out and the option -Dinteractive
added to gcc (resulting in a further decrease in throughput) in order
to get the morph tools to return results immediately when used via
unix pipes inside other programs.
N.B. Recompiling the morph tools requires an adapted version of Flex.
The Flex source code is freely available from:
http://www.go.dlr.de/fresh/unix/src/misc/.warix/flex-2.5.4a.tar.gz.html
The Flex source should be changed to allow for more internal states by
increasing the definitions in flexdef.h of:
#define JAMSTATE -32766
...
#define MAXIMUM_MNS 31999
...
#define BAD_SUBSCRIPT -32767
to:
#define JAMSTATE -800000
...
#define MAXIMUM_MNS 800000
...
#define BAD_SUBSCRIPT -800000
and recompiling Flex. When recompiling the morph tools ensure that the
Makefile points to the new version of Flex.
4. Acknowledgements, copyrights etc.
------------------------------------
Copyright (c) 1995-2000 University of Sheffield, University of Sussex
All rights reserved.
Redistribution and use of source and derived binary forms are
permitted without fee provided that:
- they are not used in commercial products
- the above copyright notice and this paragraph are duplicated in
all such forms
- any documentation, advertising materials, and other materials
related to such distribution and use acknowledge that the software
was developed by Kevin Humphreys <[email protected]>, John
Carroll <[email protected]> and Guido Minnen
and refer to the following related publication:
Guido Minnen, John Carroll and Darren Pearce. 2001. `Applied
morphological processing of English'. Natural Language Engineering,
7(3). 207-223.
The name of University of Sheffield may not be used to endorse or
promote products derived from this software without specific prior
written permission.
This software is provided "as is" and without any express or implied
warranties, including, without limitation, the implied warranties of
merchantibility and fitness for a particular purpose.
The exception lists were derived semi-automatically from WordNet 1.5,
and various other corpora and MRDs.
Many thanks to Tim Baldwin, Chris Brew, Bill Fisher, Gerald Gazdar,
Dale Gerdemann, Adam Kilgarriff and Ehud Reiter for suggested
improvements.
WordNet 1.5 Copyright 1995 by Princeton University.
All rights reseved.
THIS SOFTWARE AND DATABASE IS PROVIDED "AS IS" AND PRINCETON
UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED.
BY WAY OF EXAMPLE, BUT NOT LIMITATION, PRINCETON UNIVERSITY MAKES NO
REPRESENTATIONS OR WARRANTIES OF MERCHANT- ABILITY OR FITNESS FOR ANY
PARTICULAR PURPOSE OR THAT THE USE OF THE LICENSED SOFTWARE, DATABASE
OR DOCUMENTATION WILL NOT INFRINGE ANY THIRD PARTY PATENTS,
COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS.
The name of Princeton University or Princeton may not be used in
advertising or publicity pertaining to distribution of the software
and/or database. Title to copyright in this software, database and
any associated documentation shall at all times remain with Princeton
University and LICENSEE agrees to preserve same.