spyysalo / conlleval.py Goto Github PK
View Code? Open in Web Editor NEWPython version of the evaluation script from CoNLL'00-
License: MIT License
Python version of the evaluation script from CoNLL'00-
License: MIT License
The break in sequences where an E
tag is followed by a B
tag is ignored. For example,
North B-MISC B-MISC
African E-MISC I-MISC
Grand B-MISC I-MISC
Prix E-MISC E-MISC
(correct: "North African" and "Grand Prix", two mentions; predicted: "North African Grand Prix", one mention) is evaluated as
processed 4 tokens with 1 phrases; found: 1 phrases; correct: 1.
accuracy: 50.00%; precision: 100.00%; recall: 100.00%; FB1: 100.00
MISC: precision: 100.00%; recall: 100.00%; FB1: 100.00 1
Giving a contradictory 100% f-score with 50% accuracy.
This occurs also in the original perl version. While not an issue for the IOB and IOE schemes that the script explicitly supports (see http://arxiv.org/pdf/cs/9907006.pdf), this is arguably wrong for IOBES (which mostly evaluates as expected).
The script takes input as a sequence of strings in CoNLL format:
https://github.com/spyysalo/conlleval.py/blob/master/conlleval.py#L57
This is fine for command-line use, but cumbersome for calling from Python. There should be a way to invoke evaluation with Python objects.
h/t: @marekrei
My output.txt file is the following format:
Sao NC B-LOC B-LOC
Paulo VMI I-LOC I-LOC
( Fpa O O
Brasil NC B-LOC B-LOC
) Fpt O O
, Fc O O
23 Z O O
may NC O O
( Fpa O O
EFECOM NP B-ORG B-ORG
) Fpt O O
. Fp O O
Which should be okay according to the conll format. But after running the conll.py I am getting this error:
OSError: conlleval: unexpected number of features in line Sao NC B-LOC B-LOC
Do you have any idea why? @spyysalo
Thanks in advance!
IOBES tags should be supported (related to #1).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.