Comments (5)
Hi, I'm not sure what you mean exactly. Can you give an example?
from errant.
Example: (to generate .m2 file like this one, assuming two annotators and some sentences having only single annotator)
'''
This is a tet sentence.
A 3 3 ||..........|||0
A 3 3 ||..........|||1
This is second tet sentence.
A 3 3 ||...........|||0
This is the third tet sentence.
A 4 4 ||............|||1
'''
from errant.
Aha right ok. So in some files (mainly NUCLE and CoNLL) whenever an annotator is "missing", the implication is that they made no changes to the sentence. So in your above example, Annotator 1 thought the second sentence is already correct, and Annotator 0 thought the third sentence is already correct.
If it makes things easier, you can add a noop edit for each "missing" annotator to explicitly indicate that they made no changes to the sentence; e.g.
'''
This is second tet sentence.
A 3 3 ||...........|||0
A -1 -1|||noop|||-NONE-|||REQUIRED|||-NONE-|||1
This is the third tet sentence.
A -1 -1|||noop|||-NONE-|||REQUIRED|||-NONE-|||0
A 4 4 ||............|||1
'''
from errant.
I think the case you mentioned can be handled by the current errant_parallel apis by making correction= source sentence.
In my case, all the annotators have not annotated all the sentences.
Continuing the example, the second sentence was not annotated by Annotator 1, i.e. Annotator 1 does not know that the second sentence is correct or not. In such cases, noop will not work.
I was wondering if in the correction file, we can insert a symbol saying the annotation is not present.
from errant.
Ah I see. Yes, ERRANT was never designed for this as it's generally assumed that all annotators will annotate the same number of sentences otherwise correct sentences are indistinguishable from unannotated sentences.
There's no easy way around this; I wouldn't want to add a special symbol for missing annotations because this would also affect other datasets too.
The only other option I can think of is to change the annotator IDs based on how many annotations there are; e.g.
S Sentence1
A 2 2|||...|||0 - Annotator 0
A 2 3|||...|||1 - Annotator 1
A 4 6|||...|||2 - Annotator 2
S Sentence2
Missing - Annotator 0
A 4 5|||...|||0 - Annotator 1
A -1 -1|||...|||1 - Annotator 2
S Sentence3
A 2 2|||...|||0 - Annotator 0
Missing - Annotator 1
A 2 3|||...|||1 - Annotator 2
In Sentence1, all 3 annotators saw the sentence and made edits.
In Sentence2, Annotator 0 missed the sentence, Annotator 1 made an edit and became Annotator 0, and Annotator 2 saw the sentence but made no edits and became Annotator 1.
In Sentence 3, Annotator 0 made an edit and stayed as Annotator 0, Annotator 1 missed the sentence, and Annotator 2 made an edit and became Annotator 1.
This kind of structure will work in ERRANT, but note that the M2 annotator IDs will no longer refer to a specific annotator (if that's important, which it usually isn't).
from errant.
Related Issues (20)
- Add a test to check spacy and errant work properly
- How can I transform the M2 back to plain text HOT 2
- Errant for Arabic language HOT 2
- Errant incompatible with spacy 3 HOT 9
- Expose errant_compare functionality via the API HOT 3
- Merge Casing Issue HOT 2
- Edits missed for a substitute -> Delete -> Substitute sequence. HOT 3
- OSError: [E053] Could not read meta.json from en\meta.json HOT 3
- Implementation issue HOT 6
- UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2490: ordinal not in range(128) HOT 2
- Parallel_to_m2 is not working HOT 1
- Licensing concerns HOT 6
- Errant parse method not working HOT 5
- Wrong format for incorr_sentences.txt HOT 4
- ‘’AttributeError: 'English' object has no attribute 'tagger'” when running the "Quick Start" code in API given in README.md HOT 4
- Ignore temporary files generated by installation HOT 1
- cancelling
- Edit indices HOT 3
- Simulate Errors HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from errant.