lokeshchinthala / ehost

Annotation Tool: The extensible Human Oracle Suite of Tools (eHOST)

New to NLP:

If you are new to NLP, when developing an NLP system often researchers must manually mark or annotate the information that the NLP system should identify automatically. The extensible Human Oracle Suite of Tools (eHOST) enables an expert to annotate texts; thereby, marking the span of the text string representing the information of interest. eHOST also supports comparison of two or more experts annotations and adjudication of disagreements creating a reference standard e.g., the set of agreed-upon correct answers used to bench mark the performance of an NLP system.

Know NLP:

If you are familiar with NLP, the extensible Human Oracle Suite of Tools (eHOST) is a prototype annotation system that provides an open-source stand-alone client for manual annotation of clinical texts.The eHOST annotation tool has been used by several institutions and projects for a variety of tasks, including both the 2010 and 2011 i2b2/VA Challenges, annotation tasks for the Consortium for Healthcare Informatics Research (CHIR) projects. This annotation tool supports annotation of concepts, their attributes, and relations between concepts. The tool has the ability to 1) generate and pre-annotate texts using dictionaries, 2) provide machine-assisted annotation using real-time string machine to represent concepts, 3) support encoding of standard clinical vocabularies such as the UMLS and SNOMED-CT with fast API calls and query, and 4) compute inter-annotator agreement using flexible match criteria. eHOST is also reads/writes annotations from/to Knowtator-formatted xml files.

System development was funded by University of Utah and VA Consortium for Healthcare Informatics Research between 2009-2011.

Online help:

Guide: Online Guide