bdferr / masters-project

Python scripts used to prepare historical German corpora for master's project

masters-project

These are the Python scripts I used to prepare corpora for my master's project, entitled Application of a POS Tagger to a Novel Chronological Division of Early Modern German Text. The tagger used was TnT (Trigrams 'n' Tags), developed by Thorsten Brants and discussed in his 2000 paper TnT - A Statistical Part of Speech Tagger, while the corpora were composed of documents from the University of Manchester's GerManC corpus, which covered the years from 1654 to 1799. I ran the tagger with a Linux Oracle VM VirtualBox. I split the documents into two halves with the boundary years 1732/1733, with both halves used for both training and testing in various configurations, and found that the earlier half was generally more difficult to tag correctly, as expected due to a lesser degree of standardization in the German of that period. The scripts were necessary both to group the documents into training and test sets and to format them so as to be comprehensible to the tagger. More information on the methodology (beginning on page 52) and results (page 64) can be found in my master's paper here: https://cdr.lib.unc.edu/record/uuid:353164ad-87b2-48bb-a665-f82cf5755c19