FinnPos: an open-source morphological tagging and lemmatization toolkit for Finnish.

This paper describes FinnPos, an open-source morphological tagging and lemmatization toolkit for Finnish. The morphological tagging model is based on the averaged structured perceptron classifier. Given training data, new taggers are estimated in a computationally efficient manner using a combinatio...

Full description

Bibliographic Details
Published in:Language Resources & Evaluation Vol. 50; no. 4; pp. 863 - 879
Main Authors: Silfverberg, Miikka, Ruokolainen, Teemu, Lindén, Krister, Kurimo, Mikko
Format: Article
Published: Springer Nature Dec2016
Subjects:
Online Access:View this record in EBSCOhost
Description
Summary:This paper describes FinnPos, an open-source morphological tagging and lemmatization toolkit for Finnish. The morphological tagging model is based on the averaged structured perceptron classifier. Given training data, new taggers are estimated in a computationally efficient manner using a combination of beam search and model cascade. The lemmatization is performed employing a combination of a rule-based morphological analyzer, OMorFi, and a data-driven lemmatization model. The toolkit is readily applicable for tagging and lemmatization of running text with models learned from the recently published Finnish Turku Dependency Treebank and FinnTreeBank. Empirical evaluation on these corpora shows that FinnPos performs favorably compared to reference systems in terms of tagging and lemmatization accuracy. In addition, we demonstrate that our system is highly competitive with regard to computational efficiency of learning new models and assigning analyses to novel sentences.