Report on the revised Corpus Acquisition & Annotation subsystem and its components

Prokopidis, Prokopis; Papavassiliou, Vassilis; Toral, Antonio; Poch, Marc; Frontini, Francesca; Rubino, Francesco; Thurmair, Gregor

Report on the revised Corpus Acquisition & Annotation subsystem and its components

Llicència

Citació

Prokopidis P, Papavassiliou V, Toral A, Poch M, Frontini F, Rubino F, Thurmair G. Report on the revised Corpus Acquisition & Annotation subsystem and its components [Internet]. Final report 02 Nov 2011. 36 p. (Panacea Project. Work Package Reports, no. D4.4). Available from:

Descripció

PANACEA WP4 targets the creation of a Corpus Acquisition and Annotation (CAA) subsystem for the acquisition and processing of monolingual and bilingual language resources (LRs). The CAA subsystem consists of tools that have been integrated as web services in the PANACEA platform of LR production. D4.2 Initial functional prototype and documentation in T13 provided documentation on the initial functional prototype of this subsystem, while this deliverable presents updates in the revised subsystem during the second development cycle of the project. The deliverable is structured as follows. A revised version of the Focused Monolingual Crawler (FMC), that has been implemented according to the results of the first evaluation cycle and the reviewers’ comments in the first annual review report, is described in section 2. New and revised versions of tools for corpus normalization (cleaning and deduplication) are detailed in section 3. Section 4 provides documentation on the NLP tools introduced for the first time in the subsystem. These tools focus mainly on sentence splitting/tokenization and POS tagging/lemmatization for English (EN), French (FR), Spanish (ES), German (DE), Italian (IT) and Greek (EL).