EVALITA: evaluation of NLP and Speech Tools for Italian

Abstract

EVALITA is the evaluation campaign of Natural Language Processing and Speech Tools for the Italian language: since 2007 shared tasks have been proposed covering the analysis of both written and spoken language with the aim of enhancing the development and dissemination of resources and technologies for Italian. EVALITA is an initiative of the Italian Association for Computational Linguistics (AILC, http://www.ai-lc.it/) and it is supported by the NLP Special Interest Group of the Italian Association for Artificial Intelligence (AI*IA, http://www.aixia.it/) and by the Italian Association of Speech Science (AISV, http://www.aisv.it/). In this volume, we collect the reports of the tasks’ organisers and of the participants to all of the EVALITA 2016’s tasks, which are the following: ArtiPhone - Articulatory Phone Recognition; FactA - Event Factuality Annotation; NEEL-IT - Named Entity rEcognition and Linking in Italian Tweets; PoSTWITA - POS tagging for Italian Social Media Texts; QA4FAQ - Question Answering for Frequently Asked Questions; SENTIPOLC - SENTIment POLarity Classification. Notice that the volume does not include reports related to the IBM Watson Services Challenge organised by IBM Italy, but information can be found at http://www.evalita.it/2016/tasks/ibm-challenge. Before the task and participant reports, we also include an overview to the campaign that describes the tasks in more detail, provides figures on the participants, and, especially, highlights the innovations introduced at this year’s edition. An additional report presents a reflection on the outcome of two questionnaires filled by past participants and organisers of EVALITA, and of the panel “Raising Interest and Collecting Suggestions on the EVALITA Evaluation Campaign” held at CLIC-it 2015.