A multimodal analytics platform for journalists analyzing large-scale, heterogeneous multilingual, and multimedia content

Vrochidis, Stefanos; Moumtzidou, Anastasia; Gialampoukidis, Ilias; Liparas, Dimitris; Casamayor, Gerard; Wanner, Leo; Heise, Nicolaus; Wagner, Tilman; Bilous, Andriy; Jamin, Emmanuel; Simeonov, Boyan; Alexiev, Vladimir; Busch, Reinhard; Arapakis, Ioannis; Kompatsiaris, Ioannis

doi:http://dx.doi.org/10.3389/frobt.2018.00123

A multimodal analytics platform for journalists analyzing large-scale, heterogeneous multilingual, and multimedia content

License

Citation

Vrochidis S, Moumtzidou A, Gialampoukidis I, Liparas D, Casamayor G, Wanner L, Heise N, Wagner T, Bilous A, Jamin E, Simeonov B, Alexiev V, Busch R, Arapakis I, Kompatsiaris I. A multimodal analytics platform for journalists analyzing large-scale, heterogeneous multilingual, and multimedia content. Front Robot AI. 2018;5:123. DOI: 10.3389/frobt.2018.00123

Abstract

Analysts and journalists face the problem of having to deal with very large, heterogeneous, and multilingual data volumes that need to be analyzed, understood, and aggregated. Automated and simplified editorial and authoring process could significantly reduce time, labor, and costs. Therefore, there is a need for unified access to multilingual and multicultural news story material, beyond the level of a nation, ensuring context-aware, spatiotemporal, and semantic interpretation, correlating also and summarizing the interpreted material into a coherent gist. In this paper, we present a platform integrating multimodal analytics techniques, which are able to support journalists in handling large streams of real-time and diverse information. Specifically, the platform automatically crawls and indexes multilingual and multimedia information from heterogeneous resources. Textual information is automatically summarized and can be translated (on demand) into the language of the journalist. High-level information is extracted from both textual and multimedia content for fast inspection using concept clouds. The textual and multimedia content is semantically integrated and indexed using a common representation, to be accessible through a web-based search engine. The evaluation of the proposed platform was performed by several groups of journalists revealing satisfaction from the user side.