Description logics and rules for multimodal situational awareness in healthcare

Meditskos, Georgios; Vrochidis, Stefanos; Kompatsiaris, Ioannis

doi:http://dx.doi.org/10.1007/978-3-319-51811-4_58

Description logics and rules for multimodal situational awareness in healthcare

Citation

Meditskos G, Vrochidis S, Kompatsiaris I. Description logics and rules for multimodal situational awareness in healthcare. In: Amsaleg L, Guðmundsson G, Gurrin C, Jónsson B, Satoh S, editors. Proceedings of Video Browser Showdown (VBS'17) at the 23rd International Conference on MultiMedia Modeling (MMM'17); 2017 4 Jan; Reykjavik, Iceland, Part II. [place unknonwn]: Springer, 2017. (LNCS, no. 10133). p. 714–25. DOI: 10.1007/978-3-319-51811-4_58

Abstract

We present a framework for semantic situation understand- ing and interpretation of multimodal data using Description Logics (DL) and rules. More precisely, we use DL models to formally describe contex- tualised dependencies among verbal and non-verbal descriptors in multi- modal natural language interfaces, while context aggregation, fusion and interpretation is supported by SPARQL rules. Both background knowl- edge and multimodal data, e.g. language analysis results, facial expres- sions and gestures recognized from multimedia streams, are captured in terms of OWL 2 ontology axioms, the de facto standard formalism of DL models on the Web, fostering reusability, adaptability and interoperabil- ity of the framework. The framework has been applied in the eminent eld of healthcare, providing the models for the semantic enrichment and fusion of verbal and non-verbal descriptors in dialogue-based systems.

Description

Comunicació presentada a: Video Browser Showdown (VBS'17) dins 23rd International Conference on MultiMedia Modeling (MMM'17); celebrats el 4 de gener de 2017 a Reykjavik, Islàndia.