Welcome to the UPF Digital Repository

Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression

Show simple item record

dc.contributor.author Gialampoukidis, Ilias
dc.contributor.author Moumtzidou, Anastasia
dc.contributor.author Liparas, Dimitris
dc.contributor.author Tsikrika, Theodora
dc.contributor.author Vrochidis, Stefanos
dc.contributor.author Kompatsiaris, Ioannis
dc.date.accessioned 2017-09-08T15:15:30Z
dc.date.issued 2017
dc.identifier.citation Gialampoukidis I, Moumtzidou A, Liparas D, Tsikrika T, Vrochidis S, Kompatsiaris I. Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression. Multimed Tools Appl. 2017;76(21):22383-403. DOI: 10.1007/s11042-017-4797-4
dc.identifier.issn 1380-7501
dc.identifier.uri http://hdl.handle.net/10230/32760
dc.description.abstract Heterogeneous sources of information, such as images, videos, text and metadata are often used to describe di erent or complementary views of the same multimedia object, especially in the online news domain and in large annotated image collections. The retrieval of multimedia objects, given a mul- timodal query, requires the combination of several sources of information in an e cient and scalable way. Towards this direction, we provide a novel unsuper- vised framework for multimodal fusion of visual and textual similarities, which are based on visual features, visual concepts and textual metadata, integrating non-linear graph-based fusion and Partial Least Squares Regression. The fu- sion strategy is based on the construction of a multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. Our framework can employ more than two modalities and high-level information, without increase in memory complexity, when com- pared to state-of-the-art baseline methods. The experimental comparison is done in three public multimedia collections in the multimedia retrieval task. The results have shown that the proposed method outperforms the baseline methods, in terms of Mean Average Precision and Precision@20.
dc.description.sponsorship This work was partially supported by the European Commission by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012).
dc.format.mimetype application/pdf
dc.language.iso eng
dc.publisher Springer
dc.relation.ispartof Multimed Tools Appl. 2017;76(21):22383-403
dc.rights © Springer The final publication is available at Springer via https://link.springer.com/article/10.1007/s11042-017-4797-4
dc.title Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression
dc.type info:eu-repo/semantics/article
dc.identifier.doi http://dx.doi.org/10.1007/s11042-017-4797-4
dc.subject.keyword Multimedia retrieval
dc.subject.keyword Non-linear fusion
dc.subject.keyword Graph-based models
dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/645012
dc.relation.projectID info:eu-repo/grantAgreement/EC/FP7/610411
dc.rights.accessRights info:eu-repo/semantics/openAccess
dc.type.version info:eu-repo/semantics/acceptedVersion


This item appears in the following Collection(s)

Show simple item record

Search DSpace

Advanced Search


My Account


Compliant to Partaking