Grounding semantic roles in images

Mostra el registre complet Registre parcial de l'ítem

  • dc.contributor.author Silberer, Carina
  • dc.contributor.author Pinkal, Manfred
  • dc.date.accessioned 2018-12-18T12:21:17Z
  • dc.date.available 2018-12-18T12:21:17Z
  • dc.date.issued 2018
  • dc.description Comunicació presentada a la 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), celebrada del 31 d'octubre al 4 de novembre de 2018 a Brussel·les, Bèlgica.
  • dc.description.abstract We address the task of visual semantic role labeling (vSRL), the identification of the participants of a situation or event in a visual scene, and their labeling with their semantic relations to the event or situation. We render candidate participants as image regions of objects, and train a model which learns to ground roles in the regions which depict the corresponding participant. Experimental results demonstrate that we can train a vSRL model without reliance on prohibitive image-based role annotations, by utilizing noisy data which we extract automatically from image captions using a linguistic SRL system. Furthermore, our model induces frame–semantic visual representations, and their comparison to previous work on supervised visual verb sense disambiguation yields overall better results.
  • dc.description.sponsorship We thank the anonymous reviewers, Leonie Harter, Christine Sch¨afer, Michael Roth, Gemma Boleda, Anna Rohrbach and Bernt Schiele. This research was supported by the German Research Foundation (DFG EXC 285), by the European Research Council (ERC Horizon 2020 grant agreement No 715154), and the Spanish Ramon y Cajal programme (grant RYC-2015-18907).
  • dc.format.mimetype application/pdf
  • dc.identifier.citation Silberer C, Pinkal M. Grounding semantic roles in images. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing; 2018 Oct 31 - Nov 4; Brussels, Belgium. Stroudsburg (PA): Association for Computational Linguistics; 2018. p. 2616–26.
  • dc.identifier.uri http://hdl.handle.net/10230/36137
  • dc.language.iso eng
  • dc.publisher ACL (Association for Computational Linguistics)
  • dc.relation.ispartof Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing; 2018 Oct 31 - Nov 4; Brussels, Belgium. Stroudsburg (PA): Association for Computational Linguistics; 2018. p. 2616–26.
  • dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/715154
  • dc.rights © ACL, Creative Commons Attribution 4.0 License
  • dc.rights.accessRights info:eu-repo/semantics/openAccess
  • dc.rights.uri http://creativecommons.org/licenses/by/4.0/
  • dc.title Grounding semantic roles in images
  • dc.type info:eu-repo/semantics/conferenceObject
  • dc.type.version info:eu-repo/semantics/publishedVersion

Col·leccions

Congressos (Departament de Traducció i Ciències del Llenguatge)
Documents OpenAIRE (Open Access Infrastructure for Research in Europe)