Welcome to the UPF Digital Repository

Interpretable semantic textual similarity: Finding and explaining differences between sentences

Show simple item record

dc.contributor.author Lopez-Gazpio, Iñigo
dc.contributor.author Maritxalar, Montserrat
dc.contributor.author Gonzalez Agirre, Aitor
dc.contributor.author Rigau Claramunt, German
dc.contributor.author Uria, Larraitz
dc.contributor.author Agirre, Eneko
dc.date.accessioned 2017-11-21T11:38:56Z
dc.date.issued 2017
dc.identifier.citation Lopez-Gazpio I, Maritxalar M, Gonzalez-Agirre A, Rigau G, Uria L, Agirre E. Interpretable semantic textual similarity: Finding and explaining differences between sentences. Knowl-Based Syst. 2017;119: 189-99. DOI: 10.1016/j.knosys.2016.12.013
dc.identifier.issn 0950-7051
dc.identifier.uri http://hdl.handle.net/10230/33295
dc.description.abstract User acceptance of artificial intelligence agents might depend on their ability to explain their reasoning to the users. We focus on a specific text processing task, the Semantic Textual Similarity task (STS), where systems need to measure the degree of semantic equivalence between two sentences. We propose to add an interpretability layer (iSTS for short) formalized as the alignment between pairs of segments across the two sentences, where the relation between the segments is labeled with a relation type and a similarity score. This way, a system performing STS could use the interpretability layer to explain to users why it returned that specific score for the given sentence pair. We present a publicly available dataset of sentence pairs annotated following the formalization. We then develop an iSTS system trained on this dataset, which given a sentence pair finds what is similar and what is different, in the form of graded and typed segment alignments. When evaluated on the dataset, the system performs better than an informed baseline, showing that the dataset and task are well-defined and feasible. Most importantly, two user studies show how the iSTS system output can be used to automatically produce explanations in natural language. Users performed the two tasks better when having access to the explanations, providing preliminary evidence that our dataset and method to automatically produce explanations do help users understand the output of STS systems better.
dc.description.sponsorship Aitor Gonzalez-Agirre and Inigo Lopez-Gazpio are supported by doctoral grants from MINECO. The work described in this project has been partially funded by MINECO in projects MUSTER (PCIN-2015-226) and TUNER (TIN 2015-65308-C5-1-R), as well as the Basque Government (A group research team, IT344-10).
dc.format.mimetype application/pdf
dc.language.iso eng
dc.publisher Elsevier
dc.relation.ispartof Knowledge-Based Systems. 2017;119: 189-99.
dc.rights © Elsevier http://dx.doi.org/10.1016/j.knosys.2016.12.013
dc.title Interpretable semantic textual similarity: Finding and explaining differences between sentences
dc.type info:eu-repo/semantics/article
dc.identifier.doi http://dx.doi.org/10.1016/j.knosys.2016.12.013
dc.subject.keyword Interpretability
dc.subject.keyword Tutoring systems
dc.subject.keyword Semantic textual similarity
dc.subject.keyword Natural language understanding
dc.relation.projectID info:eu-repo/grantAgreement/ES/1PE/TIN 2015-65308-C5-1-R
dc.rights.accessRights info:eu-repo/semantics/openAccess
dc.type.version info:eu-repo/semantics/acceptedVersion
dc.embargo.liftdate 2018-12-12
dc.date.embargoEnd info:eu-repo/date/embargoEnd/2018-12-12

This item appears in the following Collection(s)

Show simple item record

Search DSpace

Advanced Search


My Account


Compliant to Partaking