Welcome to the UPF Digital Repository

Detection of disaster-affected cultural heritage sites from social media images using deep learning techniques

Show simple item record

dc.contributor.author Kumar, Pakhee
dc.contributor.author Ofli, Ferda
dc.contributor.author Imran, Muhammad
dc.contributor.author Castillo, Carlos
dc.date.accessioned 2021-02-05T07:17:07Z
dc.date.available 2021-02-05T07:17:07Z
dc.date.issued 2020
dc.identifier.citation Kumar P, Ofli F, Imran M, Castillo C. Detection of disaster-affected cultural heritage sites from social media images using deep learning techniques. J Comput Cult Herit. 2020 Aug;13(3):23. DOI: 10.1145/3383314
dc.identifier.issn 1556-4673
dc.identifier.uri http://hdl.handle.net/10230/46354
dc.description.abstract This article describes a method for early detection of disaster-related damage to cultural heritage. It is based on data from social media, a timely and large-scale data source that is nevertheless quite noisy. First, we collect images posted on social media that may refer to a cultural heritage site. Then, we automatically categorize these images according to two dimensions: whether they are indeed a photo in which a cultural heritage resource is the main subject, and whether they represent damage. Both categorizations are challenging image classification tasks, given the ambiguity of these visual categories; we tackle both tasks using a convolutional neural network. We test our methodology on a large collection of thousands of images from the web and social media, which exhibit the diversity and noise that is typical of these sources, and contain buildings and other architectural elements, heritage and not-heritage, damaged by disasters as well as intact. Our results show that while the automatic classification is not perfect, it can greatly reduce the manual effort required to find photos of damaged cultural heritage by accurately detecting relevant candidates to be examined by a cultural heritage professional.
dc.description.sponsorship C. Castillo is partially funded by La Caixa project LCF/PR/PR16/11110009.
dc.format.mimetype application/pdf
dc.language.iso eng
dc.publisher ACM Association for Computer Machinery
dc.relation.ispartof Journal on Computing and Cultural Heritage. 2020 Aug;13(3):23
dc.rights © ACM, 2020. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in J Comput Cult Herit, 2020 Aug;13(3):23 http://doi.acm.org/10.1145/3383314
dc.title Detection of disaster-affected cultural heritage sites from social media images using deep learning techniques
dc.type info:eu-repo/semantics/article
dc.identifier.doi http://dx.doi.org/10.1145/3383314
dc.subject.keyword Cultural heritage sites
dc.subject.keyword Social media
dc.subject.keyword Damage assessment
dc.subject.keyword Deep learning
dc.rights.accessRights info:eu-repo/semantics/openAccess
dc.type.version info:eu-repo/semantics/acceptedVersion

Thumbnail

This item appears in the following Collection(s)

Show simple item record

Search DSpace


Advanced Search

Browse

My Account

Statistics

In collaboration with Compliant to Partaking