Welcome to the UPF Digital Repository

A hybrid framework for news clustering based on the DBSCAN-Martingale and LDA

Show simple item record

dc.contributor.author Gialampoukidis, Ilias
dc.contributor.author Vrochidis, Stefanos
dc.contributor.author Kompatsiaris, Ioannis
dc.contributor.author Wanner, Leo
dc.date.accessioned 2017-01-31T17:22:58Z
dc.date.issued 2016
dc.identifier.citation Gialampoukidis I, Vrochidis S, Kompatsiaris I. A hybrid framework for news clustering based on the DBSCAN-Martingale and LDA. In: Pemer P, editor. Machine Learning and Data Mining in Pattern Recognition. 12th International Conference, MLDM 2016; 2016 July 16-21; New York (NY, USA). [place unknown]: Springer; 2016. p. 170-84. DOI: 10.1007/978-3-319-41920-6_13
dc.identifier.uri http://hdl.handle.net/10230/28023
dc.description.abstract Nowadays there is an important need by journalists and media monitoring companies to cluster news in large amounts of web articles, in order to ensure fast access to their topics or events of interest. Our aim in this work is to identify groups of news articles that share a common topic or event, without a priori knowledge of the number of clusters. The estimation of the correct number of topics is a challenging issue, due to the existence of “noise”, i.e. news articles which are irrelevant to all other topics. In this context, we introduce a novel density-based news clustering framework, in which the assignment of news articles to topics is done by the well-established Latent Dirichlet Allocation, but the estimation of the number of clusters is performed by our novel method, called “DBSCAN-Martingale”, which allows for extracting noise from the dataset and progressively extracts clusters from an OPTICS reachability plot. We evaluate our framework and the DBSCAN-Martingale on the 20newsgroups-mini dataset and on 220 web news articles, which are references to specific Wikipedia pages. Among twenty methods for news clustering, without knowing the number of clusters k, the framework of DBSCAN-Martingale provides the correct number of clusters and the highest Normalized Mutual Information.
dc.description.sponsorship This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.
dc.format.mimetype application/pdf
dc.language.iso eng
dc.publisher Springer
dc.relation.ispartof Pemer P, editor. Machine Learning and Data Mining in Pattern Recognition. 12th International Conference, MLDM 2016; 2016 July 16-21; New York (NY, USA). [place unknown]: Springer; 2016. p. 170-84.
dc.rights © Springer The final publication is available at Springer via/nhttp://dx.doi.org/10.1007/978-3-319-41920-6_13.
dc.title A hybrid framework for news clustering based on the DBSCAN-Martingale and LDA
dc.type info:eu-repo/semantics/conferenceObject
dc.identifier.doi http://dx.doi.org/10.1007/978-3-319-41920-6_13
dc.subject.keyword Clustering news articles
dc.subject.keyword Latent Dirichlet Allocation
dc.subject.keyword DBSCAN-Martingale
dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/645012
dc.relation.projectID info:eu-repo/grantAgreement/EC/FP7/610411
dc.rights.accessRights info:eu-repo/semantics/openAccess
dc.type.version info:eu-repo/semantics/acceptedVersion

Thumbnail

This item appears in the following Collection(s)

Show simple item record

Search DSpace


Advanced Search

Browse

My Account

Statistics

Compliant to Partaking