Welcome to the UPF Digital Repository

Multimodal visual data registration for web-based visualization in media production

Show simple item record

dc.contributor.author Kim, Hansung
dc.contributor.author Evans, Alun
dc.contributor.author Blat, Josep
dc.contributor.author Hilton, Adrian
dc.date.accessioned 2024-02-07T14:21:53Z
dc.date.available 2024-02-07T14:21:53Z
dc.date.issued 2018
dc.identifier.citation Kim H, Evans A, Blat J, Hilton A. Multimodal visual data registration for web-based visualization in media production. IEEE Trans Circuits Syst Video Technol. 2018;28(4):863-77. DOI: 10.1109/TCSVT.2016.2642825
dc.identifier.issn 1051-8215
dc.identifier.uri http://hdl.handle.net/10230/58987
dc.description.abstract Recent developments of video and sensing technology have led to large volumes of digital media data. Current media production relies on videos from the principal camera together with a wide variety of heterogeneous source of supporting data [photos, light detection and ranging point clouds, witness video camera, high dynamic range imaging, and depth imagery]. Registration of visual data acquired from various 2D and 3D sensing modalities is challenging because current matching and registration methods are not appropriate due to differences in structure, format, and noise characteristics for multimodal data. A combined 2D/3D visualization of this registered data allows an integrated overview of the entire data set. For such a visualization, a Web-based context presents several advantages. In this paper, we propose a unified framework for registration and visualization of this type of visual media data. A new feature description and matching method is proposed, adaptively considering local geometry, semiglobal geometry, and color information in the scene for more robust registration. The resulting registered 2D/3D multimodal visual data are too large to be downloaded and viewed directly via the Web browser, while maintaining an acceptable user experience. Thus, we employ hierarchical techniques for compression and restructuring to enable efficient transmission and visualization over the Web, leading to interactive visualization as registered point clouds, 2D images, and videos in the browser, improving on the current state-of-the-art techniques for Web-based visualization of big media data. This is the first unified 3D Web-based visualization of multimodal visual media production data sets. The proposed pipeline is tested on big multimodal data set typical of film and broadcast production, which are made publicly available. The proposed feature description method shows two times higher precision of feature matching and more stable registration performance than existing 3D feature descriptors.
dc.format.mimetype application/pdf
dc.language.iso eng
dc.publisher Institute of Electrical and Electronics Engineers (IEEE)
dc.relation.ispartof IEEE Trans Circuits Syst Video Technol. 2018;28(4):863-77.
dc.rights © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. http://dx.doi.org/10.1109/TCSVT.2016.2642825
dc.title Multimodal visual data registration for web-based visualization in media production
dc.type info:eu-repo/semantics/article
dc.identifier.doi http://dx.doi.org/10.1109/TCSVT.2016.2642825
dc.subject.keyword 2D–3D registration
dc.subject.keyword 3D feature descriptors
dc.subject.keyword 3D feature matching
dc.subject.keyword Multimodal visual data processing
dc.subject.keyword Progressive rendering
dc.subject.keyword WebGL visualization
dc.rights.accessRights info:eu-repo/semantics/openAccess
dc.type.version info:eu-repo/semantics/acceptedVersion


This item appears in the following Collection(s)

Show simple item record

Search DSpace

Advanced Search


My Account


In collaboration with Compliant to Partaking