Scene understanding from image and video : segmentation, depth configuration

Oliver Parera, Maria

Scene understanding from image and video : segmentation, depth configuration

Document Type

Doctoral thesis
Published Version

Author

Oliver Parera, Maria

Director

Ballester, Coloma
Haro Ortega, Gloria

Publication Date

2018-10-17

Pages

170 p.

License

Citation

Oliver Parera, M. Scene understanding from image and video : segmentation, depth configuration. Universitat Pompeu Fabra; 2018. handle: http://hdl.handle.net/10803/663870

This citation was generated automatically.

Doctoral program

Programa de doctorat en Tecnologies de la Informació i les Comunicacions

Abstract

In this thesis we aim at analyzing images and videos at the object level, with the goal of decomposing the scene into complete objects that move and interact among themselves. The thesis is divided in three parts. First, we propose a segmentation method to decompose the scene into shapes. Then, we propose a probabilistic method, which works with shapes or objects at two diﬀerent depths, to infer which objects are in front of the others, while completing the ones which are partially occluded. Finally, we propose two video related inpainting method. On one hand, we propose a binary video inpainting method that relies on the optical ﬂow of the video in order to complete the shapes across time taking into account their motion. On the other hand, we propose a method for optical ﬂow that takes into account the informational from the frames.
Aquesta tesi té per objectiu analitzar imatges i vídeos a nivell d’objectes, amb l’objectiu de descompondre l’escena en objectes complets que es mouen i interaccionen entre ells. La tesi està dividida en tres parts. En primer lloc, proposem un mètode de segmentació per descompondre l’escena en les formes que la componen. A continuació, proposem un mètode probabilístic, que considera les formes o objectes en dues profunditats de l’escena diferents, i infereix quins objectes estan davant dels altres, completant també els objectes parcialment ocults. Finalment, proposem dos mètodes relacionats amb el vídeo inpainting. Per una banda, proposem un mètode per vídeo inpainting binari que utilitza el ﬂux òptic del vídeo per completar les formes al llarg del temps, tenint en compte el seu moviment. Per l’altra banda, proposem un mètode per inpainting de ﬂux òptic que té en compte la informació provinent dels frames.

Keywords

Visual completion, Bayesian model, Relatability, Elastica, Shape Completion, Threshold Dynamics, Segmentation, Variational methods, L1-norm, Aﬃne invariant patch similarity, Inpainting, PDE, AMLE, Anisotroy