Comparison of vision transformers and convolution neural networks

Abstract

This thesis explores the differences between Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) to understand how these architectures perceive and learn from images. We first provide an in-depth explanation and comparative literature review of CNNs and ViTs. We then investigate how both models adapt to classifying satellite images and rotated scene images, evaluated in terms of rotational invariance and learned representations using Centered Kernel Alignment (CKA). ViTs demonstrated better performance and stability, which we attribute to their ability to integrate global information through self-attention mechanisms, while CNNs showed more variation due to their hierarchical feature learning and local receptive fields.Esta tesis compara Convolutional Neural Networks (CNNs) y Vision Transformers (ViTs) en la percepción y aprendizaje de imágenes. Se ofrece una revisión bibliográfica comparativa y se investiga su adaptación a la clasificación de imágenes de satélite y escenas rotadas, evaluadas en términos de invarianza rotacional y representaciones aprendidas con Centered Kernel Alignment (CKA). Los ViT mostraron mejor rendimiento y estabilidad, atribuida a su capacidad para integrar información global mediante autoatención. Las CNN mostraron más variación debido a su aprendizaje jerárquico de características y campos receptivos locales.