CNNs found to jump around more skillfully than RNNs: compositional generalization in seq2seq convolutional networks

Dessi, Roberto; Baroni, Marco

doi:https://doi.org/10.18653/v1/P19-1381

CNNs found to jump around more skillfully than RNNs: compositional generalization in seq2seq convolutional networks

Llicència

Citació

Dessi R, Baroni M. CNNs found to jump around more skillfully than RNNs: compositional generalization in seq2seq convolutional networks. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019; 2019 Jul 28 - Aug 2; Florence, Italy. Stroudsburg: Association for Computational Linguistics; 2019. p. 3919-23.

Resum

Lake and Baroni (2018) introduced the SCAN dataset probing the ability of seq2seq models to capture compositional generalizations, such as inferring the meaning of “jump around” 0-shot from the component words. Recurrent networks (RNNs) were found to completely fail the most challenging generalization cases. We test here a convolutional network (CNN) on these tasks, reporting hugely improved performance with respect to RNNs. Despite the big improvement, the CNN has however not induced systematic rules, suggesting that the difference between compositional and noncompositional behaviour is not clear-cut.

Descripció

Comunicació presentada a la 57th Annual Meeting of the Association for Computational Linguistics, celebrada del 28 de juliol al 2 d'agost de 2019 a Florència, Itàlia