Linguistic generalization and compositionality in modern artificial neural networks.

Citació

  • Baroni M. Linguistic generalization and compositionality in modern artificial neural networks. Philos Trans R Soc Lond B Biol Sci. 2020 Feb 3; 375(1791). DOI: 10.1098/rstb.2019.0307

Enllaç permanent

Descripció

  • Resum

    In the last decade, deep artificial neural networks have achieved astounding performance in many natural language processing tasks. Given the high productivity of language, these models must possess e ective generalization abilities. It is widely assumed that humans handle linguistic productivity by means of algebraic compositional rules: Are deep networks similarly compositional? After reviewing the main innovations characterizing current deep language processing networks, I discuss a set of studies suggesting that deep networks are capable of subtle grammar-dependent generalizations, but also that they do not rely on systematic compositional rules. I argue that the intriguing behaviour of these devices (still awaiting a full understanding) should be of interest to linguists and cognitive scientists, as it o ers a new perspective on possible computational strategies to deal with linguistic productivity beyond rule-based compositionality, and it might lead to new insights into the less systematic generalization patterns that also appear in natural language.
  • Mostra el registre complet