As large pre-trained image-processing neural networks are being
embedded in autonomous agents such as self-driving cars or robots,
the question arises of how such systems can communicate with
each other about the surrounding world, despite their different
architectures and training regimes. As a first step in this direction,
we explore the task of referential communication in a community of
state-of-the-art pre-trained visual networks, showing that they can
develop a shared protocol to refer ...
As large pre-trained image-processing neural networks are being
embedded in autonomous agents such as self-driving cars or robots,
the question arises of how such systems can communicate with
each other about the surrounding world, despite their different
architectures and training regimes. As a first step in this direction,
we explore the task of referential communication in a community of
state-of-the-art pre-trained visual networks, showing that they can
develop a shared protocol to refer to a target image among a set
of candidates. Such shared protocol, induced in a self-supervised
way, can to some extent be used to communicate about previously
unseen object categories. Finally, we show that a new neural network can learn the shared protocol developed in a community with
remarkable ease, and the process of integrating a new agent into
a community more stably succeeds when the original community
includes a larger set of heterogeneous networks.
+
