Emojis are small images that are commonly
included in social media text messages. The
combination of visual and textual content in
the same message builds up a modern way
of communication, that automatic systems are
not used to deal with. In this paper we extend
recent advances in emoji prediction by putting
forward a multimodal approach that is able to
predict emojis in Instagram posts. Instagram
posts are composed of pictures together with
texts which sometimes include emojis. We
show ...
Emojis are small images that are commonly
included in social media text messages. The
combination of visual and textual content in
the same message builds up a modern way
of communication, that automatic systems are
not used to deal with. In this paper we extend
recent advances in emoji prediction by putting
forward a multimodal approach that is able to
predict emojis in Instagram posts. Instagram
posts are composed of pictures together with
texts which sometimes include emojis. We
show that these emojis can be predicted by using
the text, but also using the picture. Our
main finding is that incorporating the two synergistic
modalities, in a combined model, improves
accuracy in an emoji prediction task.
This result demonstrates that these two modalities
(text and images) encode different information
on the use of emojis and therefore can
complement each other.
+