On the design of consequential ranking algorithms

Tabibian, Behzad; Gómez, Vicenç; De, Abir; Schölkopf, Bernhard; Gomez-Rodriguez, Manuel

On the design of consequential ranking algorithms

License

Citation

Tabibian B, Gomez V, De A, Schölkopf B, Gomez Rodriguez M. On the design of consequential ranking algorithms. In: Peters J, Sontag D, editors. Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI); 2020 Aug 3-6; Massachusetts, USA. Massachusetts: PMLR; 2020. p. 171-80.

Abstract

Ranking models are typically designed to optimize some measure of immediate utility to the users. As a result, they have been unable to anticipate an increasing number of undesirable long-term consequences of their proposed rankings, from fueling the spread of misinformation and increasing polarization to degrading social discourse. Can we design ranking models that anticipate the consequences of their proposed rankings and are able to avoid the undesirable ones? In this paper, we first introduce a joint representation of rankings and user dynamics using Markov decision processes. Then, we show that this representation greatly simplifies the construction of consequential ranking models that trade off the immediate utility and the long-term welfare. In particular, we can obtain optimal consequential rankings by applying weighted sampling on the rankings provided by models that maximize measures of immediate utility. However, in practice, such a strategy may be inefficient and impractical, specially in high dimensional scenarios. To overcome this, we introduce an efficient gradient-based algorithm to learn parameterized consequential ranking models that effectively approximate optimal ones. We illustrate our methodology using synthetic and real data gathered from Reddit and show that our consequential rankings may mitigate the spread of misinformation and improve the civility of online discussions.