Analysis of the exploration-exploitation trade-off in the Koa Foundations app usage

Abstract

Decision-making is a fundamental human behavior. In our day-to-day lives, we are constantly coming across the problem of choosing among multiple options, which we aim to solve by maximizing the outcomes. To achieve this, the process of making a decision involves the selection of a course of action among all the possibilities in order to stumble upon the best alternative, which becomes especially critical when the decision-maker’s resources are limited. In such cases, the usage of optimal strategies for evaluating the options becomes key for the agent to succeed with the task. The decision of the path to

take is usually not straightforward, and involves dilemmas such as the exploration- exploitation trade-off, in which the decision-maker agent is faced with deciding whether

to sample new options to gain new knowledge (exploration) or instead choose options already sampled and known to perform well (exploitation). A specific case of study in which such aforementioned tasks take place is in the users’ utilization of the Foundations application. This mental-health mobile app offers a wide catalog of activities, grouped into modules, and does not initially provide users with any explicit guidance or recommendations. In this work, data generated by users using the app has been analyzed by tackling the exploration-exploitation strategy framework, ultimately aiming at relating it to users' engagement. This is done under the premise that they aspire to find the activities and modules that work best for them. To that end, in the beginning, they are expected to tend to explore the app more, but then, as their time on the app goes by, a transition towards any form of exploitation conduct might appear. As part of the objectives, the identification of behavioral features that may correlate to the users’ engagement with Foundations has been sought.