e-Repositori UPF :: Treballs de recerca de màster

Enviaments recents

Empirical analysis of exploration strategies in QMIX
(2021-09) Colom, Arnau
In real world scenarios, to solve a gran majority of problems, there is the necessity for different agents to cooperate under the condition of local observations. Fortunately, in the recent years, significant advances in Multi-Agent Reinforcement Learning have been done regarding this matter. To tackle this kind of problems, a lot of approaches are based on the on Centralized Training with Decentralized Execution, which allow the agents to be trained in a simulated environment where they can have access to the global information to later solve the problem relying only on local observations. Some popular methods are Value-Decomposition Networks (VDN) and QMIX. They undertake the problem by computing the joint action-value function Qtot as a combination of the individual action-value functions Qa, that only condition on individual action-observation histories. Specifically, this work focuses on QMIX, which has been gaining a lot of popularity in the last year due to its capacity to compute a richer representation of the joint action-value function than VDN by combining the individual Q-values in a non-linear approach. However, despite the fact that there have been a lot of improvements on QMIX, there have been small advances on how different exploration techniques could boost the learning in this context. In this work, by performing an experimental evaluation, its shown how some exploration methods outperform the greedy approach used on the original implementation of QMIX on different cases.
Understanding Alzheimer’s disease progression through phenotypes discovery using manifold learning techniques
(2021-07) Pattarone, Natalia Karina
Alzheimer’s disease (AD) is clinically highly heterogeneous, varying in terms of rates of progression, test and cognitive symptoms among patients, as well as from a neuroimaging perspective. In the datasets provided by The Alzheimer’s Disease Neuroimaging Initiative (ADNI), researchers collect, validate and utilize data, including MRI and PET images, genetics, cognitive tests, CSF and blood biomarkers as predictors of the disease. Data coming from these datasets allow discovering phenotypes that could help to better understand the disease and provide targeted treatment. The objective of this thesis is to identify data-driven phenotypes using manifold learning and unsupervised clustering on multimodal longitudinal imaging and nonimaging data. First, we apply a novel approach for dimensionality reduction called PHATE that captures both local and global nonlinear structure using an informationgeometric distance between datapoints that would facilitate the discovery of possible AD phenotypes. Over PHATE output space, we performed a multiple-kernel unsupervised clustering to obtain profiles and describe AD phenotypes where features are weighted to construct kernels. Our results show that our approach can reveal AD progression trajectories in a lower dimensionality space, improving the results of the profiling where we obtained 4 possible profile subgroups using MRI cross-sectional baseline data and 8 possible profile subgroups when using longitudinal data. Furthermore, longitudinal data established clearer separation among profiles and higher significance for cognitive tests and general volumetric cerebral values than baseline data. Identifying these profiles could be useful for more personalized treatment of such a heterogeneous disease as AD.
Exploring neural paraphrasing to improve fluency of rule-based generation
(2021-09) Du, Shixiao
Data-to-text generation is a greatly significant task in the field of natural language processing. FORGe, a typical rule-based generator, has excellent performance in the task of mapping from RDF triples to text. As this generator is strongly reliant on rules, despite the fact that the generated text is with highly semantic accuracy, and faithful to the input RDF triples, the generated sentence could be somewhat rigid in terms of fluency. This thesis would like to explore a possible way to improve the fluency of output text generated by FORGe. A neural paraphrase method is suggested to act as a post-processing method to achieve our goal. This method can control the tradeoff between two models, the fluency and semantic similarity model, and the lexical and/or syntactic diversity model by setting a parameter . In this way, it can not only make the output semantically consistent with the input, but also diversify the lexical and syntactic items of the sentence. In order to verify our idea, we designed and conducted related experiments. Furthermore, the performance of deep-learning based generator OSU Neural NLG, which also performs well in English D2T tasks, is considered as a baseline. Since we need to evaluate all the generated text, an automatic evaluation method is used in order to ensure an uniform evaluation criterion on these outputs in terms of semantic accuracy. Based on the result of this automatic evaluation method, we also conducted manual verification to make the evaluation result more reliable and have reference value. Through our experimental results, we believe that applying this neural paraphrase method as a post-processing stage is promising in improving the fluency of the output text generated by FORGe.
Chatbots as educational assistants: teaching about the digital footprint
(2021-07) Mancheno Gutiérrez, María Fernanda
The digital footprint is a trace of everything a user does online. Everyone has one, but not everyone is aware of how to remain safe when being online. Every action a user takes when navigating through the Web or social media matters. Unlike other methods of communication, the Internet holds a record of each user’s activity, the key part being that this record will be held permanently. Therefore, each user should have the responsibility of making sure his/her digital footprint is protected. As the world becomes more technologically advanced, so should teaching methods. Young adults are fast learners and sometimes learning techniques do not adequately fit in with the teaching methods still being used in classrooms. Students want immediate and concise help, which is not something they can always find in other humans. Several studies show how chatbots have proven to be efficient teaching methods as they not only consider the technological aspect, but also emotional aspects. They also have one major advantage over humans, chatbots have infinite patience and they do not ever get tired. Therefore, this Master thesis answers the following question: to what extent can a chatbot prove to be efficient enough to teach young adults about their social media digital footprint? This Master thesis seeks, through the creation of an intelligent chatbot that takes in young adults’ responses, to provide knowledge about the digital footprint and the risks of being a social media user. The process behind this work includes three main stages: developing the chatbot, testing the chatbot, and evaluating the chatbot. The chatbot uses NLP (Natural Language Processing) techniques and Chatterbot (Python package) to become intelligent enough to recognize users’ answers. Six Pompeu Fabra University bachelor students participate in this chatbot interaction to provide feedback and a proper evaluation about their experience and knowledge gained. Finally, based on this feedback, the Master thesis shows that this teaching method is successful in teaching young users about their digital footprint.
Reinforcement learning with options in semi Markov decision processes
(2021-09) Goswami, Sayan
The options framework incorporates temporally extended actions (termed options) to the reinforcement learning paradigm. A wide variety of prior works exist that experimentally illustrate the significance of options on the performance of a learning algorithm in a complex domains. However, the work by Fruit et al. on the semi-Markov Decision Process (SMDP) version of the UCRL2 algorithm introduced a formal understanding of circumstance that make options conducive to the performance of a learning algorithm. In this work we present our implementation of the algorithm proposed by Fruit et al. We perform experimentation on a navigation task characterized by a grid world domain. We achieve a sub-linear trend in accumulated regret as well as a linear trend in accumulated reward in the grid world domain using empirical Bernstein peeling as confidence bound.
How are decisions made in decidim.barcelona?
(2018-07) Solà, Jaume
Decidim Barcelona is the citizen engagement platform that Barcelona’s city hall put in motion on February 2016. One of its first processes has been the development of the "Plà d’Actuació Municipal" (PAM) which has led to the creation of 10; 860 proposals that eventually resulted in 1; 467 actions included in the plan. In this project we build a statistical model with the aim to predict whether an individual proposal would be accepted or not. The objectives of this work are twofold: first, we want to gain understanding in the processes that took place during the development of the PAM and second, we want to test to what extent this type of statistical modeling can capture the decision making process in order to better aid future deliberative processes in the platform. We first analyze the data generated by citizens in the city of Barcelona that participated actively on the platform. After a preliminary statistical analysis of the features that characterize each proposal, we proceed to build a model that is able to predict if a proposal would be accepted or not from that data. We consider the logistic regression model because its computational simplicity as well as its potential interpretability. We are be able to extract conclusions from the parameters and unveil the decision process which resulted the acceptance/rejection of each proposal in the platform. We show that such a model is able to characterize some particularities of the process, but also how this classifier compares to other methods like random forests, and what do the differences we find between them mean.
Aiding the platform of decidim.barcelona: clustering of proposals
(2018-09) González, Esther
Decidim Barcelona is an online participatory-democracy platform that boosts citizens participation to draw The Municipality Plan. A key step to build this plan is the selection of the proposals that will be included. This requires a manual selection and grouping of proposals that intend to tackle the same issue. The use of machine learning can speed up this process as well as improve user experience suggesting similar proposals, also decreasing the number of duplication. We have experimented several techniques to discover which one better solves the problem of automatically grouping similar proposals.

Navegar

Examinar

Enviaments recents