e-Repositori UPF :: Informes (Departament de Tecnologies de la Informació i les Comunicacions)

Recent Submissions

PBL4SDGs guide: designing projects to develop the key competencies for sustainability
(Universitat Pompeu Fabra, 2024) Carrió, Mar; Gay, Marina; Hernández-Leo, Davinia; Kun Masvidal, Mireia; Lope Pastor, Silvia
The aim of this document is to help educational centres to create and specify useful tools and materials for designing, implementing and assessing activities that promote the development of the competencies for sustainability proposed by UNESCO, on the basis of project-based learning (PBL) methodology. This material is aimed at schools and teachers of different educational stages interested in creating classroom activities or projects that deal with topics related to global justice, the climate emergency, planetary welfare or sustainable development with a clear competency-based approach. It can be useful both for teachers who want to start working on these project types and those already working on them, yet who wish to introduce a focus on the key competencies for sustainability.
Digital vs. paper-based implementation of the pyramid collaborative learning flow pattern
(2024-08-28) Hernández Leo, Davinia; Pérez-Basso, Francisco; Beltrán-Beltrán, Belén
PyramidApp is designed to support collaborative learning scripting structured according to the Pyramid (a.k.a. Snowball) Collaborative Learning Flow Pattern (CLFP). Available as a web platform for computer or mobile devices, PyramidApp supports the generation of self-explanations to be integrated in collaborative explanations in increasingly larger groups. A physical version mimicking the PyramidApp implementation is also available in a poster format. This study compares the affordances of both versions of PyramidApp in workshops with non-university teachers in Catalonia. 77 participants delivered feedback on the two tool formats through a questionnaire. Overall, the results show that the teachers highlight the importance of scalability and dynamism in collaborative learning technologies. Teachers state that the digital version offers distinct advantages such as fostered active participation, scalability, dynamism, and real-time monitoring via the PyramidApp teacher-facing dashboard. Despite these benefits, primary education teachers often say to prefer the physical format due to its ease of use and visual appeal. The study highlights the potential of the digital version in facilitating large group activities, saving time, and providing greater flexibility. The findings suggest that both versions are valuable, each suited to the needs of different educational contexts.
Encouraging collaborative learning on a shared tablet through the split screen feature and crisscross technique
(2024-08-28) Čarapina, Mia
This paper explores the use of a split screen feature on tablets to facil-itate collaborative learning among primary school students. In known research, the split screen approach has been utilized on personal computers and in large display setups for shared interactions. Still, its application in mobile devices for multi-user educational activities remains underexplored. We developed a modu-lar system called CoCo (Colocated Collaboration), which divides a tablet's screen into independent segments, allowing students to interact simultaneously on a shared device. The system supports a structured activity flow consisting of indi-vidual work, discussion, and correction phases. This study, conducted with 89 Croatian primary school students, evaluated the effectiveness of this approach in supporting collaborative learning. The results showed an increase in students' success rates, indicating improved learning outcomes at the end of the evaluated activity. Despite the overall positive results of students’ perceived level of col-laboration, the study highlighted the need for refinements in activity design to better support sustained interaction and discussion. Future research will include a more in-depth analysis of results and focus on optimizing the system to enhance collaborative educational experiences on mobile devices.
Improving social presence in collaborative learning platforms: the case of PyramidApp
(2024-08-28) Szafranek, Karolina Martyna; Hernández Leo, Davinia
This research examines the role of social presence in Computer- Supported Collaborative Learning (CSCL) scripts and its facilitation through digital tools, specifically evaluating a tool implementing the Pyramid Collaborative Learning Flow Pattern (CLFP) from the perspective of the social presence theoretical framing aiming to suggest potential improvements. The study reveals that while the digital CSCL environment provides some level of social presence, it also has limitations that can be improved to support better emotional engagement and group cohesion. Enhancements proposed based on co-design interviews include upgrading the user interface with more visual cues, improving chat functionalities with emojis, direct replies and AI (Artificial Intelligence) integration, redesigning the answer improvement process for more intuitive collaboration, and refining the social awareness feature for clearer feedback. These improvements aim to address the current digital tool’s shortcomings and support a richer collaborative learning experience.
Multimodal intermediary agents for 3D-LLM-Protocols
(2024-08-28) Utsugi, Kei; Fujiwara, Takayuki; Kuwamoto, Satoshi; Yamashiro, Masao; Mitani, Keiichi; Hasebe, Tatsuya
We introduce our industrial metaverse systems, which centralizes the management of various documents in a CAD-based industorial digital twin 3D model. In the metaverse environment, we also developed a portable implementation method for manipulating information in this space by using LLM agents. The presentation will explain the concept of a protocol to link the 3D space with the LLM and the technical issues that need to be addressed.
Study on predicting collaborative performance: from personality traits vs. behavior in a different task
(2024-08-28) Kodama, Daiki; Takayama, Chihiro; Shimizu, Shinya
Most social activities are conducted with multiple entities to improve efficiency. It is difficult to estimate post-learning performance in advance when multiple entities (e.g., another user, robot, or autonomous agent) make decisions because they should consider each other’s subsequent behavior. Personality traits analyzed from responses to the questionnaire are easy to use for estimating performance but have low estimation accuracy. Machine learning using behavioral data on a target task can provide high accuracy, but such data is not available in advance. To solve this trade-off between in-advance estimability and estimation accuracy, we focused on the consistency of how people adjust their behavior to other collaborators in multiple tasks. We propose utilizing this consistency to estimate collaborative performance without performing the target task. We assumed that analyzing behavior adjustment in another task helps estimate performance of the target task. Our experimental results indicated that analyzing behavior adjustment in another task was beneficial to estimate collaborative performance in the target task, compared with personality-related questionnaire answers or performance of another task.
LA-ReflecT platform’s affordances for distributed multimodal learning analytics
(2024-08-28) Majumdar, Rwitajit; Singh, Daevesh; Rajendran, Ramkumar; Narayanan, Soumya; Gatare, Kinnari
We present the affordances of LA-ReflecT, an LTI-enabled platform, and its potential to collect geographically distributed and asynchronous multimodal learning data. As a user, teachers can author micro-learning activities with multimedia content and distribute it to the learners through the LMS. The system captures learners’ interaction as a log for analytics and presents it back in the dashboard for self-reflection. For multimodal data, it synchronizes API-based action logs such as YouTube video-watching behaviors and mobile-applicationbased data streaming from wearable sensors synchronized with their particular activity. It can be linked to their learning log, a time-series data upload option linked to a specific activity session, and synchronizing web-sensors data through API. It helps track interactions and artifacts students create within the platform as xAPI logs. The platform facilitates reflection on the action by presenting information on a dashboard as a part of the learning activity. LA-ReflecT also enables an app connecting with Bluetooth sensors that stream time series signals. The sensor data from a user is synchronized with their particular activity and can be linked to their learning log. We are doing multiple pilots with research partners in Japan and India. We are open to discussing adaptors who want to potentially pilot LA-ReflecT to create microlearning material and investigate the potential of the data logged in the platform.
CC BY-NC-SA, No doubt! License preference for sharing OERs on teacher community platforms
(2024-08-28) Hernández Leo, Davinia; Pérez-Basso, Francisco; Carrió, Mar
Open Educational Resources (OERs) are open and free-access teaching- learning materials, based on author-led sharing, readapting, and communitybuilding logics. OER are usually created and shared by teachers in open community platforms. Nonetheless, even if these teacher-led actions are altruistic and aimed at a collective effort towards improving education, it is important to ensure the protection of intellectual property and authorship. This also guarantees the legal and appropriate use, modification, development, and implementation of these resources. In order to respect the original creator's will, there are different licenses under which an OER can be shared, and it must be handled according to the stipulations established by the license. This study collects the data from a sample of in-service school teachers who participated in teacher training in digital competence development. The course introduced various Creative Commons (CC) licenses along with examples of teacher community platforms available for sharing and reusing OERs. Later, teachers were asked to provide their insights regarding which CC license they considered more suitable for publishing their materials based on their own conceptions and beliefs, and explain the reason(s). This information was collected from 18 responses to a proposed task. Results showed that the totality of teachers considered that the “Attribution-NonCommercial- ShareAlike” (CC BY-NC-SA) license is the one that best aligns with the mission and vision of public education and its institutions. It guarantees intellectual property by citing the original source, ensures the non-profit creation of resources, and preserves the same license conditions, thus avoiding extra-educational interests.
The elusive nature of inattentional deafness: assessing the influence of visual attention on background music perception in TV programmes
(2024-07-03) Batlle-Roca, Roser; Tyman, Diana; Meléndez Catalán, Blai; Molina, Emilio; Herrera-Boyer, Perfecto
Evaluating the audibility level of background music in TV programmes is a significant challenge as some music industry copyright regulations arrange music remuneration accordingly. Besides the loudness level, other characteristics, such as sensory attention, may influence its perception, raising or lowering its audibility threshold. Yet, there is limited literature exploring this particular problem. Our study examines how visually motivated attention impacts the perception of background music in TV programmes, contributing to inattentional deafness—the failure to perceive auditory stimuli due to visual perceptual load. We conducted two experiments based on forced-choice and detection tasks focused on assessing the influence of visually motivated attention on background music perception in TV programmes. Experiment 1 shows that participants may experience inattentional deafness when visually motivated, but not strongly enough to support our hypothesis. Hence, we refined our methodology in Experiment 2 with a dual-task paradigm to guarantee forced visual-motivated attention. Analysis via a one-way ANOVA demonstrates a statistically significant influence of forced visual attention (Task 1) towards the music perception assignment (Task 2). Thus, our findings indicate that the audibility of background music is subject to the visual stimuli load of the TV programme.
Simulating piano performance mistakes for music learning
(2024-07-03) Morsi, Alia; Zhang, Huan; Maezawa, Akira; Dixon, Simon; Serra, Xavier
The development of machine-learning based technologies to support music instrument learning needs large-scale datasets that capture the different stages of learning in a manner that is both realistic and computation-friendly. We are interested in modeling the mistakes of beginnerintermediate piano performances in practice or work-inprogress settings. In the absence of large-scale data representing our target case, our approach is to start by understanding such mistakes from real data and then provide a methodology for their simulation, thus creating synthetic data to support the training of performance assessment models. The main goals of this paper are: a) to propose a taxonomy of performance mistakes, specifically apt for simulating or reproducing/recreating them on mistake-free MIDI performances, and b) to provide a pipeline for creating synthetic datasets based on the former. We incorporate prior research in related contexts to facilitate the understanding of common mistake behaviours. Then, we design a hierarchical mistake taxonomy to categorize two real-world datasets capturing relevant piano performance contexts. Finally, we discuss our approach with 3 music teachers through a listening test and subsequent discussions.
Modeling the Pultec EQP-1A with wave digital filters
(2024-07-02) Barrera, Barrera,; Lizarraga Seijas, Xavier; Font, Frederic
This paper details the development of a virtual analog model of the passive equalizer stage of the classic Pultec EQP-1A studio equalizer using Wave Digital Filters. The modeling process involves analyzing the unit’s schematics, LTspice simulations, and implementing a Wave Digital Filter structure using the pywdf library in Python. The proposed structure initially used R-Type adaptors, that are removed in a second model, better optimized for realtime. The Python prototype is compared to LTspice simulations, showing that, at sufficiently high sampling rates, the discrepancies between the two are marginal. The Wave Digital Filter model was then ported to C++ using the chowdsp_wdf library. Both implementations of the model in C++ (the one that uses R-Type adaptors and the real-time optimized one) are tested. Our optimized implementation proved to be much faster than the other opensource, Faust-based emulation of the EQP1-A that we evaluated, and a more accurate emulation of the original unit’s equalizer stage.
GrooveTransformer: a generative drum sequencer Eurorack module
(2024-06-07) Evans, Nicholas; Haki, Behzad; Jordà Puig, Sergi
This paper presents the GrooveTransformer, a Eurorack module designed for generative drum sequencing. Central to its design is a Variational Auto-Encoder (VAE), around which we have designed a deployment context enabling performance through accompaniment and/or user interaction. This module allows the user to use the system as an accompaniment generator while interacting with the generative processes in real-time. In this paper, we review the design principles and technical architecture of the module, while also discussing the potentials and short-comings of our work.
Making democratic deliberation and participation more accessible: the iDEM project
(2024-05-27) Saggion, Horacio; O’Flaherty, John; Blanchet, Thomas; Sharoff, Serge; Sanfilippo, Silvia; Muñoz, Lian; Gollegger, Martin; Rascón, Almudena; Martí, José Luis; Szasz, Sandra; Bott, Stefan Markus; Sayman, Volkan
Deliberative and participatory processes currently don’t have full legitimacy due to the exclusion and marginalisation of several vulnerable communities from democratic spaces. It is well documented that people with limited language skills, such as people with cognitive disabilities, struggle to participate in democratic processes. This happens in spite of the advocacy work of organizations which promote human rights. The iDEM project aims at addressing the barriers in deliberative and participatory democratic practices through a thorough intersectional analysis of conditions under which current structures and systems limit participation of marginalised and vulnerable communities. Specially those with limited skills in reading, writing or understanding a fairly complex language, which is often required for deliberative and participatory processes. iDEM will lay the theoretical foundations for the analysis of current marginalisation from deliberative processes of diverse under-represented groups due to a lack of language skills. It will adopt a user-centred approach for making participatory processes more accessible and inclusive, developing advanced natural language processing technologies (NLP) and artificial intelligence (AI) to empower under-represented groups, with tools to facilitate communication and dialog in democratic spaces. iDEM will co-create the next-generation multilingual models aimed at: (1) detecting possible sources of problems in understanding messages for several European languages and audiences, (2) automatically adapting texts in those languages to be accessible and unbiased for these audiences, (3) providing AI tools for enhancing the generation of appropriate messages and discourses. iDEM aims to create more accessible democratic spaces in Italy and Spain with customised technology enhancing the participation and representation of marginalised groups by providing unbiased and inclusive technology.
Leveraging pre-trained autoencoders for interpretable prototype learning of music audio
(Institute of Electrical and Electronics Engineers (IEEE), 2024) Alonso Jiménez, Pablo; Pepino, Leonardo; Batlle-Roca, Roser; Zinemanas, Pablo; Bogdanov, Dmitry; Serra, Xavier; Rocamora, Martín
We present PECMAE an interpretable model for music audio classification based on prototype learning. Our model is based on a previous method, APNet, which jointly learns an autoencoder and a prototypical network. Instead, we propose to decouple both training processes. This enables us to leverage existing self-supervised autoencoders pre-trained on much larger data (EnCodecMAE), providing representations with better generalization. APNet allows prototypes’ reconstruction to waveforms for interpretability relying on the nearest training data samples. In contrast, we explore using a diffusion decoder that allows reconstruction without such dependency. We evaluate our method on datasets for music instrument classification (Medley-Solos-DB) and genre recognition (GTZAN and a larger in-house dataset), the latter being a more challenging task not addressed with prototypical networks before. We find that the prototype-based models preserve most of the performance achieved with the autoencoder embeddings, while the sonification of prototypes benefits understanding the behavior of the classifier
Computers in education: how can we support the teachers?
(2024-01-09) Hernández Leo, Davinia
Completing audio drum loops with symbolic drum suggestions
(2023-11-20) Haki, Behzad; Pelinski, Teresa; Nieto, Marina; Jordà Puig, Sergi
Sampled drums can be used as an affordable way of creating human-like drum tracks, or perhaps more interestingly, can be used as a mean of experimentation with rhythm and groove. Similarly, AI-based drum generation tools can focus on creating human-like drum patterns, or alternatively, focus on providing producers/musicians with means of experimentation with rhythm. In this work, we aimed to explore the latter approach. To this end, we present a suite of Transformer-based models aimed at completing audio drum loops with stylistically consistent symbolic drum events. Our proposed models rely on a reduced spectral representation of the drum loop, striking a balance between a raw audio recording and an exact symbolic transcription. Using a number of objective evaluations, we explore the validity of our approach and identify several challenges that need to be further studied in future iterations of this work. Lastly, we provide a real-time VST plugin that allows musicians/ producers to utilize the models in real-time production settings.
Carnatic singing voice separation using cold diffusion on training data with bleeding
(2023-10-30) Plaja-Roglans, Genís; Miron, Marius; Shankar, Adithi; Serra, Xavier
Supervised music source separation systems using deep learning are trained by minimizing a loss function between pairs of predicted separations and ground-truth isolated sources. However, open datasets comprising isolated sources are few, small, and restricted to a few music styles. At the same time, multi-track datasets with source bleeding are usually found larger in size, and are easier to compile. In this work, we address the task of singing voice separation when the ground-truth signals have bleeding and only the target vocals and the corresponding mixture are available. We train a cold diffusion model on the frequency domain to iteratively transform a mixture into the corresponding vocals with bleeding. Next, we build the final separation masks by clustering spectrogram bins according to their evolution along the transformation steps. We test our approach on a Carnatic music scenario for which solely datasets with bleeding exist, while current research on this repertoire commonly uses source separation models trained solely with Western commercial music. Our evaluation on a Carnatic test set shows that our system improves Spleeter on interference removal and it is competitive in terms of signal distortion. Code is open sourced.
Efficient notation assembly in optical music recognition
(2023-10-30) Penarrubia, Carlos; Garrido-Muñoz, Carlos; Valero-Mas, Jose J.; Calvo Zaragoza, Jorge
Optical Music Recognition (OMR) is the field of research that studies how to computationally read music notation from written documents. Thanks to recent advances in computer vision and deep learning, there are successful approaches that can locate the music-notation elements from a given music score image. Once detected, these elements must be related to each other to reconstruct the musical notation itself, in the so-called notation assembly stage. However, despite its relevance in the eventual success of the OMR, this stage has been barely addressed in the literature. This work presents a set of neural approaches to perform this assembly stage. Taking into account the number of possible syntactic relationships in a music score, we give special importance to the efficiency of the process in order to obtain useful models in practice. Our experiments, using the MUSCIMA++ handwritten sheet music dataset, show that the considered approaches are capable of outperforming the existing state of the art in terms of efficiency with limited (or no) performance degradation. We believe that the conclusions of this work provide novel insights into the notation assembly step, while indicating clues on how to approach the previous stages of the OMR and improve the overall performance.
Activities using smart IoT planters in learning spaces: human-centred design of a dashboard
(2023-10-25) Hernández Leo, Davinia; Ferrer, Josep; Vujovic, Milica; Tabuenca, Bernardo; Ortiz-Beltran, Ariel; Greller, Wolfgang; Carrió, Mar; Moyano Claramunt, Elisabet
Education plays a transversal key role in the UN's Sustainable Development Goals agenda. Educating young people on natural health and the interpretation of scientific evidence can contribute to increased levels of informed social sensitivity towards our natural environment and awareness about its effect on our planet and human wellbeing. To enable experienced-based environmental awareness learning activities, this paper proposes the development of a dashboard that visualizes data captured by sensors located in plants (smart IoT planters) available in learning spaces. The possibilities for learning activities using smart planters can be diverse (plant care, ambient or emotional implications, data analysis, etc.) and its design can influence the shape of desirable dashboard features. This paper offers an answer for such a type of dashboard design based on a human-centred methodology which involves stakeholders (experts and practitioners) in its co-design through guided hands-on workshops. The results show insights related to what are the types of learning activities supported by smart planters that can be especially valuable to educators and what design principles should be considered in the creation of the supporting dashboard. Resulting representative proposals for activities include plant monitoring, correlation of sensed data and observations, and collaborative tasks. Key values perceived by participants include expected high levels of students' engagement, critical thinking and familiarity with the scientific method. Design principles for a supporting dashboard include the use of a traffic light metaphor or enabling data collection that could serve for contrasting variables and observations at a moment in time and across time. The paper illustrates how the results achieved can lead to the design of a human-centred dashboard for situated environmental awareness education.
TapTamDrum: a dataset for dualized drum patterns
(2023-10-24) Haki, Behzad; Kotowski, Błażej; Lee, Cheuk Lun Isaac; Jordà Puig, Sergi
Drummers spend extensive time practicing rudiments to develop technique, speed, coordination, and phrasing. These rudiments are often practiced on "silent" practice pads using only the hands. Additionally, many percussive instruments across cultures are played exclusively with the hands. Building on these concepts and inspired by Einstein's probably apocryphal quote, "Make everything as simple as possible, but not simpler," we hypothesize that a dual-voice reduction could serve as a natural and meaningful compressed representation of multi-voiced drum patterns. This representation would retain more information than its corresponding monotonic representation while maintaining relative simplicity for tasks such as rhythm analysis and generation. To validate this potential representation, we investigate whether experienced drummers can consistently represent and reproduce the rhythmic essence of a given drum pattern using only their two hands. We present TapTamDrum: a novel dataset of repeated dualizations from four experienced drummers, along with preliminary analysis and tools for further exploration of the data.

Navigate

Browse

Recent Submissions