publications

2024

1. NeurIPS

   

   Amplified Early Stopping Bias: Overestimated Performance with Deep Learning

   Nona Rajabi, Antonio H. Ribeiro, Miguel Vasco, and 1 more author

   In NeurIPS 2024 Workshop on Scientific Methods for Understanding Deep Learning, 2024

   Abs HTML

   Cross-validation is commonly used to estimate machine learning model performance on new samples. However, using it for both hyperparameter selection and error estimation can lead to overestimating model performance, especially with extensive hyperparameter searches that overly tailor models to validation data. We demonstrate that deep learning further amplifies this bias, with even minor model adjustments causing significant overestimation. Our extensive experiments on simulated and real data focus on the bias from early stopping during cross-validation. We find that overestimation intensifies with network depth and is especially severe in small datasets, which are common in physiological signal processing applications. Selecting the early stopping point during cross-validation can result in ROC-AUC estimates exceeding 90% on random data, and this effect persists across various sample sizes, architectures, and network sizes.

2. TMLR

   

   Reducing Variance in Meta-Learning via Laplace Approximation for Regression Tasks

   Alfredo Reichlin, Gustaf Tegnér, Miguel Vasco, and 3 more authors

   Transactions on Machine Learning Research, 2024

   Abs HTML Code

   Given a finite set of sample points, meta-learning algorithms aim to learn an optimal adaptation strategy for new, unseen tasks. Often, this data can be ambiguous as it might belong to different tasks concurrently. This is particularly the case in meta-regression tasks. In such cases, the estimated adaptation strategy is subject to high variance due to the limited amount of support data for each task, which often leads to sub-optimal generalization performance. In this work, we address the problem of variance reduction in gradient-based meta-learning and formalize the class of problems prone to this, a condition we refer to as \emphtask overlap. Specifically, we propose a novel approach that reduces the variance of the gradient estimate by weighing each support point individually by the variance of its posterior over the parameters. To estimate the posterior, we utilize the Laplace approximation, which allows us to express the variance in terms of the curvature of the loss landscape of our meta-learner. Experimental results demonstrate the effectiveness of the proposed method and highlight the importance of variance reduction in meta-learning.

3. NeurIPS

   

   Can Transformers Smell Like Humans?

   Farzaneh Taleb, Miguel Vasco, Antonio H. Ribeiro, and 2 more authors

   In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

   Spotlight Abs HTML

   Spotlight Presentation at NeurIPS 2024

   The human brain encodes stimuli from the environment into representations that form a sensory perception of the world. Despite recent advances in understanding visual and auditory perception, olfaction remains an under-explored topic in the machine learning community due to the lack of large-scale datasets annotated with labels related to human olfactory perception. In this work, we ask the question of whether transformer models trained on chemical structures encode representations that are aligned with human olfactory perception, i.e., \emphcan transformers smell like humans? We demonstrate, by means of three analyses, that representations encoded from transformers pre-trained on general chemical structures are highly aligned with human olfactory perception. We use 5 different datasets and 3 different types of perceptual representations to show that the representations encoded by transformer models are able to predict 1) labels associated with odorants‌‌ provided by experts; 2) ratings provided by human participants with respect to pre-defined descriptors; 3) similarity ratings between odorants provided by human participants. Finally, we also evaluate the extent to which this alignment is associated with chemical features of odorants known to be relevant for olfactory perception.

4. NeurIPS

   

   NeuralSolver: Learning Algorithms For Consistent and Efficient Extrapolation Across General Tasks

   Bernardo Esteves, Miguel Vasco, and Francisco S. Melo

   In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

   Abs HTML

   We contribute NeuralSolver, a novel recurrent solver that can efficiently and consistently extrapolate, i.e., learn algorithms from smaller problems (in terms of observation size) and execute those algorithms in large problems. Contrary to previous recurrent solvers, NeuralSolver can be naturally applied in both same-size problems, where the input and output sizes are the same, and in different-size problems, where the size of the input and output differ. To allow for this versatility, we design NeuralSolver with three main components: a recurrent module, that iteratively processes input information at different scales, a processing module, responsible for aggregating the previously processed information, and a curriculum-based training scheme, that improves the extrapolation performance of the method. To evaluate our method we introduce a set of novel different-size tasks and we show that NeuralSolver consistently outperforms the prior state-of-the-art recurrent solvers in extrapolating to larger problems, considering smaller training problems and requiring less parameters than other approaches.

5. RLC

   

   A Super-human Vision-based Reinforcement Learning Agent for Autonomous Racing in Gran Turismo

   Miguel Vasco^*, Takuma Seno^*, Kenta Kawamoto, and 3 more authors

   Reinforcement Learning Journal, 2024

   Outstanding Paper Award Abs HTML PDF

   Outstading Paper Award in Applications of RL

   Racing autonomous cars faster than the best human drivers has been a longstanding grand challenge for the fields of Artificial Intelligence and robotics. Recently, an end-to-end deep reinforcement learning agent met this challenge in a high-fidelity racing simulator, Gran Turismo. However, this agent relied on global features that require instrumentation external to the car. This paper introduces, to the best of our knowledge, the first super-human car racing agent whose sensor input is purely local to the car, namely pixels from an ego-centric camera view and quantities that can be sensed from on-board the car, such as the car’s velocity. By leveraging global features only at training time, the learned agent is able to outperform the best human drivers in time trial (one car on the track at a time) races using only local input features. The resulting agent is evaluated in Gran Turismo 7 on multiple tracks and cars. Detailed ablation experiments demonstrate the agent’s strong reliance on visual inputs, making it the first vision-based super-human car racing agent.

6. 

   Will You Participate? Exploring the Potential of Robotics Competitions on Human-Centric Topics

   Yuchong Zhang, Miguel Vasco, Mårten Björkman, and 1 more author

   In International Conference on Human-Computer Interaction, 2024

   Abs HTML PDF

   This paper presents findings from an exploratory needfinding study investigating the research current status and potential participation of the competitions on the robotics community towards four human-centric topics: safety, privacy, explainability, and federated learning. We conducted a survey with 34 participants across three distinguished European robotics consortia, nearly 60% of whom possessed over five years of research experience in robotics. Our qualitative and quantitative analysis revealed that current mainstream robotic researchers prioritize safety and explainability, expressing a greater willingness to invest in further research in these areas. Conversely, our results indicate that privacy and federated learning garner less attention and are perceived to have lower potential. Additionally, the study suggests a lack of enthusiasm within the robotics community for participating in competitions related to these topics. Based on these findings, we recommend targeting other communities, such as the machine learning community, for future competitions related to these four human-centric topics.

7. 

   Goal-Conditioned Offline Reinforcement Learning via Metric Learning

   Alfredo Reichlin, Miguel Vasco, Hang Yin, and 1 more author

   arXiv preprint arXiv:2402.10820, 2024

   Abs HTML

   We address the problem of learning optimal behavior from sub-optimal datasets for goal-conditioned offline reinforcement learning. To do so, we propose the use of metric learning to approximate the optimal value function for goal-conditioned offline RL problems under sparse rewards, invertible actions and deterministic transitions. We introduce distance monotonicity, a property for representations to recover optimality and propose an optimization objective that leads to such property. We use the proposed value function to guide the learning of a policy in an actor-critic fashion, a method we name MetricRL. Experimentally, we show that our method estimates optimal behaviors from severely sub-optimal offline datasets without suffering from out-of-distribution estimation errors. We demonstrate that MetricRL consistently outperforms prior state-of-the-art goal-conditioned RL methods in learning optimal policies from sub-optimal offline datasets.

8. AAMAS

   

   Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning

   Pedro P. Santos, Diogo S. Carvalho, Miguel Vasco, and 4 more authors

   In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, 2024

   Abs HTML PDF

   We introduce hybrid execution in multi-agent reinforcement learn- ing (MARL), a new paradigm in which agents aim to successfully complete cooperative tasks with arbitrary communication levels at execution time by taking advantage of information-sharing among the agents. Under hybrid execution, the communication level can range from a setting in which no communication is allowed between agents (fully decentralized), to a setting featuring full communi- cation (fully centralized), but the agents do not know beforehand which communication level they will encounter at execution time. To formalize our setting, we define a new class of multi-agent par- tially observable Markov decision processes (POMDPs) that we name hybrid-POMDPs, which explicitly model a communication process between the agents.

2023

1. 

   Pre-training with Augmentations for Efficient Transfer in Model-Based Reinforcement Learning

   Bernardo Esteves, Miguel Vasco, and Francisco S. Melo

   In EPIA Conference on Artificial Intelligence, 2023

   Abs HTML

   This work explores pre-training as a strategy to allow reinforcement learning (RL) algorithms to efficiently adapt to new (albeit similar) tasks. We argue for introducing variability during the pre-training phase, in the form of augmentations to the observations of the agent, to improve the sample efficiency of the fine-tuning stage. We categorize such variability in the form of perceptual, dynamic and semantic augmentations, which can be easily employed in standard pre-training methods. We perform extensive evaluations of our proposed augmentation scheme in model-based algorithms, across multiple scenarios of increasing complexity. The results consistently show that our augmentation scheme significantly improves the efficiency of the fine-tuning to novel tasks, outperforming other state-of-the-art pre-training approaches.

2. 

   Detecting the Intention of Object Handover in Human-Robot Collaborations: An EEG Study

   Nona Rajabi, Parag Khanna, Sumeyra U Demir Kanik, and 5 more authors

   In 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2023

   Abs HTML Video

   Human-robot collaboration (HRC) relies on smooth and safe interactions. In this paper, we focus on the human-to-robot handover scenario, where the robot acts as a taker. We investigate the feasibility of detecting the intention of a human-to-robot handover action through the analysis of electroencephalogram (EEG) signals. Our study confirms that temporal patterns in EEG signals provide information about motor planning and can be leveraged to predict the likelihood of an individual executing a motor task with an average accuracy of 94.7%. We also suggest the effectiveness of the time-frequency features of EEG signals in the final second prior to the movement for distinguishing between handover action and other actions. Furthermore, we classify human intentions for different tasks based on time-frequency representations of pre-movement EEG signals and achieve an average accuracy of 63.5% for contrasting every two tasks against each other. The result encourages the possibility of using EEG signals to detect human handover intention in HRC tasks.

3. 

   Mental Face Image Retrieval Based on a Closed-Loop Brain-Computer Interface

   Nona Rajabi, Charles Chernik, Alfredo Reichlin, and 5 more authors

   In International Conference on Human-Computer Interaction, 2023

   Abs HTML

   Retrieval of mental images from measured brain activity may facilitate communication, especially when verbal or muscular communication is impossible or inefficient. The existing work focuses mostly on retrieving the observed visual stimulus while our interest is on retrieving the imagined mental image. We present a closed-loop BCI framework to retrieve mental images of human faces. We utilize EEG signals as binary feedback to determine the relevance of an image to the target mental image. We employ the feedback to traverse the latent space of a generative model to propose new images closer to the actual target image. We evaluate the proposed framework on 13 volunteers. Unlike previous studies, we do not restrict the possible attributes of the resulting images to predefined semantic classes. Subjective and objective tests validate the ability of our model to retrieve face images similar to the actual target mental images.

2022

1. 

   Perceive, Represent, Generate: Translating Multimodal Information to Robotic Motion Trajectories

   Fábio Vital, Miguel Vasco, Alberto Sardinha, and 1 more author

   In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022

   Abs HTML PDF

   We present Perceive-Represent-Generate (PRG), a novel three-stage framework that maps perceptual information of different modalities (e.g., visual or sound), corresponding to a sequence of instructions, to an adequate sequence of movements to be executed by a robot. In the first stage, we perceive and pre-process the given inputs, isolating individual commands from the complete instruction provided by a human user. In the second stage we encode the individual commands into a multimodal latent space, employing a deep generative model. Finally, in the third stage we convert the multimodal latent values into individual trajectories and combine them into a single dynamic movement primitive, allowing its execution in a robotic platform. We evaluate our pipeline in the context of a novel robotic handwriting task, where the robot receives as input a word through different perceptual modalities (e.g., image, sound), and generates the corresponding motion trajectory to write it, creating coherent and readable handwritten words.

2. ICML

   

   Geometric Multimodal Contrastive Representation Learning

   Petra Poklukar^*, Miguel Vasco^*, Hang Yin, and 3 more authors

   In Proceedings of the 39th International Conference on Machine Learning, 2022

   Abs HTML PDF Code

   Learning representations of multimodal data that are both informative and robust to missing modalities at test time remains a challenging problem due to the inherent heterogeneity of data obtained from different channels. To address it, we present a novel Geometric Multimodal Contrastive (GMC) representation learning method consisting of two main components: i) a two-level architecture consisting of modality-specific base encoders, allowing to process an arbitrary number of modalities to an intermediate representation of fixed dimensionality, and a shared projection head, mapping the intermediate representations to a latent representation space; ii) a multimodal contrastive loss function that encourages the geometric alignment of the learned representations. We experimentally demonstrate that GMC representations are semantically rich and achieve state-of-the-art performance with missing modality information on three different learning problems including prediction and reinforcement learning tasks.

3. AAMAS

   

   How to Sense the World: Leveraging Hierarchy in Multimodal Perception for Robust Reinforcement Learning Agents

   Miguel Vasco, Hang Yin, Francisco S. Melo, and 1 more author

   In 21st International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), 2022

   Abs HTML PDF Code

   This work addresses the problem of sensing the world: how to learn a multimodal representation of a reinforcement learning agent’s environment that allows the execution of tasks under incomplete perceptual conditions. To address such problem, we argue for hierarchy in the design of representation models and contribute with a novel multimodal representation model, MUSE. The proposed model learns hierarchical representations: low-level modality-specific representations, encoded from raw observation data, and a high-level multimodal representation, encoding joint-modality information to allow robust state estimation. We employ MUSE as the sensory representation model of deep reinforcement learning agents provided with multimodal observations in Atari games. We perform a comparative study over different designs of reinforcement learning agents, showing that MUSE allows agents to perform tasks under incomplete perceptual experience with minimal performance loss. Finally, we evaluate the performance of MUSE in literature-standard multimodal scenarios with higher number and more complex modalities, showing that it outperforms state-of-the-art multimodal variational autoencoders in single and cross-modality generation.

4. Neural Networks

   

   Leveraging hierarchy in multimodal generative models for effective cross-modality inference

   Miguel Vasco, Hang Yin, Francisco S. Melo, and 1 more author

   Neural Networks (2021 Special Issue on AI and Brain Science: Brain-inspired AI), 2022

   Abs HTML Code

   This work addresses the problem of sensing the world: how to learn a multimodal representation of a reinforcement learning agent’s environment that allows the execution of tasks under incomplete perceptual conditions. To address such problem, we argue for hierarchy in the design of representation models and contribute with a novel multimodal representation model, MUSE. The proposed model learns hierarchical representations: low-level modality-specific representations, encoded from raw observation data, and a high-level multimodal representation, encoding joint-modality information to allow robust state estimation. We employ MUSE as the sensory representation model of deep reinforcement learning agents provided with multimodal observations in Atari games. We perform a comparative study over different designs of reinforcement learning agents, showing that MUSE allows agents to perform tasks under incomplete perceptual experience with minimal performance loss. Finally, we evaluate the performance of MUSE in literature-standard multimodal scenarios with higher number and more complex modalities, showing that it outperforms state-of-the-art multimodal variational autoencoders in single and cross-modality generation.

2021

1. 

   Exploiting Symmetry in Human Robot-Assisted Dressing Using Reinforcement Learning

   Pedro Ildefonso, Pedro Remédios, Rui Silva, and 4 more authors

   In EPIA Conference on Artificial Intelligence, 2021

   Abs HTML

   In this work, we address the problem of symmetry transfer in human-robot collaborative tasks, i.e., how certain actions can be extended to their symmetrical by exploiting symmetries in their execution. We contribute an approach capable of considering the symmetry inherent to a given task, such as the human or robot’s lateral symmetry, abstracting them from the robot’s decision process. We instantiate our approach in a robot-assisted backpack dressing scenario. A two-manipulator Baxter robot assists a human user in sequentially putting on both straps of a backpack. We evaluate the proposed symmetry-transfer approach in two complementary perspectives: the quality of the agent’s learned policy in a simulated environment and the efficiency of the complete system in a real-life scenario with a robotic platform. The results show that our approach allows the extension of the execution of single-side trained collaborative tasks to their symmetrical with no additional training and minimal performance loss.

2020

1. AAMAS

   

   Playing Games in the Dark: An Approach for Cross-Modality Transfer in Reinforcement Learning

   Rui Silva, Miguel Vasco, Francisco S. Melo, and 2 more authors

   In 19th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), 2020

   Abs HTML PDF Code

   In this work we explore the use of latent representations obtained from multiple input sensory modalities (such as images or sounds) in allowing an agent to learn and exploit policies over different subsets of input modalities. We propose a three-stage architecture that allows a reinforcement learning agent trained over a given sensory modality, to execute its task on a different sensory modality-for example, learning a visual policy over image inputs, and then execute such policy when only sound inputs are available. We show that the generalized policies achieve better out-of-the-box performance when compared to different baselines. Moreover, we show this holds in different OpenAI gym and video game environments, even when using different multimodal generative models and reinforcement learning algorithms.

2. 

   Explainable Agency by Revealing Suboptimality in Child-Robot Learning Scenarios

   Silvia Tulli, Marta Couto, Miguel Vasco, and 3 more authors

   In International Conference on Social Robotics (ICSR), 2020

   Best Student Paper Award Abs HTML

   Best Student Paper Award

   Revealing the internal workings of a robot can help a human better understand the robot’s behaviors. How to reveal such workings, e.g., via explanation generation, remains a significant challenge. This gets even more complex when these explanations are targeted towards children. Therefore, we propose a search-based approach to generate contrastive explanations using optimal and sub-optimal plans and implement it in a scenario for children. In the application scenario, the child and the robot learn together how to play a zero-sum game that requires logical and mathematical thinking. We report results around our explanation generation system that was successfully deployed among seven-year-old children. Our results show trends that the generated explanations were able to positively affect the children’s perceived difficulty in learning the zero-sum game.

3. AAMAS

   

   Multimodal Representation Learning for Robotic Cross-Modality Policy Transfer

   Miguel Vasco

   In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), 2020

   Doctoral Consortium of AAMAS 2020

   Abs PDF Video

   In this thesis, we aim at endowing robots with mechanisms to learn multimodal representations from sensory data and to allow them to execute tasks considering different subsets of available perceptions. We address the learning of these representations from supervised, unsupervised and reinforcement learning methodologies in the context of virtual agents and robots. We hope that, by achieving the proposed goals, the contributions of this thesis might prompt future research on applications of multimodal representations in robots and other artificial agents.

4. 

   MHVAE: a Human-inspired Deep Hierarchical Generative Model for Multimodal Representation Learning

   Miguel Vasco, Francisco S. Melo, and Ana Paiva

   arXiv preprint arXiv:2006.02991, 2020

   Abs HTML PDF

   Humans are able to create rich representations of their external reality. Their internal representations allow for cross-modality inference, where available perceptions can induce the perceptual experience of missing input modalities. In this paper, we contribute the Multimodal Hierarchical Variational Auto-encoder (MHVAE), a hierarchical multimodal generative model for representation learning. Inspired by human cognitive models, the MHVAE is able to learn modality-specific distributions, of an arbitrary number of modalities, and a joint-modality distribution, responsible for cross-modality inference. We formally derive the model’s evidence lower bound and propose a novel methodology to approximate the joint-modality posterior based on modality-specific representation dropout. We evaluate the MHVAE on standard multimodal datasets. Our model performs on par with other state-of-the-art generative models regarding joint-modality reconstruction from arbitrary input modalities and cross-modality inference.

2019

1. 

   Learning multimodal representations for sample-efficient recognition of human actions

   Miguel Vasco, Francisco S. Melo, David Martins Matos, and 2 more authors

   In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019

   Abs HTML

   Humans interact in rich and diverse ways with the environment. However, the representation of such behavior by artificial agents is often limited. In this work we present motion concepts, a novel multimodal representation of human actions in a household environment. A motion concept encompasses a probabilistic description of the kinematics of the action along with its contextual background, namely the location and the objects held during the performance. We introduce a novel algorithm which learns and recognizes motion concepts from action demonstrations, named Online Motion Concept Learning (OMCL). The algorithm is evaluated on a virtual-reality household environment with the presence of a human avatar. OMCL outperforms standard motion recognition algorithms on an one-shot recognition task, attesting to its potential for sample-efficient recognition of human actions.

2. 

   Project INSIDE: towards autonomous semi-unstructured human–robot social interaction in autism therapy

   Francisco S. Melo, Alberto Sardinha, David Belo, and 20 more authors

   Artificial intelligence in medicine, 2019

   Abs HTML

   This paper describes the INSIDE system, a networked robot system designed to allow the use of mobile robots as active players in the therapy of children with autism spectrum disorders (ASD). While a significant volume of work has explored the impact of robots in ASD therapy, most such work comprises remotely operated robots and/or well-structured interaction dynamics. In contrast, the INSIDE system allows for complex, semi-unstructured interaction in ASD therapy while featuring a fully autonomous robot. In this paper we describe the hardware and software infrastructure that supports such rich form of interaction, as well as the design methodology that guided the development of the INSIDE system. We also present some results on the use of our system both in pilot and in a long-term study comprising multiple therapy sessions with children at Hospital Garcia de Orta, in Portugal, highlighting the robustness and autonomy of the system as a whole.

3. AAMAS

   

   Online Motion Concept Learning: A Novel Algorithm for Sample-Efficient Learning and Recognition of Human Actions

   Miguel Vasco, Francisco S. Melo, David Matos, and 2 more authors

   In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), 2019

   PDF

2017

1. 

   3D map distribution of metallic nanoparticles in whole cells using MeV ion microscopy

   Miguel Vasco, Luís Cerqueira Alves, Victoria Corregidor, and 6 more authors

   Journal of Microscopy, 2017

   Abs HTML

   In this work, a new tool was developed, the MORIA program that readily translates Rutherford backscattering spectrometry (RBS) output data into visual information, creating a display of the distribution of elements in a true three-dimensional (3D) environment. The program methodology is illustrated with the analysis of yeast Saccharomyces cerevisiae cells, exposed to copper oxide nanoparticles (CuO-NP) and HeLa cells in the presence of gold nanoparticles (Au-NP), using different beam species, energies and nuclear microscopy systems. Results demonstrate that for both cell types, the NP internalization can be clearly perceived. The 3D models of the distribution of CuO-NP in S. cerevisiae cells indicate the nonuniform distribution of NP in the cellular environment and a relevant confinement of CuO-NP to the cell wall. This suggests the impenetrability of certain cellular organelles or compartments for NP. By contrast, using a high-resolution ion beam system, discretized agglomerates of Au-NP were visualized inside the HeLa cell. This is consistent with the mechanism of entry of these NPs in the cellular space by endocytosis enclosed in endosomal vesicles. This approach shows RBS to be a powerful imaging technique assigning to nuclear microscopy unparalleled potential to assess nanoparticle distribution inside the cellular volume.