Search

Several studies have reported that the COVID-19 pandemic has had deleterious effects on sleep quality and mood, but the mechanisms underlying these effects are not clearly understood. Recently, it has been shown that the acceptance component of mindfulness reduces anxiety, and, in turn, lower anxiety improves sleep quality. The purpose of this cross-sectional study was to assess changes in mindfulness traits, sleep–wake quality, and general distress, before, during, and after the first COVID-19 wave, testing the model in which acceptance influences sleep through anxiety in each period. A total of 250 participants were recruited before (Pre-Lockdown group: 69 participants, 29 females, 33.04 ± 12.94 years), during (Lockdown group: 78 participants, 59 females, 29.174 ± 8.50 years), and after (After-Lockdown group: 103 participants, 86 females, 30.29 ± 9.46 years) the first Italian lockdown. In each group, self-report questionnaires, assessing mindfulness facets, distress, and sleep–wake quality, were administered and assessed. The Lockdown group reported lower acceptance and higher depression, while the After-Lockdown group reported lower sleep–wake quality and higher anxiety. The results of the path analysis confirmed that higher acceptance reduced anxiety and higher anxiety decreased sleep–wake quality in all groups. Our results confirm that acceptance influences sleep through the mediating role of anxiety.

Attentional bias to sleep-related information is thought to be a core feature for developing and/or maintaining insomnia. This study used a hallmark measure of attentional bias, the dot-probe task, to determine whether this bias toward sleep-related stimuli was a function of the severity of insomnia symptoms. A sample of 231 volunteers (175 females; mean age of 26.91 ± 8.05 years) participated in this online study, filling out the Insomnia Severity Index (ISI) and performing a visual dot-probe task. After categorizing individuals based on the ISI score into normal, subclinical, and moderate/severe sleep groups, we only found a marginally significant interaction between sleep groups and the type of stimuli on RTs, suggesting that subclinical and moderate/severe sleep groups reported slower RTs for sleep-related words than for neutral words. When we calculated the attentional bias score (ABS), we found that ABS significantly differed from zero in the moderate/severe sleep group only, suggesting a disengagement for sleep-related information as a function of the severity of insomnia symptoms. This finding seems to suggest that insomnia is related to greater difficulties in shifting away from sleep-related stimuli.

The experimental works presented in this thesis have been carried out and published together with my supervisor Marco Mirolli and with Gianluca Baldassarre. In particular, chapter 3 is adapted from (Mirolli et al. 2013, Santucci et al. 2014b, 2010, 2012a); chapter 4 is adapted from (Santucci et al. 2013b,a); chapter 5 is adapted from (Santucci et al. 2014a); and chapter 6 is adapted from (Santucci et al. 2016). Moreover, (Santucci et al. 2012b) focused on the same topics of the research described in chapter 4. Since the experiments presented in that paper constitute preliminary results obtained in a simplified experimental scenario, they have not been included in this thesis. However, the insights provided by that work have been used in the research presented in chapter 4. Developing artificial agents able to autonomously discover new goals, to select them and learn the related skills is an important challenge for robotics. This becomes even crucial if we want robots to interact with real environments where they have to face many unpredictable problems and where it is not clear which skills will be the more suitable to solve them. The ability to learn and store multiple skills in order to use them when required is one of the main characteristics of biological agents: forming ample repertoires of actions is important to widen the possibility for an agent to better adapt to different environments and to improve its chance of survival and reproduction. Moreover, humans and other mammals explore the environment and learn new skills not only on the basis of reward-related stimuli but also on the basis of novel or unexpected neutral stimuli. The mechanisms related to this kind of learning processes have been studied under the heading of “Intrinsic Motivations” (IMs), and in the last decades the concept of IMs have been used in developmental and autonomous robotics to foster an artificial curiosity that can improve the autonomy and versatility of artificial agents. In the research presented in this thesis I focus on the development of open-ended learning robots able to autonomously discover interesting events in the environment and autonomously learn the skills necessary to reproduce those events. In particular, this research focuses on the role that IMs can play in fostering those processes and in improving the autonomy and versatility of artificial agents. Taking inspiration from recent and past research in this field, I tackle some of the interesting open challenges related to IMs and to the implementation of intrinsically motivated robots. I first focus on the neurophysiology underlying IM learning signals, and in particular on the relations between IMs and phasic dopamine (DA). With the support of a first computational model, I propose a new hypothesis that addresses the dispute over the nature and the functions of phasic DA activations: reconciling two contrasting theories in the literature and taking xi into account the different experimental data, I suggest that phasic DA can be considered as a reinforcement prediction error learning signal determined by both unexpected changes in the environment (temporary, intrinsic reinforcements) and biological rewards (permanent, extrinsic reinforcements). The results obtained with my computational model support the presented hypothesis, showing how such a learning signal can serve two important functions: driving both the discovery and acquisition of novel actions and the maximisation of rewards. Moreover, those results provide a first example of the power of IMs to guide artificial agents in the cumulative learning of complex behaviours that would not be learnt simply providing a direct reward for the final tasks. In a second work, I move to investigate the issues related to the implementation of IMs signal in robots. Since the literature still lacks a specific analysis of which is the best IM signal to drive skill acquisition, I compare in a robotic setup different typologies of IMs, as well as the different mechanisms used to implement them. The results provide two important contributions: 1) they show how IM signals based on the competence of the system are able to generate a better guidance for skill acquisition with respect to the signals based on the knowledge of the agent; 2) they identify a proper mechanism to generate a competence-based IM signal, showing that the stronger the link between the IM signal and the competence of the system, the better the performance. Following the aim of widening the autonomy and the versatility of artificial agents, in a third work I focus on the improvement of the control architecture of the robot. I build a new 3-level architecture that allows the system to select the goals to pursue, to search for the best way to achieve them, and acquire the related skills. I implement this architecture in a simulated iCub robot and test it in a 3D experimental scenario where the agent has to learn, on the basis of IMs, a reaching task where it is not clear which arm of the robot is the most suitable to reach the different targets. The performance of the system is compared to the one of my previous 2-level architecture system, where tasks and computational resources are associated at design time. The better performance of the system endowed with the new 3-level architecture highlights the importance of developing robots with different levels of autonomy, and in particular both the high-level of goal selection and the low-level of motor control. Finally, I focus on a crucial issue for autonomous robotics: the development of a system that is able not only to select its own goals, but also to discover them through the interaction with the environment. In the last work I present GRAIL, a Goal-discovering Robotic Architecture for Intrisically-motivated Learning. Building on the insights provided by my previous research, GRAIL is a 4-level hierarchical architecture that for the first time assembles in unique system different features necessary for the development of truly autonomous robots. GRAIL is able to autonomously 1) discover new goals, 2) create and store representations of the events associated to those goals, 3) select the goal to pursue, 4) select the computational resources to learn to achieve the desired goal, and 5) self-generate its own learning signals on the basis of the achievement of the selected goals. I implement GRAIL in a simulated iCub and test it in three different 3D experimental setup, comparing its performance to my previous systems, showing its capacity to generate new goals in unknown scenarios, and testing its ability to cope with stochastic environments. The experiments highlight on the one hand the importance of an appropriate hierarchical architecture for supporting the development of autonomous robots, and on the other hand how IMs (together with goals) can play a crucial role in the autonomous learning of multiple skills.

The role of dopamine has been strongly implicated in reward processes, but recent work shows an additional role as a signal that promotes the stable incorporation of novel information into long-term hippocampal memory. Indeed, dopamine neurons, in addition to being activated by reward, can be activated by novelty in the absence of reward. The computation of novelty is thought to occur in the hippocampus and is carried to the dopamine cells of the VTA through a polysynaptic pathway. Although a picture of novelty-dependent processes in the VTA and hippocampus is beginning to emerge, many aspects of the process remain unclear. Here, we will consider several issues: (1) What is relationship of novelty signals coming to the VTA from the superior colliculus, as compared to those that come from the hippocampus? (2) Can dopamine released by a reward enhance the learning of novel information? (3) Is there an interaction between motivational signals and hippocampal novelty signals? (4) What are the properties of the axons that generate dopamine release in the hippocampus? We close with a discussion of some of the outstanding open issues in this field.

Artificial Life since its first inception has changed the way many scientists use to design and carry out experiments in different disciplines such as psychology, sociology, biology, chemistry and the like. In the last two decades Artificial Life has become a great tool to shed light on different and profound scientific questions, however its potential in educational environments has not been completely exploited. In this article wedescribetwocasestudiesin whichArtificialLifehasbeen used as a powerful learning tool for the theory of evolution, BreedBot and co. (actually Breedbot, BestBot and BrainFarm), and for group dynamics, Learn2Lead. BreedBot and its sequels Bestbot and Brainfarm are integrated hardware/software platforms that allow users to evolve a robot able to explore effectively a customized environment made up of walls and colored obstacles. Each robot is controlled by an artificial neural network whose parameters are encoded in a genetic string. Users can opt either for a natural or an artificial selection. In Learn2Lead, each learner manages a simulated team of employees which competes against other teams to maximize its objectives (e.g. profit, volume of services delivered, customer satisfaction). Both serious games provide children and older users a very easy tool to understand complex concepts like evolution an group dynamics within a playful learning context.