Search

Prolonged musical training induces important audio-visuo-motor plastic processes. However, little is known about how the musicians’ brain resolves multimodal conflicts while preparing for musical action. We run an electroencephalographic (EEG) investigation on how visual processing for action (score reading) is affected by preceding task-irrelevant piano sounds, usually associated to the same or to a different action. Presentation of an incongruent sound, 100 msec before a musical score with one single note, reduces Event-Related Potentials (ERPs) associated to score reading (N170) localised in the right temporo-parietal junction, as well as ERPs associated to conflict strength (N2) localised in the anterior cingulate cortex, superior and inferior right frontal cortex. These results suggest that listening to task-irrelevant auditory action effects (musical notes) interferes with both higher-order visual and frontal conflict monitoring processes. We conclude that, in the musicians’ brain, the automatic translation of musical sounds into motor plans, spread its effects to visually specific processing as well as strategic and amodal action monitoring mechanisms.

Previous studies show beneficial effects of musicality on the acquisition of a second language (L2). While most research focused on perceptual aspects, only few studies investigated the effects of musicality on productive phonology. The present study tested if musicality can predict productive phonological skills in L2 acquisition. Sixty-three students with no previous exposure to Arabic were asked to repeatedly listen to and immediately reproduce short sentences in standard Arabic. Before the sentence reproduction task, they completed an auditory discrimination task in three different between-subjects condition: attentive, in which participants were asked to discriminate phonological variations in the same Arabic sentence that they were asked to reproduce later; non-attentive, in which participants were asked to detect beeps in the same Arabic sentences without paying attention to their phonological content; and no-exposure, in which participants performed the discrimination task in another language (Serbian). The first, third and seventh reproductions of each participant were rated for intelligibility, accent, and syllabic errors by two independent evaluators, both native speakers of Arabic. Primary results showed that the intelligibility of the reproduced sentences was higher in participants with high musicality scores in the Advanced Measures of Music Audiation. Moreover, the intelligibility of sentences produced by highly musical participants improved more over time than the intelligibility of participants with lower musicality scores. Previous exposure to the Arabic sentence was beneficial in both the attentive and non-attentive conditions. Our results support the idea that musicality can have effects on productive skills even in the very first stages of L2 acquisition.

This study aims to understand the comprehensive relationship between indoor temperature, physiological signals, thermal sensation, and cognitive performance and estimate thermal sensation for optimum productivity. A series of human experiments were conducted with 48 participants, and local skin temperatures and thermal sensation data were collected in 6 temperatures. The participants took two cognitive performance tests, OSPAN (Operation Span Task) and Vigilance test, to quantify their productivity. This study found that relatively high skin temperature of specific local body spots (Arm, wrist (back), wrist (In), back, and neck) were correlated negatively with the occupant's cognitive performance. Also, there were significant differences by gender in cognitive performances and physiological signals in the same indoor temperature. The mean skin temperature differences between males and females of most local body spots were more significant in the high-performing group than the low-performing group for the vigilance test. Local skin temperature differences by gender were more significant in the vigilance test (response time) than the OSPAN (working memory). Additionally, gender and two local skin temperatures (wrist (back) and neck) were accountable to estimate thermal sensation for optimum productivity by utilizing the J48 decision tree algorithm. This study provides new insights that the occupant's local body skin temperatures are correlated significantly with the occupant's cognitive performance (response time and working memory) in different indoor temperature conditions, and a single or combined local skin temperatures can be used as a variable to estimate individualized thermal comfort for the occupant's optimum productivity.

Contrasting results in visual and auditory spatial memory stimulate the debate over the role of sensory modality and attention in identity-to-location binding. We investigated the role of sensory modality in the incidental/deliberate encoding of the location of a sequence of items. In 4 separated blocks, 88 participants memorised sequences of environmental sounds, spoken words, pictures and written words, respectively. After memorisation, participants were asked to recognise old from new items in a new sequence of stimuli. They were also asked to indicate from which side of the screen (visual stimuli) or headphone channel (sounds) the old stimuli were presented in encoding. In the first block, participants were not aware of the spatial requirement while, in blocks 2, 3 and 4 they knew that their memory for item location was going to be tested. Results show significantly lower accuracy of object location memory for the auditory stimuli (environmental sounds and spoken words) than for images (pictures and written words). Awareness of spatial requirement did not influence localisation accuracy. We conclude that: (a) object location memory is more effective for visual objects; (b) object location is implicitly associated with item identity during encoding and

The Corsi Block Tapping Task is a widespread test used to assess spatial working memory. Previous research hypothesized that the discrepancy found in some cases between the traditional and the digital (touchscreen) version of the Corsi block tapping task may be due to a direct motor resonance between the experimenter's and the participant's hand movements. However, we hypothesize that this discrepancy might be due to extra movement-related information included in the traditional version, lacking in the digital one. We investigated the effects of such task-irrelevant information using eCorsi, a touchscreen version of the task. In Experiment 1, we manipulate timing in sequence presentation, creating three conditions. In the Congruent condition, the inter-stimulus intervals reflected the physical distance in which the stimuli were spatially placed: The longer the spatial distance, the longer the temporal interval. In the Incongruent condition the timing changed randomly. Finally, in the Isochronous condition every stimulus appeared after a fixed interval, independently from its spatial position. The results showed a performance enhancement in the Congruent condition, suggesting an incidental spatio-temporal binding. In Experiment 2, we added straight lines between each location in the sequences: In the Trajectories condition participants saw trajectories from one spatial position to the other during sequence presentation, while a condition without such trajectories served as control. Results showed better performances in the Trajectories condition. We suggest that the timing and trajectories information play a significant role in the discrepancies found between the traditional and the touchscreen version of the Corsi Block Tapping Task, without the necessity of explanations involving direct motor resonance (e.g. seeing an actual hand moving) as a causal factor.

Contrasting results in visual and auditory working memory studies suggest that the mechanisms of association between location and identity of stimuli depend on the sensory modality of the input. In this auditory study, we tested whether the association of two features both encoded in the “what” stream is different from the association between a “what” and a “where” feature. In an old–new recognition task, blindfolded participants were presented with sequences of sounds varying in timbre, pitch and location. They were required to judge if either the timbre, pitch or location of a single-probe stimulus was identical or different to the timbre, pitch or location of one of the sounds of the previous sequence. Only variations in one of the three features were relevant for the task, whereas the other two features could vary, with task-irrelevant changes. Results showed that task-irrelevant variations in the “what” features (either timbre or pitch) caused an impaired recognition of sound location and in the othe...

The processing of extramusical meaning can be reflected in N400 effects of the ERP. However, how conceptual representations can be activated in music still needs to be specified. We investigated the activation of iconic meaningful representations in music by using a cross-modal semantic priming paradigm with an implicit task. Pictures of spatial scenes were semantically congruent or incongruent to preceding music in three stimulus onset asynchrony (SOA) conditions. The results revealed that the semantically incongruent target pictures elicited larger N400 amplitude than the congruent target pictures. Moreover, the semantic priming effect was modulated by the SOAs. The N400 effect was observed in the 200-ms and 800-ms SOA conditions, but not in the 1,200-ms SOA condition. These results suggest that extramusical meaning purely due to iconic sign quality can be activated, and that the conceptual activation in music can be rapid and automatic.

Previous research shows that music ability provides positive effects on language processing. This study aims at better clarifying the involvement of different linguistic subdomains in this cross-domain link, assessing whether or not musicality and music expertise enhance phonological and lexical tone processing of Mandarin Chinese. In two experiments different groups of adults and children with no previous experience in tonal languages, were invited to perform a same–different task trying to detect phonological and tonal variations in pairs of sequences of monosyllabic Mandarin Chinese words. Main results show that all subjects perform significantly better in detecting phonological variations rather than tonal ones. They also show that both melodic proficiency and music expertise are good predictors for a better tonal, but not phonological identification. Data lead to a model of music-to-language transfer effect in which musicality selectively affects linguistic intonation while leaving phonological proce...

In spite of a large body of empirical research demonstrating the importance of multisensory integration in cognition, there is still little research about multimodal encoding and maintenance effects in working memory. In this study we investigated multimodal encoding in working memory by means of an immediate serial recall task with different modality and format conditions. In a first non-verbal condition participants were presented with sequences of non-verbal inputs representing familiar (concrete) objects, either in visual, auditory or audio-visual formats. In a second verbal condition participants were presented with written, spoken, or bimodally presented words denoting the same objects represented by pictures or sounds in the non-verbal condition. The effects of articulatory suppression were assessed in both conditions. We found a bimodal superiority effect on memory span with non-verbal material, and a larger span with auditory (or bimodal) versus visual presentation with verbal material, with a significant effect of articulatory suppression in the two conditions.

Consumption of sugar-sweetened beverages (SSBs) is considered to be a contributor to diabetes and the epidemic of obesity in many countries. The popularity of non-caloric carbonated soft drinks as an alternative to SSBs may be a factor in reducing the health risks associated with SSBs consumption. This study focuses on the perceptual discrimination of SSBs from artificially sweetened beverages (ASBs). Fifty-five college students rated 14 commercially available carbonated soft drinks in terms of sweetness and likeability. They were also asked to recognize, if the drinks contained sugar or a non-caloric artificial sweetener. Overall, participants showed poor accuracy in discriminating drinks' sweeteners, with significantly lower accuracy for SSBs than ASBs. Interestingly, we found a dissociation between sweetener recognition and drink pleasantness. In fact, in spite of a chance-level discrimination accuracy of SSBs, their taste was systematically preferred to the taste of non-caloric beverages. Our findings support the idea that hedonic value of carbonated soft drinks is dissociable from its identification and that the activation of the pleasure system seems not to require explicit recognition of the sweetener contained in the soft drink. We hypothesize that preference for carbonated soft drinks containing sugar over non-caloric alternatives might be modulated by metabolic factors that are independent from conscious and rational consumers' choices.

In this study we tested incidental feature-to-location binding in a spatial task, both in unimodal and cross-modal conditions. In Experiment 1 we administered a computerised version of the Corsi Block-Tapping Task (CBTT) in three different conditions: the first one analogous to the original CBTT test; the second one in which locations were associated with unfamiliar images; the third one in which locations were associated with non-verbal sounds. Results showed no effect on performance by the addition of identity information. In Experiment 2, locations on the screen were associated with pitched sounds in two different conditions: one in which different pitches were randomly associated with locations and the other in which pitches were assigned to match the vertical position of the CBTT squares congruently with their frequencies. In Experiment 2 we found marginal evidence of a pitch facilitation effect in the spatial memory task. We ran a third experiment to test the same conditions of Experiment 2 with a within-subject design. Results of Experiment 3 did not confirm the pitch-location facilitation effect. We concluded that the identity of objects does not affect recalling their locations. We discuss our results within the framework of the debate about the mechanisms of "what" and "where" feature binding in working memory.

The present study was designed to obtain indications about unskilled children's music performances by means of a computer-based performance task. In particular, we assessed children's abilities to recognize their own performances and their self-awareness of such competence. A total of 240 children from Southern Italy, subdivided into 3 age groups, performed their own version of a familiar song. After 4, 6, or 8 days according to the group, participants were asked to recognize their own interpretation among two other performances recorded by participants in the same age group. Finally, the reasons for their recognition were elicited. Children's verbal responses were sorted by independent judges into different categories reflecting the cognitive strategies used in the recognition task. Results show that children are able to discriminate their own musical interpretation with a great degree of accuracy. Concerning the verbal responses, children mentioned the Dynamics category more frequently than other categories like Duration, Tempo, Structure, Elimination strategy, Error detection, Aural and tactile discrimination. No significant differences among groups with different retention intervals have been found, suggesting that memory of children's own musical performances is stable for several days. The comparison with a similar American sample shows cultural differences in the strategies used for recognition. Theoretical and methodological implications are also discussed.

Many studied reported that working memory components receive remarkable changes during lifespan. In order to better investigate this, we evaluated working memory components on human subjects belonging to five groups (10 subjects each) at different ages 6, 8 and 10 years old, young adult (age) and old adult (age). Our pattern of results shows a major transition in object sequence manipulation performance between ages 8 and 10 years. If related to young adults results, both 10-year-old children and old adults differed in accuracy and RT (specificare cosa significa) in both maintenance and manipulation conditions. In particular, young adults and old adults differ in RTs in the manipulation condition. Our results also suggest that a change in response strategy from 6 to 8 years of age, to prioritize accuracy may be present. Our findings appear consistent with recent neuroscientific findings, and lead to novel predictions.

The processing of extramusical meaning can be reflected in N400 effects of the ERP. However, how conceptual representations can be activated in music still needs to be specified. We investigated the activation of iconic meaningful representations in music by using a cross-modal semantic priming paradigm with an implicit task. Pictures of spatial scenes were semantically congruent or incongruent to preceding music in three stimulus onset asynchrony (SOA) conditions. The results revealed that the semantically incongruent target pictures elicited larger N400 amplitude than the congruent target pictures. Moreover, the semantic priming effect was modulated by the SOAs. The N400 effect was observed in the 200-ms and 800-ms SOA conditions, but not in the 1,200-ms SOA condition. These results suggest that extramusical meaning purely due to iconic sign quality can be activated, and that the conceptual activation in music can be rapid and automatic.

Purpose. This study aims at evaluating the effectiveness of sonification as a mean to provide access to geo-referenced information to users with visual impairments. Method. Thiry-five participants (10 congenitally blind, 10 with acquired blindness and 15 blindfolded sighted) completed four tasks of progressive difficulty. During each task, participants first explored a sonified map by using either a tablet or a keyboard to move across regions and listened to sounds giving information about the current location. Then the participants were asked to identify, among four tactile maps, the one that crossmodally corresponds to the sonifed map they just explored. Finally, participants answered a self-report questionnaire of understanding and satisfaction. Results. Participants achieved high accuracy in all of the four tactile map discrimination tasks. No significant performance difference was found neither between subjects that used keyboard or tablet, nor between the three groups of blind and sighted participants. Differences between groups and interfaces were found in the usage strategies. High levels of satisfaction and understanding of the tools and tasks emerged from users’ reports.

In three experiments, using a two-alternative forced-choice task, we obtained depth judgments of displays containing transparent regions. The regions varied in lightness, size, and animation. Observers nearly always strongly preferred one certain depth ordering among the regions, even though their lightness conditions were expected to give rise to ambiguity among possible orderings. This expectation was based on the contrast polarity model, which expects ambiguity in the absence of contrast polarity reversal. The expectation was founded also on a stronger condition based on the transmittance anchoring principle, which gives preference to the largest lightness contrast between regions. In the absence of contrast polarity reversal and in conditions of balanced regional contrast, preferences were shown to depend on additional conditions of contrast between two respective regions and their overlap. Depth ordering judgment seems to be based on a critical decision threshold, independently of the coordinate system used to specify lightness. We also investigated the role of non-photometric factors such as motion and relative size, and concluded that these variables can modulate depth ordering judgments in transparency.

Representing geographical information in non-visual modalities is a complex problem that deserves special attention since this could be a way to facilitate the access of visual impaired subjects to this type of special information once proved the correspondence among the different sensory representation. Tactile maps and, more recently, on-screen text readers have sought to solve this problem but tactile maps are difficult and expensive to produce and text readers do not allow an analogical representation of information. Recently alternative solutions have been found by using sonification (the use of nonspeech audio to con-vey information). iSonic, a prototype developed by the H.C.I Lab. at the University of Maryland brings a new interactive approach allowing users to navigate into acoustic sonificated maps. In iSonic each subject action on the map triggers sounds that inform about the placement and the features of a given region. This sonification design allows two different kinds of map exploration: one using the computer keyboard (KB), the other one using a touch tablet (TT). KB exploration proceeds by discrete steps. Subjects can move through the map using the arrow keys. KB exploration is analytical, systematic, and symbolic. Differently, TT exploration allows users to move freely in every direction with no constraints. TT exploration is global, analogical, and leads to a more isomorphic representation of the map space. Our research aimed at: • Evaluating how well iSonic unable users to perceive and represent shapes and data patterns on maps. • Assessing the effects of exploration methods (KB- TT) on map knowledge acquisition and pattern recognition. Method Seventeen blindfolded sighted students completed four tasks of progressive difficulty. Sonificated and tactile maps were used as experimental stimuli changing in shape (task 4) or in the distribution of the different levels of the variable across the states (tasks 1, 2, and 3). Subject started exploring for 3 min the sonificated map by means of a touch-tablet or a keyboard. Then they were presented with four tactile maps. One of hem was equal to that one just acoustically explored and the other three were distractors. The task consisted in identifying among the four tactile maps the one they previously heard during the exploration. They had to assign a value from 0 to 10 that rates each map for the probability of being the actual map they heard. Results Globally considered our results indicate that geo- graphic map exploration by means of a sonification tool leads to a fairly good representation of the map. In fact rates of recognition of the target maps in the four tasks are, almost always, significantly higher than the rates of distractors. Only in task 1 we observed a dis- tractor map to be confused with the target map. The confusion could be due to the great similarity between these specific target and distractor. Considering the relative effectiveness of the two kind of interfaces we found that in the easier tasks (1, 2, 4) Keyboard and Touch Tablet showed no differences in recognition performances. However, in task 3, the hardest one, in which subjects were asked to recognize complex pat- terns, it seems that the Touch Tablet having no direction constraints allows more accurate explorations, leading to higher scores of recognition.

In tonal languages, as Mandarin Chinese and Thai, word meaning is partially determined by lexical tones. Previous studies suggest that lexical tones are processed by native listeners as linguistic information and not as pure tonal information. This study aims at verifying if, in nontonal languages speakers, the discrimination of lexical Mandarin tones varies in function of the melodic ability. Forty-six students with no previous experience of Mandarin or any other tonal language were presented with two short lists of spoken monosyllabic Mandarin words and invited to perform a same-different task trying to identify whether the variation were phonological or tonal. Main results show that subjects perform significantly better in identifying phonological variations rather than tonal ones and interestingly, the group with a high melodic ability (assessed by Wing subtest 3) shows a better performance exclusively in detecting tonal variations.