Search

Funding Information: This work was supported by the Academy of Finland (Project Nos. 274058 and 274075), the Doctoral Programme in Psychology, Learning and Communication, the Finnish Cultural Foundation, and Otto A. Malm Foundation. Publisher Copyright: Copyright © 2023 Junttila, Smolander, Karhila, Kurimo and Ylinen. Children with dyslexia often face difficulties in learning foreign languages, which is reflected as weaker neural activation. However, digital language-learning applications could support learning-induced plastic changes in the brain. Here we aimed to investigate whether plastic changes occur in children with dyslexia more readily after targeted training with a digital language-learning game or similar training without game-like elements. We used auditory event-related potentials (ERPs), specifically, the mismatch negativity (MMN), to study learning-induced changes in the brain responses. Participants were 24 school-aged Finnish-speaking children with dyslexia and 24 age-matched typically reading control children. They trained English speech sounds and words with “Say it again, kid!” (SIAK) language-learning game for 5 weeks between ERP measurements. During the game, the players explored game boards and produced English words aloud to score stars as feedback from an automatic speech recognizer. To compare theeffectiveness of the training type (game vs. non-game), we embedded in the game some non-game levels stripped of all game-like elements. In the dyslexia group, the non-game training increased the MMN amplitude more than the game training, whereas in the control group the game training increased the MMN response more than the non-game training. In the dyslexia group, the MMN increase with the non-game training correlated with phonological awareness: the children with poorer phonological awareness showed a larger increase in the MMN response. Improved neural processing of foreign speech sounds as indicated by the MMN increase suggests that targeted training with a simple application could alleviate some spoken foreign-language learning difficulties that are related to phonological processing in children with dyslexia.

The association between impaired speech perception and reading difficulty has been well established in native language processing, as can be observed from brain activity. However, there has been scarce investigation of whether this association extends to brain activity during foreign language processing. The relationship between reading skills and neuronal speech representation of foreign language remains unclear. In the present study, we used event-related potentials (ERPs) with high-density EEG to investigate this question. Eleven- to 13-year-old children typically developed (CTR) or with reading difficulties (RD) were tested via a passive auditory oddball paradigm containing native (Finnish) and foreign (English) speech items. The change-detection-related ERP responses, the mismatch response (MMR), and the late discriminative negativity (LDN) were studied. The cluster-based permutation tests within and between groups were performed. The results showed an apparent language effect. In the CTR group, we found an atypical MMR in the foreign language processing and a larger LDN response for speech items containing a diphthong in both languages. In the RD group, we found unstable MMR with lower amplitude and a nonsignificant LDN response. A deficit in the LDN response in both languages was found within the RD group analysis. Moreover, we observed larger brain responses in the RD group and a hemispheric polarity reversal compared to the CTR group responses. Our results provide new evidence that language processing differed between the CTR and RD groups in early and late discriminatory responses and that language processing is linked to reading skills in both native and foreign language contexts. publishedVersion

Publisher Copyright: © Copyright © 2021 Ylinen, Smolander, Karhila, Kakouros, Lipsanen, Huotilainen and Kurimo. Digital and mobile devices enable easy access to applications for the learning of foreign languages. However, experimental studies on the effectiveness of these applications are scarce. Moreover, it is not understood whether the effects of speech and language training generalize to features that are not trained. To this end, we conducted a four-week intervention that focused on articulatory training and learning of English words in 6–7-year-old Finnish-speaking children who used a digital language-learning game app Pop2talk. An essential part of the app is automatic speech recognition that enables assessing children’s utterances and giving instant feedback to the players. The generalization of the effects of such training in English were explored by using discrimination tasks before and after training (or the same period of time in a control group). The stimuli of the discrimination tasks represented phonetic contrasts from two non-trained languages, including Russian sibilant consonants and Mandarin tones. We found some improvement with the Russian sibilant contrast in the gamers but it was not statistically significant. No improvement was observed for the tone contrast for the gaming group. A control group with no training showed no improvement in either contrast. The pattern of results suggests that the game may have improved the perception of non-trained speech sounds in some but not all individuals, yet the effects of motivation and attention span on their performance could not be excluded with the current methods. Children’s perceptual skills were linked to their word learning in the control group but not in the gaming group where recurrent exposure enabled learning also for children with poorer perceptual skills. Together, the results demonstrate beneficial effects of learning via a digital application, yet raise a need for further research of individual differences in learning.

We investigated user experiences from 117 Finnish children aged between 8 and 12 years in a trial of an English language learning programme that used automatic speech recognition (ASR). We used measures that encompassed both affective reactions and questions tapping into the children' sense of pedagogical utility. We also tested their perception of sound quality and compared reactions of game and nongame-based versions of the application. Results showed that children expressed higher affective ratings for the game compared to nongame version of the application. Children also expressed a preference to play with a friend compared to playing alone or playing within a group. They found that assessment of their speech is useful although they did not necessarily enjoy hearing their own voices. The results are discussed in terms of the implications for user interface (UI) design in speech learning applications for children.

We introduce a digital game for children’s foreign-language learning that uses automatic speech recognition (ASR) for evaluating children’s utterances. Our first prototype focuses on the learning of English words and their pronunciation. The game connects to a network server, which handles the recognition and pronunciation grading of children’s foreign-language speech. The server is reusable for different applications. Given suitable acoustic models, it can be used for grading pronunciations in any language.

The spatiotemporal dynamics of the neural processing of spoken morphologically complex words are still an open issue. In the current study, we investigated the time course and neural sources of spoken inflected and derived words using simultaneously recorded electroencephalography (EEG) and magnetoencephalography (MEG) responses. Ten participants (native speakers) listened to inflected, derived, and monomorphemic Finnish words and judged their acceptability. EEG and MEG responses were time-locked to both the stimulus onset and the critical point (suffix onset for complex words, uniqueness point for monomorphemic words). The ERP results showed that inflected words elicited a larger left-lateralized negativity than derived and monomorphemic words approximately 200 ms after the critical point. Source modeling of MEG responses showed one bilateral source in the superior temporal area similar to 100 ms after the critical point, with derived words eliciting stronger source amplitudes than inflected and monomorphemic words in the right hemisphere. Source modeling also showed two sources in the temporal cortex approximately 200 ms after the critical point. There, inflected words showed a more systematic pattern in source locations and elicited temporally distinct source activity in comparison to the derived word condition. The current results provide electrophysiological evidence for at least partially distinct cortical processing of spoken inflected and derived words. In general, the results support models of morphological processing stating that during the recognition of inflected words, the constituent morphemes are accessed separately. With regard to derived words, stem and suffix morphemes might be at least initially activated along with the whole word representation.

Kaksi ensiluokkalaista lasta, oppimispulmaton venäjänkielinen suomenoppija ja suomenkielinen luku- ja kirjoitushäiriöinen lapsi, joilla kummallakin oli ongelmia vokaalikvantiteetin tunnistustaidossa, osallistuivat tietokonepohjaiseen kvantiteettipeli-interventioon. Tunnistustaito mitattiin jakson aikana seitsemän kertaa, ja kielelliset ja kognitiiviset taidot alku-, loppu- ja seurantamittauksessa. Venäjänkielisen lapsen tunnistustaito sekä kvantiteetin yleinen erottelutaito paranivat heti harjoittelun myötä. Myös kvantiteetin luku- ja oikeinkirjoitustarkkuus, foneemien ja grafeemien tuntemus ja fonologinen tietoisuus parantuivat. Lukuja kirjoitushäiriöisellä lapsella tunnistustaito parantui seurantamittauksessa, mutta hänellä oli horjuntaa kvantiteetin yleisessä erottelutaidossa. Lapsi paransi lukusujuvuutta sekä kvantiteetin oikeinkirjoitustarkkuutta. Kvantiteetin oppiminen voi vaatia enemmän harjoitusta natiiveilla luku- ja kirjoitushäiriöisillä lapsilla kuin venäjänkielisillä suomea opettelevilla lapsilla. Avainsanat: kvantiteetti, kesto, toisen kielen oppiminen, luku- ja kirjoitushäiriö, interventio

Different languages use temporal speech cues in different linguistic functions. In Finnish, speech-sound duration is used as the primary cue for the phonological quantity distinction ― i.e., a distinction between short and long phonemes. For the second-language (L2) learners of Finnish, quantity is often difficult to master if speech-sound duration plays a less important role in the phonology of their native language (L1). The present studies aimed to investigate the cortical representations for phonological quantity in native speakers and L2 users of Finnish by using behavioral and electrophysiological methods. Since long-term memory representations for different speech units have been previously shown to participate in the elicitation of the mismatch negativity (MMN) brain response, MMN was used to compare the neural representation for quantity between native speakers and L2 users of Finnish. The results of the studies suggested that native Finnish speakers' MMN response to quantity was determined by the activation of native-language phonetic prototypes rather than by phoneme boundaries. In addition, native speakers seemed to process phoneme quantity and quality independently from each other by separate brain representations. The cross-linguistic MMN studies revealed that, in native speakers of Finnish, the MMN response to duration or quantity-degree changes was enhanced in amplitude selectively in speech sounds, whereas this pattern was not observed in L2 users. Native speakers' MMN enhancement is suggested to be due to the pre-attentive activation of L1 prototypes for quantity. In L2 users, the activation of L2 prototypes or other L2 learning effects were not reflected in the MMN, with one exception. Even though L2 users failed to show native-like brain responses to duration changes in a vowel that was similar in L1 and L2, their duration MMN response was native-like for an L2 vowel with no counterpart in L1. Thus, the pre-attentive activation of L2 users' representations was determined by the degree of similarity of L2 sounds to L1 sounds. In addition, behavioral experiments suggested that the establishment of representations for L2 quantity may require several years of language exposure. Eri kielet käyttävät äänteiden kestovihjeitä erilaisissa kielellisissä tehtävissä. Suomessa äänteen kesto on kvantiteetin eli lyhyiden ja pitkien foneemien erottamisen tärkein vihje. Suomea toisena kielenä puhuvien on usein vaikea oppia kvantiteetti, jos äänteen kestolla on vähäisempi merkitys heidän äidinkielessään. Tämän väitöskirjan tutkimuksissa tarkasteltiin aivovastemittausten ja foneettisten testien avulla kvantiteettikategorioiden muistijälkiä äidinkieleltään suomenkielisten ja suomea toisena kielenä puhuvien aivokuorella. Koska äänteiden pitkäkestoisten muistijälkien on aiemmin todettu osallistuvan MMN-aivovasteen (engl. mismatch negativity) syntymiseen, sitä käytettiin selvittämään, miten äidinkieleltään suomenkielisten ja suomea toisena kielenä puhvien kvantiteettikategorioiden muistijäljet poikkeavat toisistaan. Tulosten mukaan äidinkieleltään suomenkielisten MMN-vasteen voimakkuus määräytyi äidinkielen foneettisten prototyyppien mukaan eikä foneemien rajojen mukaan. Lisäksi suomenkieliset näyttivät käsittelevän foneemin laadun ja kvantiteetin toisistaan riippumattomasti, eri kategorioiden kautta. Kieliryhmiä vertailevat MMN-tutkimukset puolestaan osoittivat, että äidinkieleltään suomenkielisillä MMN-vaste keston muutokselle oli nimenomaan puheäänteiden kohdalla voimakkaampi kuin toisen kielen oppijoilla. Tämä saattaa johtua foneettisten prototyyppien esitietoisesta aktivoitumisesta äidinkieleltään suomenkielisten aivokuorella. Toisen kielen oppijoilla toisen kielen prototyyppien aktivoituminen tai kielenoppiminen ylipäänsä eivät näkyneet MMN-vasteessa yhtä poikkeusta lukuun ottamatta. Vaikka suomea toisena kielenä puhuvien aivovasteiden voimakkuus äänteen keston muutokselle ei saavuttanut äidinkielisten tasoa vokaalissa, joka oli samankaltainen heidän äidinkielessään ja toisessa kielessään, he saavuttivat suomenkielisten tason vokaalissa, jolla ei ollut vastinetta heidän äidinkielessään. Näin ollen äidinkielen ja toisen kielen äänteiden samankaltaisuus näyttäisi vaikuttavan suomea toisena kielenä puhuvien muistijälkien esitietoiseen aktivoitumiseen. Tutkimuksen foneettisten kokeiden mukaan kategorioiden syntyminen toisen kielen kvantiteetille saattaa vaatia useiden vuosien altistusta toiselle kielelle.