Search

Introduction: As scholarly interest in metaphorical gestures has burgeoned in recent years, the exploration of these non-verbal communicative cues has primarily centered on domains, such as speeches, advertisements, and movies. Surprisingly, there exists a notable dearth of research specifically delving into the nuanced role of metaphorical gestures within the realm of English teaching, especially in middle school settings. This study sought to address this gap by strategically selecting two nationally recognized English courses taught in middle school English classrooms. The study aimed to explore the role of multimodal metaphors in gestures and provide some inspiration for middle school English teachers. Methodology: In this research, two videos from National Excellent Courses were selected for analysis using a multimodal annotation tool, namely EUDICO Linguistic Annotator (ELAN). The course selected was the middle school English teaching design course, and the students involved were from Beijing Normal University, China. All gestures analyzed were classified into three types of metaphors, namely ontological, structural, and orientational. Results: The findings indicated that ontological metaphors occurred most frequently, followed by orientational metaphors, with structural metaphors being the least frequent This research demonstrated the cognitive process of conceptual metaphor in classroom gestures with concrete examples. Conclusion: In contemporary middle school English classrooms, relying solely on textual material is insufficient for fully supporting the teaching process. This study substantiates that employing multimodal metaphors in gestures can enhance teachers' ability to motivate students effectively and facilitate improved interaction and semantic communication. The study provides English teachers with valuable guidance on implementing multimodal teaching strategies utilizing metaphorical gestures. [ABSTRACT FROM AUTHOR]

Purpose: The heterogeneity of autism is well documented, but few studies have studied the heterogeneity of gesture production ability in autistic children. The present study aimed to identify subgroups of autistic children who displayed heterogeneous gesture production abilities and explore the underlying factors, including autism characteristics, intellectual ability, and language ability, that were associated with the heterogeneity., Methods: A total of 65 Chinese autistic children (mean age = 5;3) participated. Their autism characteristics and intellectual ability were assessed by standardized measurements. Language output and gesture production were captured from a parent-child interaction task., Results: We conducted a hierarchical cluster analysis and identified four distinct clusters. Cluster 1 and Cluster 2 both had low gesture production whereas Cluster 3 and Cluster 4 had high gesture production. Both Clusters 1 and 2 had relatively strong autism characteristics, in comparison to Clusters 3 and 4., Conclusions: Our findings revealed that children with stronger autism characteristics may gesture less often than those with weaker characteristics. However, the relationship between language ability and intellectual ability and gesture production was not clear. These findings shed light on the directions of intervention on gesture production for autistic children, especially those with stronger autism characteristics., (© 2023. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.)

An instructor's gestures have an effect not only on students' learning but also on teaching itself. In two experiments, this study tested whether instructors' use of gestures while recording video lectures affected their teaching performance, stress, and cognitive load. In Experiment 1, participants recorded video lectures with gestures or without gestures. As hypothesized, t tests showed that participants in the gestures condition had better teaching performance and lower cognitive load than participants in the no‐gestures condition, whereas there was no effect on stress level. In Experiment 2, participants recorded video lectures with either pointing gestures or representational gestures. The t tests indicated that participants in the pointing gestures condition showed better teaching performance and experienced lower stress than those in the representational gestures condition, but there was no difference in cognitive load. Overall, our findings suggest that in the new educational environment of video lectures, instructors should consider using gestures, especially pointing gestures, to improve their teaching and their experience of teaching. Lay Description: What is already known about this topic Instructors' gestures have benefits on learners' learning.Different types of gestures have different effect.Gestures can affect speaker's cognition. What this paper adds This study explored the effect of whether instructors' gestures and different types of gestures (pointing gestures and representational gestures) during recording instructional videos on their teaching.Instructors' gestures improved their teaching performance, induced cognitive load, and enhanced teaching efficacy.Instructors' pointing gestures were outperformed than representational gestures in teaching performance and stress level. Implications for practice and/or policy Instructors should consider using gestures when recording instructional videos, especially pointing gestures. [ABSTRACT FROM AUTHOR]

Previous research has shown language-specific features play a guiding role in how children develop expression of events with speech and gestures. This study adopts a multimodal approach and examines Mandarin Chinese, a language that features context use and verb serializations. Forty children (four-to-seven years old) and ten adults were asked to describe fourteen video stimuli depicting different types of causal events involving location/state changes. Participants' speech was segmented into clauses and co-occurring gestures were analyzed in relation to causation. The results show that the older the children, the greater the use of contextual clauses which contribute meaning to event descriptions. It is not until the age of six that children used adult-like structures - namely, using single gestures representing causing actions and aligning them with verb serializations in single clauses. We discuss the implications of these findings for the guiding role of language specificity in multimodal language development.

Sign language (SL) has strong structural features. Various gestures and the complex trajectories of hand movements bring challenges to sign language recognition (SLR). Based on the inherent correlation between gesture and trajectory of SL action, SLR is organically divided into gesture-based recognition and gesture-related movement trajectory recognition. One hundred and twenty commonly used Chinese SL words involving 9 gestures and 8 movement trajectories, are selected as research and test objects. The method based on the amplitude state of surface electromyography (sEMG) signal and acceleration signal is used for vocabulary segmentation. The multi-sensor decision fusion method of coupled hidden Markov model is used to complete the recognition of SL vocabulary, and the average recognition rate is 90.41%. Experiments show that the method of sEMG signal and motion information fusion has good practicability in SLR., Competing Interests: The authors have declared that no competing interests exist., (Copyright: © 2023 Li et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Purpose: Children with autism are found to have delayed and heterogeneous gesture abilities. It is important to understand the growth of gesture abilities and the underlying factors affecting its growth. Addressing these issues can help to design effective intervention programs., Method: Thirty-five Chinese-speaking preschoolers with autism spectrum disorder ( M age = 0.91; four girls) participated in four play sessions with their parents over 9 months. Their child-based factors including autism severity, intellectual functioning, and expressive language abilities were assessed. The gestures (deictic, iconic, and conventional) of the children and their parents were coded. Growth curve analyses were conducted to examine individual growth trajectories and the roles of child-based factors and parental input in shaping the children's gesture development.SD = 0.91; four girls) participated in four play sessions with their parents over 9 months. Their child-based factors including autism severity, intellectual functioning, and expressive language abilities were assessed. The gestures (deictic, iconic, and conventional) of the children and their parents were coded. Growth curve analyses were conducted to examine individual growth trajectories and the roles of child-based factors and parental input in shaping the children's gesture development., Results: Child-based factors and parental input predicted gesture development differently. Parents' gestures positively predicted their children's gestures of the same type. Autism severity negatively predicted iconic and conventional gestures. Overall growth was found in deictic rather than iconic and conventional gestures. Subgroup variation was also found. Specifically, children with better expressive language ability showed a decrease in deictic gestures. An increase in iconic and conventional gestures was found in children with more severe autism and those with poorer expressive language ability and intellectual functioning, respectively., Conclusions: Different types of gestures may have different growth trajectories and be predicted by different child-based factors. Particular attention should be given to children who never produced iconic gestures, which is more challenging and may not develop over a short period, and hence require direct instruction.

Abstract: When children learn language, they apply their language-learning skills to the linguistic input they receive. But what happens if children are not exposed to input from a conventional language? Do they engage their language-learning skills nonetheless, applying them to whatever unconventional input they have? We address this question by examining gesture systems created by four American and four Chinese deaf children. The children’s profound hearing losses prevented them from learning spoken language, and their hearing parents had not exposed them to sign language. Nevertheless, the children in both cultures invented gesture systems that were structured at the morphological/word level. Interestingly, the differences between the children’s systems were no bigger across cultures than within cultures. The children’s morphemes could not be traced to their hearing mothers’ gestures; however, they were built out of forms and meanings shared with their mothers. The findings suggest that children construct morphological structure out of the input that is handed to them, even if that input is not linguistic in form. [Copyright &y& Elsevier]

• A distinct methodology using two different but complementary models is used to analyze gestures by teachers in elementary Chinese literacy classrooms. • Teachers use deictic and iconic gestures to draw attention to specific features of Chinese characters. • Gestures and language center on concrete entities and attributes. • Gestures reflect a focus on sound-character correspondence. • Teachers' gestures reflect a recognition literacy pedagogy, to fulfil the aims of the Chinese curriculum guidelines. This article presents an analysis and interpretation of teachers' gestures in elementary literacy classrooms in China. Our underlying questions are: what types of gestures are used and for what purposes are they used, and how are gestures distributed throughout the process of teaching children to read and write Chinese characters? Our overall aim is to better understand the communicative learning context in which young children enter school and the ways in which certain types of gestures are incorporated into and at the same time reflect the pedagogical approach taken. Our distinctive analytical approach uses two different but complementary frameworks and perspectives on gestures. Our results show that the gestures and meanings conveyed through the gestures, reflect a 'recognition' pedagogy in which the sound-form correspondence and specific features of a required set of characters for reading and for writing are focused on. The results also highlight the 'here-and-now' nature of the teaching strategies, concrete gesture-linguistic meanings, and gestures associated with a traditional classroom arrangement. [ABSTRACT FROM AUTHOR]

It has been suggested that mature reading systems consist of both a visual analysis decoding system and a motor gesture decoding system that facilitates reading by processing writing motor information. However, there is still uncertainty about the mechanisms and effectiveness of the latter system when reading Chinese characters. This study therefore aimed to provide empirical neural evidence for this phenomenon using a writing video as prompt in a delayed sequential same/different judgment task. We investigated whether and how the orthographic processing of target characters was modulated by the writing direction of the prompt (forward vs. backward) and the character repetition (repeat vs. nonrepeat) between the prompt and target characters. The results indicated that (a) the N170 component was more negative under the forward condition than under the backward condition; and (b) both writing direction and character repetition modulated the centro-parietal N200 component. These writing-direction effects were specific to Chinese characters--they did not apply to Korean characters, which are visually similar to Chinese characters but unfamiliar to the participants. These results suggest that the experience of learning Chinese might establish a motor gesture decoding system for reading, which begins to perform general orthographic representation at an early stage and works together with the visual analysis decoding system to achieve deep orthographic processing., (© 2020 Society for Psychophysiological Research.)

In this paper, deep belief net (DBN) was applied into the field of wearable-sensor based Chinese sign language (CSL) recognition. Eight subjects were involved in the study, and all of the subjects finished a five-day experiment performing CSL on a target word set consisting of 150 CSL subwords. During the experiment, surface electromyography (sEMG), accelerometer (ACC), and gyroscope (GYRO) signals were collected from the participants. In order to obtain the optimal structure of the network, three different sensor fusion strategies, including data-level fusion, feature-level fusion, and decision-level fusion, were explored. In addition, for the feature-level fusion strategy, two different feature sources, which are hand-crafted features and network generated features, and two different network structures, which are fully-connected net and DBN, were also compared. The result showed that feature level fusion could achieve the best recognition accuracy among the three fusion strategies, and feature-level fusion with network generated features and DBN could achieve the best recognition accuracy. The best recognition accuracy realized in this study was 95.1% for the user-dependent test and 88.2% for the user-independent test. The significance of the study is that it applied the deep learning method into the field of wearable sensors-based CSL recognition, and according to our knowledge it's the first study comparing human engineered features with the network generated features in the correspondent field. The results from the study shed lights on the method of using network-generated features during sensor fusion and CSL recognition.

The temporal-focus hypothesis claims that whether people conceptualize the past or the future as in front of them depends on their cultural attitudes toward time; such conceptualizations can be independent from the space-time metaphors expressed through language. In this paper, we study how Chinese people conceptualize time on the sagittal axis to find out the respective influences of language and culture on mental space-time mappings. An examination of Mandarin speakers' co-speech gestures shows that some Chinese spontaneously perform past-in-front/future-at-back (besides future-in-front/past-at-back) gestures, especially when gestures are accompanying past-in-front/future-at-back space-time metaphors (Exp. 1). Using a temporal performance task, the study confirms that Chinese can conceptualize the future as behind and the past as in front of them, and that such space-time mappings are affected by the different expressions of Mandarin space-time metaphors (Exp. 2). Additionally, a survey on cultural attitudes toward time shows that Chinese tend to focus slightly more on the future than on the past (Exp. 3). Within the Chinese sample, we did not find evidence for the effect of participants' cultural temporal attitudes on space-time mappings, but a cross-cultural comparison of space-time mappings between Chinese, Moroccans, and Spaniards provides strong support for the temporal-focus hypothesis. Furthermore, the results of Exp. 2 are replicated even after controlling for factors such as cultural temporal attitudes and age (Exp. 3), which implies that linguistic sagittal temporal metaphors can indeed influence Mandarin speakers' space-time mappings. The findings not only contribute to a better understanding of Chinese people's sagittal temporal orientation, but also have additional implications for theories on the mental space-time mappings and the relationship between language and thought., (© 2019 The Authors. Cognitive Science published by Wiley Periodicals, Inc. on behalf of Cognitive Science Society (CSS).)

Sign language recognition (SLR) is an important communication tool between the deaf and the external world. It is highly necessary to develop a worldwide continuous and large-vocabulary-scale SLR system for practical usage. In this paper, we propose a novel phonology- and radical-coded Chinese SLR framework to demonstrate the feasibility of continuous SLR using accelerometer (ACC) and surface electromyography (sEMG) sensors. The continuous Chinese characters, consisting of coded sign gestures, are first segmented into active segments using EMG signals by means of moving average algorithm. Then, features of each component are extracted from both ACC and sEMG signals of active segments (i.e., palm orientation represented by the mean and variance of ACC signals, hand movement represented by the fixed-point ACC sequence, and hand shape represented by both the mean absolute value (MAV) and autoregressive model coefficients (ARs)). Afterwards, palm orientation is first classified, distinguishing "Palm Downward" sign gestures from "Palm Inward" ones. Only the "Palm Inward" gestures are sent for further hand movement and hand shape recognition by dynamic time warping (DTW) algorithm and hidden Markov models (HMM) respectively. Finally, component recognition results are integrated to identify one certain coded gesture. Experimental results demonstrate that the proposed SLR framework with a vocabulary scale of 223 characters can achieve an averaged recognition accuracy of 96.01% ± 0.83% for coded gesture recognition tasks and 92.73% ± 1.47% for character recognition tasks. Besides, it demonstrats that sEMG signals are rather consistent for a given hand shape independent of hand movements. Hence, the number of training samples will not be significantly increased when the vocabulary scale increases, since not only the number of the completely new proposed coded gestures is constant and limited, but also the transition movement which connects successive signs needs no training samples to model even though the same coded gesture performed in different characters. This work opens up a possible new way to realize a practical Chinese SLR system.

Daily activities of forty-eight 8- to 15-month-olds and their interlocutors were observed to test for the presence and frequency of triadic joint actions and deictic gestures across three different cultures: Yucatec-Mayans (Mexico), Dutch (Netherlands), and Shanghai-Chinese (China). The amount of joint action and deictic gestures to which infants were exposed differed systematically across settings, allowing testing for the role of social-interactional input in the ontogeny of prelinguistic gestures. Infants gestured more and at an earlier age depending on the amount of joint action and gestures infants were exposed to, revealing early prelinguistic sociocultural differences. The study shows that the emergence of basic prelinguistic gestures is socially mediated, suggesting that others' actions structure the ontogeny of human communication from early on., (© 2012 The Authors. Child Development © 2012 Society for Research in Child Development, Inc.)

Do the neural circuits for reading vary across culture? Reading of visually complex writing systems such as Chinese has been proposed to rely on areas outside the classical left-hemisphere network for alphabetic reading. Here, however, we show that, once potential confounds in cross-cultural comparisons are controlled for by presenting handwritten stimuli to both Chinese and French readers, the underlying network for visual word recognition may be more universal than previously suspected. Using functional magnetic resonance imaging in a semantic task with words written in cursive font, we demonstrate that two universal circuits, a shape recognition system (reading by eye) and a gesture recognition system (reading by hand), are similarly activated and show identical patterns of activation and repetition priming in the two language groups. These activations cover most of the brain regions previously associated with culture-specific tuning. Our results point to an extended reading network that invariably comprises the occipitotemporal visual word-form system, which is sensitive to well-formed static letter strings, and a distinct left premotor region, Exner's area, which is sensitive to the forward or backward direction with which cursive letters are dynamically presented. These findings suggest that cultural effects in reading merely modulate a fixed set of invariant macroscopic brain circuits, depending on surface features of orthographies.