Search

The current study investigates the average effect: the tendency for humans to appreciate an averaged (face, bird, wristwatch, car, and so on) over an individual instance. The effect holds across cultures, despite varying conceptualizations of attractiveness. While much research has been conducted on the average effect in visual perception, much less is known about the extent to which this effect applies to language and speech. This study investigates the attractiveness of average speech rhythms in Dutch and Mandarin Chinese, two typologically different languages. This was tested in a series of perception experiments in either language in which native listeners chose the most attractive one from a pair of acoustically manipulated rhythms. For each language, two experiments were carried out to control for the potential influence of the acoustic manipulation on the average effect. The results confirm the average effect in both languages, and they do not exclude individual variation in the listeners' perception of attractiveness. The outcomes provide a new crosslinguistic perspective and give rise to alternative explanations to the average effect. [ABSTRACT FROM AUTHOR]

This paper presents the results of three perceptual experiments investigating the role of auditory and visual channels for the identification of statements and echo questions in Brazilian Portuguese. Ten Brazilian speakers (five male) were video-recorded (frontal view of the face) while they produced a sentence (" Como você sabe "), either as a statement (meaning " As you know. ") or as an echo question (meaning " As you know ?"). Experiments were set up including the two different intonation contours. Stimuli were presented in conditions with clear and degraded audio as well as congruent and incongruent information from both channels. Results show that Brazilian listeners were able to distinguish statements and questions prosodically and visually, with auditory cues being dominant over visual ones. In noisy conditions, the visual channel improved the interpretation of prosodic cues robustly, while it degraded them in conditions where the visual information was incongruent with the auditory information. This study shows that auditory and visual information are integrated during speech perception, also when applied to prosodic patterns. [ABSTRACT FROM AUTHOR]

Speech perception is a multisensory process: what we hear can be affected by what we see. For instance, the McGurk effect occurs when auditory speech is presented in synchrony with discrepant visual information. A large number of studies have targeted the McGurk effect at the segmental level of speech (mainly consonant perception), which tends to be visually salient (lip-reading based), while the present study aims to extend the existing body of literature to the suprasegmental level, that is, investigating a McGurk effect for the identification of tones in Mandarin Chinese. Previous studies have shown that visual information does play a role in Chinese tone perception, and that the different tones correlate with variable movements of the head and neck. We constructed various tone combinations of congruent and incongruent auditory-visual materials (10 syllables with 16 tone combinations each) and presented them to native speakers of Mandarin Chinese and speakers of tone-naïve languages. In line with our previous work, we found that tone identification varies with individual tones, with tone 3 (the low-dipping tone) being the easiest one to identify, whereas tone 4 (the high-falling tone) was the most difficult one. We found that both groups of participants mainly relied on auditory input (instead of visual input), and that the auditory reliance for Chinese subjects was even stronger. The results did not show evidence for auditory-visual integration among native participants, while visual information is helpful for tone-naïve participants. However, even for this group, visual information only marginally increases the accuracy in the tone identification task, and this increase depends on the tone in question. [ABSTRACT FROM AUTHOR]

A central problem in recent research on speech production concerns the question to what extent speakers adapt their linguistic expressions to the needs of their addressees. It is claimed that speakers sometimes leak information about objects that are only visible for them and not for their listeners. Previous research only takes the occurrence of adjectives as evidence for the leakage of privileged information. The present study hypothesizes that leaked information is also encoded in the prosody of those adjectives. A production experiment elicited adjectives that leak information and adjectives that do not leak information. An acoustic analysis and prominence rating task showed that adjectives that leak information were uttered with a higher pitch and perceived as more prominent compared to adjectives that do not leak information. Furthermore, a guessing task suggested that the adjectives’ prosody relates to how listeners infer possible privileged information. [ABSTRACT FROM PUBLISHER]

We studied the effect of two social settings (collaborative versus competitive) on the visual and auditory expressions of uncertainty by children in two age groups (8 and 11). We conducted an experiment in which children played a quiz game in pairs. They either had to collaborate or compete with each other. We found that the Feeling-of-Knowing of eight-year-old children did not seem to be affected by the social setting, contrary to the Feeling-of-Knowing of 11-year-old children. In addition, we labelled children’s expressions in clips taken from the experiment for various visual and auditory features. We found that children used some of these features to signal uncertainty and that older children exhibited clearer cues than younger children. In a subsequent perception test, adults rated children’s certainty in clips used for labelling. It appeared that older children and children in competition expressed their confidence level more clearly than younger children and children in collaboration. [ABSTRACT FROM AUTHOR]

In this article we report on two experiments about the perception of audiovisual cues to emotional speech. The article addresses two questions: (1) how do visual cues from a speaker's face to emotion relate to auditory cues, and (2) what is the recognition speed for various facial cues to emotion? Both experiments reported below are based on tests with video clips of emotional utterances collected via a variant of the well-known Velten method. More specifically, we recorded speakers who displayed positive or negative emotions, which were congruent or incongruent with the (emotional) lexical content of the uttered sentence. In order to test this, we conducted two experiments. The first experiment is a perception experiment in which Czech participants, who do not speak Dutch, rate the perceived emotional state of Dutch speakers in a bimodal (audiovisual) or a unimodal (audio- or vision-only) condition. It was found that incongruent emotional speech leads to significantly more extreme perceived emotion scores than congruent emotional speech, where the difference between congruent and incongruent emotional speech is larger for the negative than for the positive conditions. Interestingly, the largest overall differences between congruent and incongruent emotions were found for the audio-only condition, which suggests that posing an incongruent emotion has a particularly strong effect on the spoken realization of emotions. [ABSTRACT FROM AUTHOR]

The article discusses various reports published within the issue including one by Scarborough, Keating, Mattys, Cho, Alwan, and Auer, Jr. about which visual cues contribute to the perception of lexical and phrasal stress, one by Dohen and Loevenbruck on determining prosodic focus and its audiovisual realization in French, and another by Guaïtella, Santi, Lagure, and Cavé on the function of rapid eyebrow movements in spoken French.

We describe two experiments on signaling and detecting uncertainty in audiovisual speech by adults and children. In the first study, utterances from adult speakers and child speakers (aged 7-8) were elicited and annotated with a set of six audiovisual features. It was found that when adult speakers were uncertain they were more likely to produce fillers, delays, high intonation, eyebrow movements, and "funny faces." The basic picture for the child speakers was somewhat similar, in that the presence of certain audiovisual cues correlated with uncertainty, but the differences were relatively small and less often significant. In the second study both adult and child judges watched responses from adult and child speakers selected from the first study to find out whether they were able to correctly estimate a speakers' level of uncertainty. It was found that both child and adult judges gave more accurate scores for answers from adult speakers than from child speakers and that child judges overall provided less accurate scores than adult judges. [ABSTRACT FROM AUTHOR]

While considerable research has been done in the past in prosody and conversation, work which combines the two has been lamentably scarce. Yet, to gain real insight into basic characteristics of either phenomena, most researchers would agree that the study of prosody is critical to the understanding of how people converse, and the study of conversational behavior is equally crucial to the understanding of prosodic phenomena outside the laboratory. On the one hand, prosody is known to be a flexible structuring device, which may signal different aspects of dialog structure, such as turn-taking, topic structure. information status, type of speech act, attitudinal and emotional characteristics of utterances, and mutual beliefs of dialog participants. Obviously, for students of prosodic function, the conversational domain should provide a fertile area of research. On the other hand, one intriguing fact about conversation is that it normally proceeds in a smooth and successful manner. Conversational analysts may find new answers to why natural spoken communication works so well by examining the prosodic as well as the textual features of their data.

The article presents information on a study which discusses prosody as a marker of information flow in spoken discourse. The analysis of spontaneous discourse prosody both from the speaker's and the listener's point of view, shows the following specific results. With regard to the speaker, it was found that larger-scale information units are indeed phonetically encoded in pausal and melodic properties. A speaker may separate two successive units by the use of a long pause. At any rate, it can be concluded that speakers use local prosodic devices to signal the boundaries of a discourse unit. From the listener's point of view, global melodic features, having a whole discourse unit as their domain, are likely to be very useful, as they may enable him/her to predict an upcoming ending.