Search

This practice-based research project explores, defines and demonstrates the state of spatio-temporal suspension, or suspended experience, where abstract characteristics of sound and light in music and imagery coalesce to afford audience and performer the experience of liminal or “between” aesthetic zones, in turn providing a gateway into imaginative worlds. Informed by the author’s background in the performance and composition of forms of experimental popular music as well as graphic design and photography, the investigation utilises a combination of creative research methods and research-led analysis within a phenomenological framework to interrogate the physiological and neurological basis for this state. In order to better understand and define this concept, relevant creative exemplars including music and music videos, experimental films and site-specific installations are examined. The analysis draws upon a range of relevant philosophies and theories of perception relating to time and space, including phenomenology, liminality, the Japanese concept of Ma and heterotopia. Perceptual and psychological theories, including affect as felt experience and its role in aesthetics are considered, as well as embodied cognition in aesthetics and ecological approaches to perception. These theories and concepts consider humans as integral parts of a dynamic ecosystem of individual and shared information and perception, providing insight into the perceptual basis of suspension and why it is often encountered as a cross-modal experience. Through the analysis of creative works and the author’s self-observation and journaling of the audiovisual exploration of suspension in performance and practice, the research identifies compositional features that are prevalent in works that facilitate and enhance the potential for suspended experience. These features are explored and realised through creative works that examine how suspension is imparted through an audiovisual composition, how it is experienced by the practitioner through the recording process and as an improvised performance, and how the works are received and experienced by others, via examination of responses to specially designed reception tests. The findings are expressed in the conception and realisation of two major bodies of work: an audiovisual suite, Suspension Studies (2020), comprised of musical and visual studies of suspension as an immobile work; and a site-specific performance work, States of Suspension (2018), which affords audiences and performers an active aesthetic experience of suspension in situ. The enquiry contributes to the understanding of a relatively unexplored phenomenon in music and visual arts and intends to encourage further discourse and investigation into this topic.

This thesis comprises a body of work that investigates affect perception of unfamiliar music, with a focus on both the role of potentially culture-independent psychoacoustic features that are intrinsic to a musical signal (e.g. roughness, harmonicity, spectral entropy, and average pitch) and extrinsic culture-dependent features (e.g. familiarity through exposure and evaluative conditioning). Much previous research in music perception has suggested that extrinsic features are of more importance than intrinsic features, but has not systematically tested the impact of intrinsic features on responses to unfamiliar music. The thesis discusses four experiments conducted to test the role of the above mentioned features using musical stimuli that are unfamiliar to participants. By using musical stimuli that are unfamiliar to participants, additional evidence can be provided for the cultural- independence of the tested intrinsic features. In order to achieve this unfamiliarity, two approaches were used. The first approach examined affective responses to chords from the unfamiliar microtonal Bohlen-Pierce system in Western listeners, the second approach tested affective responses to Western musical harmony in remote villages in Papua New Guinea, with varying levels of familiarity with Western music. The results of the listening experiments using Bohlen-Pierce suggest that the tested underlying culture-independent psychoacoustic features consistently impact affective rat- ings more strongly than do the experimentally manipulated culture-dependent factors of familiarity and evaluative conditioning. The results from the cross-cultural experiment suggest a strong role of familiarity on valence ratings of Western cadences and melodies. In summary, by using unfamiliar music (through the use of an unfamiliar microtonal system or through cross-cultural research) we can show that, in addition to extrinsic culture-dependent features, intrinsic features are fundamental for affect perception in music.

Music is ubiquitous across all human cultures. It is hypothesised that the development of music and of language in human evolution is linked (Wallin et al., 2001), and music, in addition to language, is known to be communicative. One way music – particularly music employing the widely used system of tonality – communicates is through tension and resolution, or stability and instability, where instability is the need to resolve and stability its destination. Most tonal-harmonic music today exists in a Western tuning system and experimental research into the perception of harmonic tonality is conducted almost entirely in 12-TET. This project is the first empirical study of the cognition of harmonic tonality in microtonal scales. Through the employment of novel scales in an unfamiliar tuning system, effects of familiarity are weakened, allowing a more focussed investigation of other effects. Particularly, bottom-up models for the cognition of harmonic tonality are allowed a more careful investigation, providing valuable insight into the cognition of music otherwise beyond reach. This research also provides valuable information for hopeful composers of novel music in shaping their music to elicit a desired response, thus enabling expansion of the palette of possible musical expression. This project utilizes a common experimental paradigm for research into the cognition of tonality: participants are first played context-setting stimuli, after which a probe tone or chord is sounded and they are asked to rate how well the probe tone “fits” the context, or how stable it is given the context. A psychoacoustic feature – spectral pitch class similarity – is used to predict the perceived stability of pitch classes and triads of not only familiar scales (Experiment 1), but unfamiliar (Experiment 2), and novel scales (Experiments 3-5), where models of long-term statistical learning are available only for familiar scales. Through a series of 5 experiments the perceived stability of tones and triads in novel, microtonal scales is predicted, demonstrating the usefulness of our psychoacoustic model.

When listening to music, people across cultures tend to spontaneously perceive and move the body along a periodic pulse-like meter. Increasing evidence suggests that this ability is supported by neural mechanisms that selectively amplify periodicities corresponding to the perceived metric pulses. However, the nature of these neural mechanisms, i.e., the endogenous or exogenous factors that may selectively enhance meter periodicities in brain responses to rhythm, remains largely unknown. This question was investigated in a series of studies in which the electroencephalogram (EEG) of healthy participants was recorded while they listened to musical rhythm. From this EEG, selective contrast at meter periodicities in the elicited neural activity was captured using frequency-tagging, a method allowing direct comparison of this contrast between the sensory input, EEG response, biologically-plausible models of auditory subcortical processing, and behavioral output. The results show that the selective amplification of meter periodicities is shaped by a continuously updated combination of factors including sound spectral content, long-term training and recent context, irrespective of attentional focus and beyond auditory subcortical nonlinear processing. Together, these observations demonstrate that perception of rhythm involves a number of processes that transform the sensory input via fixed low-level nonlinearities, but also through flexible mappings shaped by prior experience at different timescales. These higher-level neural mechanisms could represent a neurobiological basis for the remarkable flexibility and stability of meter perception relative to the acoustic input, which is commonly observed within and across individuals. Fundamentally, the current results add to the evidence that evolution has endowed the human brain with an extraordinary capacity to organize, transform, and interact with rhythmic signals, to achieve adaptive behavior in a complex dynamic environment.

Although music performance is generally thought of as an auditory activity in the Western tradition, the presence of continuous visual information in live music contributes to the cohesiveness of music ensembles, which presents an interesting psychological phenomenon in which audio and visual cues are presumably integrated. In order to investigate how auditory and visual sensory information are combined in the basic process of synchronising movements with music, this thesis focuses on both musicians and nonmusicians as they respond to two sources of visual information common to ensembles: the conductor, and the ancillary movements (movements that do not directly create sound; e.g. body sway or head nods) of co-performers. These visual cues were hypothesized to improve the timing of intentional synchronous action (matching a musical pulse), as well as increasing the synchrony of emergent ancillary movements between participant and stimulus. The visual cues were tested in controlled renderings of ensemble music arrangements, and were derived from real, biological motion. All three experiments employed the same basic synchronisation task: participants drummed along to the pulse of tempo-changing music while observing various visual cues. For each experiment, participants’ drum timing and upper-body movements were recorded as they completed the synchronisation task. The analyses used to quantify drum timing and ancillary movements came from theoretical approaches to movement timing and entrainment: information processing and dynamical systems. Overall, this thesis shows that basic musical timing is a common ability that is facilitated by visual cues in certain contexts, and that emergent ancillary movements and intentional synchronous movements in combination may best explain musical timing and synchronisation.

I approach timbre here primarily from the perspective of the music creator/producer, and the processes and considerations they apply in order to make recordings. Over the course of this thesis, I build a new synthesiser, whose sound manipulations (FX) are modelled after what would be typically used in a compositional or audio mixing context. These FX are, themselves, treated as the timbre descriptors. In the first experiment I try to establish units for these FX as measured in steps of smallest perceivable change of said parameter; the perception threshold (PT). This unit makes timbral changes of any kind comparable on a perceptual foundation and also relates back to manipulation features that are directly accessible to composers/mixers/musicians. A noteworthy change in this research, in comparison with the more established just-noticeable difference (JND), is that the PT is measured for change in continuous sound, rather than for sets of separate, isolated sounds as used for JNDs. This again mirrors the applicability to musical processes, where sound change is often evoked gradually over periods of time, rather than always being distinct to isolated pieces of sound. With this specific attribute of continuous sound, I test the impact on PT for changing FX gradually over a set amount of time, rather than abruptly, the time-frame for the change is here denoted transition time (TT). Following that, I explore how these FX are used by inexpert participants when they are given the task of mixing musical example pieces using the established FX. Also measured is how they rate, post-task, their own efforts when compared to both standardised and random results of the same task, or an expert audio engineer’s resulting mix. Lastly I compare the impact of both the FX and more standard audio descriptors (see sections 2.2 – 2.3) on participants’ preferences for different mixes, as well as their relations to each other. Finally I use the insight gained in the previous experiments, to create a piece of music. The creative process is primarily informed by applying the established PTs to a musical context, which in itself is created almost exclusively by the synthesiser used in the experiments. Potential research questions that could arise from these efforts are briefly discussed as well.

Music exhibits remarkable properties in the context of memory. For example, memory for melodies is long lasting, persistent, and spared from some forms of dementia and severe brain injuries. Over the course of 10 experiments, we here attempt to shed light on the question of what makes music ‘special’ by specifically investigating interference in memory for melody. Most domains show a decrease in recognition performance between the first and second presentation of an object as the number of intervening items increases. We tested this cumulative disruptive interference effect in the context of memory for melodies, with results showing that memory for melodies is not much affected by the number of intervening items. Specifically, the probability of correctly recognising a melody was statistically identical and above chance between 1 and up to nearly 200 intervening melodies. To explain these findings, we provide a new Regenerative-Multiple-Representations (RMR) conjecture. The conjecture describes a crucial link between prior experience, perception, and subsequent formation of memories. Using the theoretical framework of this conjecture, we further explored memory for melodies in a series of experiments. In the process, we revealed how to disrupt and shape memory for melodies’ resilience to cumulative interference using melodies in an unfamiliar tuning system and with pitch-only and rhythm-only sequences. In a final analysis, we used memory as a window into perception in more general terms, and analysed data from all experiments combined to measure the degree of similarity in listeners’ perception of music. The findings of this dissertation contribute to our understanding of fundamental memory phenomena, while providing practical implementations and elucidating further the mechanisms that explain why music may indeed be ‘special’.

Language and music are two human universals that share many commonalities, including processes of statistical and distributional learning in acquiring knowledge of those domains. This thesis is concerned with the role of distributional learning in the acquisition of pitch-based building blocks of speech and music. In a series of five studies, questions of theoretical and empirical interest will be examined, whether: (i) distributional learning can be used to acquire lexical tone and musical pitch; (ii) domain-general or domain-specific pitch experience facilitates distributional learning of pitch; and (iii) distributional learning plays a role in cross-domain transfer. The results of all five studies suggest that distributional learning can be used to acquire the foundations of speech and music; using distributional learning, adult learners either shift existing category boundaries to which the perceptual items assimilate or form new categories if the perceptual items are not assimilated to any native (linguistic or musical) categories. While distributional learning appears to be sensitive to top-down interferences and is modulated by domain-specific experience, it is nonetheless a powerful learning mechanism that is generalisable across domain. This thesis thus advances our understanding of speech and music by providing evidence for the commonality between the two in terms of a common learning mechanism and shared pitch processing, both of which are compatible with accounts of a common origin for language and music.(ACCESS RESTRICTED TO ABSTRACT ONLY UNTIL 20/09/2021)

Currently little is known about the brain dynamics and expressive movements that support musical engagement. We hypothesize that repetitive expressive gestures play an important role in inviting musical engagement in listeners, and that these movements can reveal the feelings experienced by the listener. Furthermore, we hypothesize that brain dynamics supporting these expressive movements play a key role in musical engagement. We trained expert and non-expert participants to communicate the feeling of music they are hearing using simple rhythmic U-shaped hand /arm "conducting" gestures that animate the 2-D movement of a spot of light on a video display while we use body motion capture and EEG to record their movements and brain activity. Periodically we introduced a "not engaged" condition during which a distractor task impedes the participant's engagement in the engaged listening task. We then asked viewers to rate the recorded 2-D spot animations of the recorded gestures on a musical emotion rating scale to test to what extent the musical affective experience of the "conductors" can be conveyed by these animations to viewers who do not hear the music. The ratings from the conductor and viewer groups were well correlated, verifying that the affective intent of the conductors' gestures are experienced by viewers. Statistically significant differences were found in the motion capture and EEG data between the fully engaged condition and the not-engaged condition. A comparison of the EEG data recorded during the two conditions revealed low alpha- and theta-synchronization in the parietal- temporaloccipital (PTO) junction which was specific to the engaged condition, and time-locked to the participants' expressive movements. The results from the viewer experiment suggest that the feeling intention of the expressive gesture task is communicable through a single point-light display, and that viewers can distinguish engaged performances from not-engaged performances. Our EEG results suggest that brain dynamics supporting engaged music listening, located at the PTO junction, are co- modulated with the expressive, rhythmic movements made by the listener. The fact that we can non-invasively monitor musical engagement gives us a useful and general tool for music perception research, with possible wider applications to music classification, technology, and therapy

Complex auditory sequences (e.g. speech and music) unfold in time. With exposure, listeners can extract regularities in these sequences and develop expectations about the identity (ordinal dimension) and the timing (temporal dimension) of upcoming events. When upheld, these expectations permit faster, and more accurate responses to events, compared to when expectations of the ordinal and/or temporal dimensions are violated (Brandon, Terry, Stevens, & Tillmann, 2012; Buchner & Steffens, 2001; Nissen & Bullemer, 1987; Shin & Ivry, 2002). In everyday life, expectations can be acquired without an intention to do so, and without attention being directed to the regularities in the sequence (Perruchet, 2008; Shanks, 2005). That is, sequential regularities can be learned implicitly. Implicit learning (IL) research has primarily investigated learning of ordinal dimensions of visual sequences. The research presented in this thesis investigated IL of complex temporal structures in auditory sequences with rhythmic features typical of music. Musical rhythms of many cultures have a hierarchical beat structure: meter. Meter is the perception of cyclic patterns of strong and weak beats. The more salient the strong beats, and the more regular and frequent their distribution across time, the stronger the meter is perceived to be. In most Western tonal music, strong beats are evenly spaced in time (even meters). However, in music from the Balkan region for example, strong beats can be unevenly spaced in time (uneven meters). Through a lifetime’s exposure, listeners develop expectations of meter characteristic of their musical environment. In the experiments reported in this thesis, rhythms with even and uneven meters are used to examine the flexibility of Western listeners to implicitly learn temporal structures. In the current research project, the development of temporal expectations was examined using an auditory serial reaction time task (SRTT). Participants with no or minimal formal musical training identified as quickly and accurately as possible sequences of auditory events (e.g. pseudo-random ordering of syllables). Unbeknownst to participants, the temporal presentation of events followed a repeating series of inter-onset intervals (IOIs). It was hypothesised that, as temporal expectations were acquired over exposure blocks, reaction time (RT) to identify the syllables would decrease. It was also hypothesised that, as temporal expectations were violated at the introduction of a new rhythm at a test block, RT would increase. The results of six experiments provide evidence of IL of temporal structures, or auditory rhythms, when the ordinal structure was unpredictable. Learning occurred not just of the grouping structure but also of the timing between groups of events. Furthermore, listeners of Western tonal music implicitly learned rhythms with culturally familiar and culturally less familiar meters. Together, these findings demonstrate the capacity of listeners to develop temporal expectations of musical rhythms that either uphold or violate long-term, culturally acquired expectations. The findings also highlight the efficacy of implicit learning as a means of developing temporal expectations in a single exposure session. Demonstrations of IL of temporal structures have important implications for settings where an explicit instruction to learn a rhythm may impeded learning (e.g. music education, motor skills rehabilitation, speech therapy).

This dissertation investigates a way of coming to terms with the heterogeneity of musical phenomena rather than attempting to tame and control it through a reductive process, such as a forced mapping of musical theory to neuropsychological theory of musical perception. I probe the boundaries of current theories, musical and other, ranging from empirical process-oriented theories, to embodied phenomenology and Schenkerian theory--exploring how such theories relate to our perception and what kind of work they are doing. Instead of assuming that acts of perceiving and acts of theorizing are fundamentally different things, I suggest that musical perception should be understood as responsive to acts of theorizing. Critical to this is questioning the plausibility of theoretical singularity with respect to musical phenomena and seeking to come to terms with the manner in which theory opens perceivers to the aspect multiplicity of music. In this sense, I show how theory helps get us closer to the music and how music works in the world as a performance of something--be it culture, gender, sexuality, policy, anxiety, intellect, physicality, or something else. By turning our attention towards the heterogeneity of musical experience, we can begin to understand both culture and the "music itself." Such theorizing is tested in multidimensional analyses of operatic works by Mozart and one of his contemporaries where I explore the essence, or the implicit, of a musical piece from within. The analyses begin from music theorists David Lewin, Eugene Narmour, and Christopher Hasty, and attempt to carry their tools forward by mapping out multiple processes that draw us further into the implicit--a state that we might refer to as an assent to the ineffable

A series of experiments investigate the roles of relational coding and expectancy in memory for melodies. The focus on memory for melodies was motivated by an argument that research on the evolutionary psychology of music cognition would be improved by further research in this area. Melody length and the use of transposition were identified in a literature review as experimental variables with the potential to shed light on the cognitive mechanisms in memory for melodies; similarly, pitch interval magnitude (PIM), melodic contour, metre, and pulse were identified as musical attributes that appear to be processed by memory for melodies. It was concluded that neither previous models of verbal short term memory (vSTM) nor previous models of memory for melodies are unable to satisfactorily explain current findings on memory for melodies. The model of relational memory for melodies that is developed here aims to explain findings from the memory for melodies literature. This model emphasises the relationship between: a) perceptual processes – specifically, a relational coding mechanism which encodes pitch and temporal information in a relational form; b) a short term store; and c) the redintegration of memory traces using schematic and veridical expectancies. The relational coding mechanism, which focuses on pitch and temporal accents (c.f., Jones, 1993), is assumed to be responsible for the salience of contour direction and note length, while the expectancy processes are assumed to be more responsible for the salience of increases in PIM or deviations from the temporal grid. Using a melody discrimination task, with key transposition within-pairs, in which melody length was manipulated, Experiments 1a, 1b, and 2 investigated the assumption that contour would be more reliant on the relational coding mechanism and PIM would be more reliant on expectancy processes. Experiment 1a confirmed this hypothesis using 8 and 16 note folk melodies. Experiment 1b used the same stimuli as Experiment 1a, except that the within-pair order was reversed in order to reduce the influence of expectancy processes. As expected, while contour was still salient under these conditions, PIM was not. Experiment 2 was similar to Experiment 1b, except that it avoided using the original melodies in same trials in order to specifically reduce the influence of veridical expectancy processes. This led to a floor effect. Overall, the results support the explanation of pitch processing in memory for melodies in the model. Experiments 3 and 4 investigated the assumption in the model that temporal processing in memory for melodies was reliant on the relational coding mechanism. Experiment 3 found that, with key transposition within-pairs, there was little difference between pulse alterations (which deviate more from the temporal grid) and metre alterations (which lengthen the note more) in short melodies, but that pulse alterations were more salient than metre alterations in long melodies. Experiment 4 showed that, with tempo transposition within-pairs, metre alterations were more salient than pulse alterations in short melodies, but that there was no difference in salience in long melodies. That metre alterations are more salient than pulse alterations in Experiment 4 strongly suggests that there is relational coding of temporal information, and that this relational coding uses note length to determine the presence of accents, as the model predicts. Experiments 5a and 5b, using a Garner interference task, transposition within-pairs, and manipulations of melody length, investigated the hypothesis derived from the model that pitch and temporal information would be integrated in the relational coding mechanism. Experiment 5b demonstrated an effect of Garner interference from pitch alterations on the discrimination of temporal alterations; Experiment 5a found a weaker effect of Garner interference from pitch alterations on the discrimination of temporal alterations. The presence of Garner interference in these tasks when there was transposition within melody pairs suggests that pitch and temporal information are integrated in the relational coding mechanism, as predicted in the model. Seven experiments therefore provide support for the assumption that a relational coding mechanism and LTM expectancies play a role in the discrimination of melodies. This has implications for other areas of research in music cognition. Firstly, theories of the evolution of music must be able to explain why features of these processing mechanisms could have evolved. Secondly, research into acquired amusia should have a greater focus on differences between perceptual, cognitive, and LTM processing. Thirdly, research into similarities between music processing and language processing would be improved by further research using PIM as a variable.

A combination of experimental and empirical studies investigate the assumption that musical expression is communicated in marimba performance through at least two channels – sound and action. A parallel is drawn between the bodily movements and gestures occurring with expressive musical sound, and gestures produced in concurrence with speech. Experiment 1 investigated the assumption that bodily movements and gestures can enhance or diminish the perception of expression and interest in solo marimba performance when presented audio-visually compared to presentation in audio-only form. Body movement is of particular relevance here as the expressive capabilities of the marimba are relatively restricted, and the movements required to play it are visible. Twenty-four musically-trained and 24 musically-untrained observers rated auditory-only and auditory-visual presentations of 20th Century solo marimba excerpts for perceived expressiveness and interest. Performances were given by a male and a female professional musician in projected (public performance expression) and deadpan (minimised expressive features) performance manners. As hypothesised, significantly higher ratings were recorded in response to projected performances than to deadpan. The hypothesised interaction between modality and performance manner was observed. Higher expressiveness ratings were observed for projected performances, and lower ratings were observed for deadpan performances when the presentation was audio-visual compared to audio-only. Higher interest ratings were observed for projected performances when the presentation was audio-visual. Musically-trained participants recorded higher ratings than musically-untrained observers upholding the final hypothesis. The results suggest that expressive functional bodily movements and bodily gestures play an important role in marimba performer-audience communication. Findings are relevant for both performers and educators. The aim of Experiment 2 was to investigate whether the results of Experiment 1, conducted in laboratory conditions, would generalise to an ecologically valid setting – a real concert. Experiment 2 investigated audience continuous self-report engagement responses from 21 participants collected using the portable Audience Response Facility (pARF). The stimulus material was a solo marimba piece performed in a live concert. A female musician performed two musically similar sections within the piece in two different performance manners (deadpan and projected). The second order standard deviation threshold method was used to analyse signal reliability. As hypothesised, mean engagement responses were greater in the projected sample than the deadpan sample. Reliable signal was only observed in the projected sample. Differences between deadpan and projected sample mean engagement responses may be due to expressive bodily movement from the performance manner manipulation; alternatively, serial order effect, necessitated by the concert setting, may be responsible. Such experimentation in ecologically valid settings enables understanding of audience perception of live music performance as it unfolds in time. Expressive qualities of marimba players’ bodily gestures, witnessed in several projected and deadpan marimba performances in the stimulus material from Experiment 1 were analysed in Study 1. Laban Movement Analysis (LMA) observation techniques, involving embodied thinking and kinaesthetic mirroring, enabled analysis of force. Force is the third element of motion additional to temporal and spatial aspects for which technology measuring only kinematics can not account. Effort-shape analysis and notation described and recorded expressive qualities of marimba players’ bodily gestures at specific locations on the musical score. With basic training, professional percussionist performers were able to understand and apply effort-shape analysis and notation. This inspired confidence that effort-shape analysis and notation has potential as an analytical tool for performers, teachers and students. The results of Experiments 1 and 2 and Study 1 lead to the formation of a theory of bodily gestures in marimba performance. This theory accounted for functional bodily movements and bodily gestures in marimba performance based on an embodied interpretation of the musical score. Combined experimental and empirical results indicate that bodily movements and gestures can enhance perception of expressive marimba performance and therefore warrant focussed attention in pedagogy and practice.

The purpose of this study was to investigate the ability of trained musicians and musically untrained college students to discriminate music instrument timbre as a function of duration. Specific factors investigated were the thresholds for timbre discrimination as a function of duration, musical ensemble participation as training, and the relative discrimination abilities of vocalists and instrumentalists. Under the conditions of this study, it can be concluded that the threshold for timbre discrimination as a function of duration is at or below 20 ms. Even though trained musicians tended to discriminate timbre better than musically untrained college students, musicians cannot discriminate timbre significantly better then those subjects who have not participated in musical ensembles. Additionally, instrumentalists tended to discriminate timbre better than vocalists, but the discrimination is not significantly different. Recommendations for further research include suggestions for a timbre discrimination measurement tool that takes into consideration the multidimensionality of timbre and the relationship of timbre discrimination to timbre source, duration, pitch, and loudness.