Search

Multimodality’s popularity as a semiotic approach has not resulted in a common voice yet. Its conceptual anchoring as well as its empirical applications often remain localized and disparate, and ideas of a theory of multimodality are heterogeneous and uncoordinated. For the field to move ahead, it must achieve a more mature status of reflection, mutual support, and interaction with regard to both past and future directions. The red thread across the disciplines reflected in this book is a common goal of capturing the mechanisms of synergetic knowledge construction and transmission using diverse forms of expressions, i.e., multimodality. The collection of chapters brought together in the book reflects both a diversity of disciplines and common interests and challenges, thereby establishing an excellent roadmap for the future. The contributions revisit and redefine theoretical concepts or empirical analyses, which are crucial to the study of multimodality from various perspectives, with a view towards evolving issues of multimodal analysis. With this, the book aims at repositioning the field as a well-grounded scientific discipline with significant implications for future communication research in many fields of study.

Multimodality, involving the study of meaning arising from the integration of language with images and other resources in multimodal texts, interactions and events, addresses the fundamental need to understand human communication in the current age of digital technology. However, multimodality is not considered to be a discipline per se at present. By drawing parallels between mathematics and linguistics, it is proposed that if multimodality is to become a discipline, then abstract context-based frameworks for modeling multimodal semiotic resources and methodologies for investigating patterns of human communication are required. An example of how this could be achieved is provided. From here, multimodality has the potential to provide the foundations for a range of multimodal sciences, in much the same way that mathematics and linguistics underpin the mathematical and language sciences respectively. In doing so, it may become possible to track the changes in human communication arising from digital technology and the resultant impact on thought and reality.

This study uses three examples of Edwardian (1901-1914) book inscriptions – a prize inscription, gift inscription and bookplate – to demonstrate how the adoption of an ethnohistorical approach, in which choices of image, colour, typography and materiality are grounded in archival research, can strengthen multimodal analysis. Furthermore, it argues that, while book inscriptions may seem insignificant markers of ownership, they, in fact, act as a material microcosm of many of the social tensions that existed between class groups in early twentieth-century Britain. The analysis reveals that inscriptions were primarily used to objectify their owners’ economic means and cultural necessities, and assert themselves in a social space, whether to uphold their rank or keep their distance from other groups. These findings demonstrate the importance of embedding hypotheses concerning the function and form of artefacts in concrete historical documents.

This article discusses one of the core axioms of social semiotic theory, namely stratification, in the light of developments in multimodality in recent years. The discussion takes a point of departure in the approaches to stratification taken by Hjelmslev, Halliday, and Kress and van Leeuwen. The article outlines a theoretical experiment exploring how an alternative way of modeling stratification and instantiation may raise some interesting ideas on the concepts of realization dynamics, system-instance, and the different contexts of the semiotic text. This is elaborated in a discussion of how the relationship between stratification and instantiation in the multimodal instantiation of resources at different strata may provide a new approach to mapping out the dynamics of multimodal meaning making.

This article revisits the concept of vectors, which, in Kress and van Leeuwen’s Reading Images (2006), plays a crucial role in distinguishing between ‘narrative’, action-oriented processes and ‘conceptual’, state-oriented processes. The use of this concept in image analysis has usually focused on the most salient vectors, and this works well, but many images contain a plethora of vectors, which makes their structure quite different from the linguistic transitivity structures with which Kress and van Leeuwen have compared ‘narrative’ images. It can also be asked whether facial expression vectors should be taken into account in discussing ‘reactions’, which Kress and van Leeuwen link only to eyeline vectors. Finally, the question can be raised as to whether actions are always realized by vectors. Drawing on a re-reading of Rudolf Arnheim’s account of vectors, these issues are outlined, their consequences discussed, and ways forward suggested.