Online Magazine of the Visual Narrative - ISSN 1780-678X

Home

Editorial Board

Policy

Issue 6. Medium Theory

The Possibility of Minimal Units in the Filmic Image
Part 1

Author: Sophie De Grauwe
Published: Part 1 February 2003; Part 2 October 2003

Abstract (E): The question of minimal units in the filmic image has been extensively discussed in traditional semiotics (e.g. Eco, Metz). Recently, the subject of minimal units has come up again in the work of Groupe µ, who propose an analytic framework of the still image in which minimal units play an important part. This paper evaluates the arguments and models of Metz, Eco and Groupe µ from the perspective of social semiotics. What will be discussed is the possibility of minimal units in relation to the characteristics of the cinematographic system. To establish these characteristics, the role of (1) abstraction and discreteness, and (2) iconicity, economy and arbitrariness in the cinematographic system are discussed. The paper concludes with a note on the integration of cinematographic grammar.

Abstract (F): La questions des unités minimales de l'image filmique a longtemps fait l'objet de discussions très pointues dans la sémiotique traditionnelle (représentée par Eco et Metz). Récemment, la question a refait surface dans les travaux du Groupe µ, qui a proposé un nouveau cadre analytique de l'image fixe dans lequel la notion d'unités minimales joue un rôle clé. Cet article examine les arguments et les modèles de Metz, Eco et du Groupe µ de la perspective de la sémiotique sociale. Plus concrètement, il s'attachera à discuter la possibilité de décrire des unités minimales en relations avec les caractéristiques du système cinématographique. Ces caractéristiques seront établies à partir d'une réflexion plus large sur deux éléments: d'abord le rôle de l'abstraction et du caractère discret des signes, ensuite les concepts d'iconicité, d'économie et d'arbitraire. L'article se termine par une note sur l'intégration de la grammaire cinématographique.

Part 1 - Part 2

1. The social semiotic approach

1.1 Traditional semiotics and social semiotics

In this article, the question of minimal units will be studied. Until now, it has been studied mostly in traditional semiotics. However, this paper will approach the question from a social semiotic background. It is worth giving an overview of the differences between the two approaches.

Traditional semiotics is based on structural linguistics, which thrived between the 1930s and the 1960s. In the 1960s, the core ideas of structural linguistics were applied to all semiotic systems by traditional semiotics. Therefore, traditional semiotics considers the semiotic system in its independent, stable, synchronous state as a closed system, the "code". The code is strictly separated from the utterance or the text. Rather than studying semiotic processes, traditional semiotics is concerned with the study of signs of all semiotic codes.

Social semiotics is based on systemic-functional linguistics, a form of functional linguistics with Halliday as its most important exponent. Social semiotics considers semiotic systems in their social and historical determination. Instead of signs, it studies signification processes, the production of utterances or texts in context. Only by studying this, the linguistic system can be understood. A system is seen as a meaning resource, employed by individuals in their own socially and individually determined ways in the construction of meaning. The social and historical relativity of the system implies the possibility of change: change is seen as inherent to the system. The system postulated is an open-ended system.

In traditional semiotics, categories are supposed to have clear boundaries and to be defined by necessary and sufficient conditions. In social semiotics, on the other hand, the notion of prototypicality entails a less determined way of categorization. Categories and definitions have vague boundaries, which ensures their flexibility, and they are structured by salient central elements (elements which have a lot of weight in the definition or category) and weak peripheral elements (elements which are marginal in the definition or category). Prototypicality ensures the open-endedness of the system while at the same time maintaining the stability necessary for a system.

In traditional semiotics, meanings, which are separated from forms, are encoded and decoded in user communication. As the focus is on the study of forms, the question "what do these forms mean" (Halliday 1994: xiv) is posed only at a later stage. In social semiotics, the language system is not considered as a code, as "there are no meanings waiting around to be encoded" (Halliday 1994: xii). Meaning is created through the use of language, and meaning and form are studied in their mutually constitutive role rather than as separate entities. The focus shifts to meaning and the relation form-meaning is reversed: the question is now: "How are the meanings expressed" (Halliday 1994: xiv). As a result, the forms are seen "as means to an end, rather than as ends in themselves" (Halliday 1994: xiv).

In traditional semiotics, the arbitrariness of verbal language is emphasized, and there is not much attention for iconic meaning in the linguistic system. In social semiotics, on the other hand, iconicity is given a much more prominent role, as it is considered very important in the establishment and use of grammatical relations. Iconicity is defined, in a Peircean way, as similarity between the structure of a certain meaning and the structure of its corresponding form (see section 4). The acceptance of iconicity is possible because the concepts of arbitrariness and conventionality are separated: relations can be iconic and conventional. This means that they can be part of a semiotic system which requires conventionality, without the latter implying that they should be, or are, arbitrary.

Traditional semiotics is characterized by a desire for scientific exactness and logic. The "code" is a logical system (preferably adhering to binary logic). In social semiotics, the system is not logical. In my view, a system is determined by a general interconnectedness which is not necessarily based on logic. As the system is socially determined, it is constituted by social selection, which can be based on a variety of considerations, like association, iconicity, and economy (see below), and which changes in time. The resulting system can be logical in some aspects or components but it is not founded on logic, as this is not the dominant mode of thinking.

Traditional semiotics often makes use of binary oppositions. This can be explained by the fact that the presence of two elements is the minimum to establish a systemic contrast. It may be that two contrasting elements are more easily found than three or more contrasting elements. However, if binary opposition becomes a rule, it becomes restrictive and oppositions of more than two elements cannot be recognized anymore.

In my opinion, traditional semiotics, although it has its merits (e.g. in the study of the inventory of signs), should not be used in a comparison of the linguistic system and other semiotic systems. In traditional semiotics, the feature of arbitrariness is absolutized, establishing it as an essential feature of semiotic systems and inhibiting a balanced and differentiated comparison between the linguistic system and other semiotic systems.

In this paper, however, a lot of attention will be given to arbitrariness and the inventory of signs, more than usual in a social semiotic approach. This is necessary if we want to counter the claims of traditional semiotics.

1.2 Presentation of the social semiotic model

Social semiotic theory of the filmic image is based on Halliday's systemic-functional theory and on its adaptation to the still image (in Western society) by, among others, Kress and Van Leeuwen (K&VL). This discussion draws on the adaptation of K&VL's theory to the filmic image in Iedema (1998) and De Grauwe (1999).

According to Halliday, semiotic systems have three metafunctions: the ideational, the interpersonal, and the textual. Firstly, semiotic systems have to represent objects of the experiential world (outside of the semiotic system concerned), and relations between them. This is their ideational or representational metafunction. In verbal language, each experience is structured as a semantic configuration of process, participants and (optionally) circumstantial elements. Processes are expressed by verbal groups, participants by nominal groups, and circumstances by adverbial groups or prepositional phrases. There are different types of processes, each with one or two inherent participants. For example, material processes are processes of "doing", in which one entity, the Actor, does something which may be directed at another entity, the Goal (see figure 1).

Figure 1. One- and two-participant material processes (based on Halliday 1994: 103)

In the filmic image, similar semantic roles and configurations can be discerned but they are represented by very different formal means. In filmic narrative representations, which represent dynamic relations between participants, processes are represented by vectors (diagonal lines) or directional movements. The vectors emanate from volumes (visually distinct entities) which are semantically construed as participants (or "represented" participants). In conceptual representations, in which participants are represented in terms of their class, structure, or meaning, processes are realized by juxtaposition of volumes (see figures 2 and 3).

Figure 2. One-participant action process (Kress and VanLeeuwen 1996: 62)

Actor: Ben Hur and his chariot
Process: action realized "by the vector created by the diagonal line of action"
(Kress and VanLeeuwen 1996: 61)

Figure 3. Symbolic attributive process (Kress and VanLeeuwen 1996: 110)

Process: symbolic, realized by juxtaposition (establishes the identity of the Carrier, i.e. what the Carrier means or is)
Carrier: girl
Symbolic Attribute: picture, made salient by special way of lighting

A second metafunction of semiotic systems is their interpersonal metafunction. Semiotic systems have to project social relations between the producer of the (complex) sign and the receiver of it (i.e. the interactive participants), and between interactive and represented participants. In the filmic image, the relation between viewer and both represented participants and (indirectly) the image-producer is established for example through the size of frame, which realizes social distance (the "close-up" realizing close personal distance, the "long shot" realizing remote social distance). Other interpersonal relations are established by horizontal and vertical angles, realizing involvement and power respectively (for a critical discussion of these relations, see De Grauwe 1999). Interpersonal relations can be realized not only statically but also dynamically, for example through a moving camera. In that case, the viewer is dynamically involved in the realization of the interpersonal meanings of social distance. In such a dynamic realization, it is not the stable interpersonal meaning which is at issue, as in a static approach, but the evolution in the interpersonal meaning. In verbal language, social distance is realized by formality of style. Verbal interpersonal relations are also realized by "speech functions" or "speech acts", in which the speaker gives or demands information or goods and services from the listener, i.e., the speaker makes a statement, asks a question, demands something. This is realized by mood (permutations of subject and finite verb). In the filmic image, two similar "image acts" can be realized: the gaze of the represented participant at the viewer realizes a demand, the absence of a gaze realizes the presentation of information. Another aspect of the interpersonal metafunction is modality. This can be defined as the "truth value or credibility of […] statements about the world" (K&VL 1996: 160). When an image has low modality it is presented as untruthful. When an image has high modality it is presented as truthful. This is a matter of degree. The modality of an image is determined on the basis of several modality markers, like colour saturation, contextualization (the degree of presence or absence of background), camera mobility, etc. The "truth" which is presented in images is not absolute, it is the truth of a social group. Different social groups can have different reality principles or coding orientations. Kress and Van Leeuwen have identified four major spatial-visual coding orientations in the present Western society - technological, sensory, abstract, and naturalistic - of which the naturalistic is the most dominant. In naturalistic coding orientation, the standard is photographic realism. In abstract coding orientation, there is high modality if "the individual is reduced to the general, and the concrete to its essential qualities" (K&VL 1996: 170). In sensory coding orientation, the pleasure principle dominates. Colour is "a source of pleasure and affective meanings" (K&VL 1996: 170), and therefore has high modality. Judged by the standards of naturalistic modality, the colours are hyperrealistic. Finally, in technological coding orientation, high modality is accorded to representations in which the qualities regarded as essential (in relation to the technological/scientific purpose of the representation) are represented in a way which is considered useful (again in relation to the technological/scientific purpose of the representation). Iedema has identified four coding orientations of the moving image: the Real, the Really Real, the Real-as-Sensation, and the "Symbolizing the Real". These are mostly connected with camera movement and editing (see figure 4).

Figure 4. The four coding orientations of the moving image (after Iedema 1997: 1)

Finally, semiotic meaning is realized on the textual level. Semiotic systems have the capacity to form texts. Texts are complexes of signs in which the signs "cohere internally and with the context" (K&VL 1996: 41). This is their textual or compositional metafunction. In the filmic image, the integration of represented and interpersonal elements is realized, for example, by salience. Salience is the weight of the depicted element, the "degree to which the element draws attention to itself" (K&VL 1996: 225). This is a matter of degree. It is determined spatially, by the size of the element, the sharpness of focus, tonal contrast, etc., and temporally, for example by deviations in the pace realized by camera movement and editing. Spatially, judging the visual weight of the elements is judging their balance: how all elements taken together create a balancing centre on the basis of their visual weight. This balancing centre contains the central message of the composition, regardless of whether it is the actual centre of the composition or not.

As is obvious from the preceding discussion, the three metafunctions work both spatially and temporally in the filmic image.

Semiotic systems can be studied at different levels of specificity. Paul Thibault distinguishes three different levels of schematicity of the linguistic system (see table 1).

Langue1 is the most abstract level; it is the level of schematic terms. It is also the level of pure values, which are negatively defined through differences. The two orders of difference, i.e. the orders of signified and signifier, are not yet cross-coupled in the signification process.

Langue2 is the level of "sign types", i.e. "typical lexicogrammatical forms and patterns". Forms are relatively positively defined through opposition and contrast with each other in syntagmatic and associative groups. The structuring principles of the syntagmatic and associative relations make possible the cross-coupling of signified and signifier. This is so because through these relations, signifiers and signified are assigned to groups with typical cross-couplings of the two orders of difference. I will explain this further. Meaningful patterns are construed by combining signs, i.e., by forming syntagmatic relations between them. This combination is possible through the association of these signs with other, similar signs and their combinations, i.e. through their paradigmatic relations. For example, the interpretation of an image in which a character points a gun at another character as being an instance of the syntagmatic combination (Actor-process-Goal) is only possible through the association of the characters and the process (and the relation between them) with similar characters and processes and the relations between these. Only then can the categories of Actor and Goal be constructed, and can they be combined into meaningful patterns.

Langue3 is the level of the typical uses of the forms according to the text type. On this level, the typical co-patterning of forms and the cross-coupling with other semiotic domains are studied.

The principle of schematicity is present in the filmic image as well (as explained in De Grauwe 2000). Both in verbal language and the filmic image, metafunctional meaning and realization can be studied on the different levels of schematicity (the different levels of langue in the case of verbal language). As regards the level of langue3 in the filmic image, this is connected with text types, like genre, and with the interaction between the different modes of film (the visual, verbal and auditive modes; the latter encompassing music, sound effects and - possibly - auditive aspects of verbal elements).

Table 1. The three perspectives on langue compared (Thibault 1997: 72)

The social focus of social semiotics calls for semiotic processes to be studied in context, i.e. as communicative processes involving interactive participants (producer/viewer and speaker/hearer). These communicative processes can occur in real time (as in speech) or with a separation between production and reception (as in film, writing etc.).

In the case of the film, social semiotics is concerned with "how I, the viewer, am positioned by the film in question, and how I see certain allegiances and values as being promoted over others" (Iedema in: Van Leeuwen 2001: 186). Although social semiotics is focused on the social aspect of semiotic processes, the emphasis on the viewer does, in my opinion, make viewer perception and cognition relevant. Since social semiotics is concerned with the construction of meaning, it seems also important to study how this construction occurs. In addition, to focus on the perceptual and especially the cognitive aspect does not mean that the social aspect is ruled out. Cognition is based on social, biological and psychological factors. This is where Groupe µ's theory will prove important.

Many people are opposed to the use of the linguistic system as point of reference for comparison with other semiotic systems, which they call linguistic imperialism. However, many objections are related to traditional semiotics' conception of rigid semiotic "codes". Social semiotics' conception of socially determined, evolving, open-ended semiotic systems is much more compatible with the filmic image. Still, this does not mean that the linguistic system should be directly applied to the filmic image but rather, that it is a useful tool in the study of the filmic image. Since the study of the linguistic system has a longer history than the study of the filmic image, valuable insights might be gained from it. However, these insights should be treated with critical caution in relation to the filmic image. As in every comparison, it is necessary to evaluate both similarities and differences. A renunciation of a black-and-white view in regarding the linguistic system and other semiotic systems makes this possible, as will be explained further.

Some of the opponents to the use of linguistics as a comparative tool see the linguistic system as the only "real" semiotic system. One could consider this another form of linguistic imperialism. Paradoxically, it results in any claim that the filmic image functions as a semiotic system having to be justified through comparison with "the only real semiotic system", the linguistic system. In the present paper, I will therefore compare the filmic image with verbal language, starting from the assumption that there can be different kinds of semiotic systems, rather than, because it would not classify as a semiotic system otherwise, the filmic image has to have the same kind of semiotic system as verbal language. This will, I hope, prevent differences between the filmic image and verbal language from being overlooked. The latter would be more likely to happen when someone who argues that the filmic image has a cinematographic semiotic system, also assumes that a significant difference with the linguistic system means that the filmic image would have no real semiotic system. The renunciation of a black-and-white view, in which differentiation too easily leads to separation, makes it possible to use the linguistic system without any danger of linguistic imperialism. As a matter of fact, it seems striking to me that proponents of "the linguistic system as the real semiotic system" can be found both among stalwarts and opponents of traditional semiotics. This shows that, essentially, the latter adhere to the same black-and-white view of traditional semiotics as the former.

I conclude this section with a note on terminology. I will refer to the system of the filmic image as the cinematographic system, and to its "text" as the cinematographic text. Meaning and form are also referred to as content and expression (after Hjelmslev) or as signified and signifier (after De Saussure). The distinction between "instance" (on the level of the text or Saussurean parole) and "type" (on the level of the system or Saussurean langue) will be used in relation to both the linguistic and the cinematographic system. The term "ranking" applies to the hierarchical and progressive constitution of larger semiotic units on the basis of smaller semiotic units. In the case of verbal language, it applies to the progressive construction of morpheme, word, word group, clause, sentence. I will also apply it to the hierarchical structure in the filmic image as established by Groupe µ, as their analysis entails a hierarchical and progressive constitution of larger semiotic units on the basis of smaller semiotic units (see section 2.3). Finally, we need to be clear about what is meant by "grammar" and "system". "System" in social semiotics is interpreted as a "network of interlocking options" (Halliday 1994: xiv), a paradigmatic network of categories. The network is structured in a way that you start from general features and gradually proceed to more specific features, each choice leading to another, more specific option. For example, in linguistics, "'a message is either about doing, or about thinking, or about being; if it is about doing, this is either plain action or action on something', and so on; or 'a syllable is either open or closed; if closed, the closure may be voiced or unvoiced'" (Halliday 1994: xiv). "Grammar", in social semiotics, is concerned with both system and structure. While "system" represents the grammar as networks of choices, "structure" shows how these options are realized. In other words, system highlights the paradigmatic side of the grammar, while structure highlights its syntagmatic side. This also means that grammar is concerned with both system and text, or, as text is studied in its aspect as a process, grammar involves both system and process.

2. Minimal units in traditional semiotics

2.1 Metz

According to Metz, there is no one cinematographic "code", but a multitude of cinematographic codes. Each of these codes has its own minimal unit, which can have widely differing shapes and sizes. He classifies those units in two classes: segmental units and suprasegmental units. Segmental units are units which "occupy a continuous segment of filmic space and time" (Metz 1971: 151; my translation), for example the "photogramme" (i.e. one frame of the 24 frames per second), the shot, the filmed object, the sequence. Suprasegmental units are more abstract, being not present in the "textual surface". For example, the distribution of colours in filmic time and space can constitute a code. Its pertinent unit will be colour, which, in contrast with the coloured object, does not occupy textual surface. Other examples are the co-presence of two objects, camera movements, editing "figures", etc.

According to Metz, the various cinematographic codes are not organized in a strict enough way to be able to speak of one langue, like for verbal language. For the latter, the "system of langue" can be defined as a "system of systems" (Metz 1971: 48, my translation), such as the phonological system, flexion, syntax, the various lexical configurations, etc. This system is, however, ultimately a unitary one, and there are close interconnections between the various systems. This is not so in the cinematographic system because of the need of "souplesse" (Metz 1971: 49). Flexibility seems to be associated with a large number of only losely integrated systems.

In the social semiotic conception, the structuralist notion of system is itself too rigid to be applied to flexible social communication and representation systems, not only in the case of film but also of verbal language. So in a social semiotic conception, a system is not equivalent with a closed entity but with a flexible, open-ended entity, and an integrated cinematographic system would not restrict flexibility. Therefore, it seems to be a matter of determining whether the various systems stipulated by Metz migh not be integrated after all in a way that minimal units can be found which are relevant to all systems. I will elaborate on this at the end of this article.

2.2 Eco

Eco distinguishes three articulations in film, as opposed to the two articulations in verbal language. In codes consisting of two articulations, "figures" of the second articulation combine to form signs of the first articulation. These figures are not part of the meaning of the signs; for example, phonemes have no part in the signified of verbal signs. In a code with three articulations, the signs of the first articulation combine to form syntagms, which constitute figures of the third articulation. These figures combine to form the signs of the third articulation, but have no part in the meaning of those signs.

Figure 5 Schematic representation of Eco's Two and Three Articulations

Key of the symbols used in Figure 5

x1, x1, x3          iconic figures (filmic image), verbal figures (verbal language)

x1, y2, y3                         

X, Y                    iconic signs (filmic image), verbal signs (verbal language)

XY                      iconic text (filmic image), verbal text (verbal language)

XY.Z                               photogramme (filmic image)

a1, b1                               kinesic figures

A, B                    kinesic signs

AB                      kinesic text

Justification for Figure 5

Eco takes the kinesic articulation as the third articulation, while he compares the first and second articulation of verbal language with the articulations of iconic signs and iconic figures respectively. Still, I chose to put the verbal signs and verbal text (verbal first articulation) on the diachronic axe (which takes the kinesic or Eco's third articulation in the filmic image), because the combination of verbal figures or phonemes occurs in the diachronic dimension.

More concretely, in the film image, iconic "figures" like /angle/, /curvature/, /rapport figure-ground/ (x1, x2, x3, etc. in figure 5), combine to form iconic signs, like /eye/, /human nose/ (X, Y in figure 5). Iconic figures constitute analytical components of iconic signs but do not convey parts of the signified of iconic signs. Iconic figures are "the conditions of perception transcribed in graphic signs" (Eco 1972: 215; my translation). In this way, a double articulation is established.

Iconic signs combine to form iconic texts (énoncés, {XY}), like for example << here, a big and blond man is wearing light clothes (est vêtu de clair) >> (Eco 1972: 226; my translation). These iconic texts can be combined to "photogrammes" (frames, {XY…Z}), for example a frame containing the frozen moment of a teacher talking to his students in a classroom. These photogrammes contain "kinetic figures" (a1, b1) which, combined, form kinetic signs (A, B) through the temporal dimension of film. Or, if we look at it the other way round: kinetic signs, which are meaningful gestural units, like for example a pointing finger, can be analysed as discrete kinetic figures (for example each of the different positions of the finger and arm), not by human perception, but by the camera or the projector which divide movement into 24 frames per second. These figures have iconic meaning (as iconic syntagms) but in themselves they do not have kinetic meaning. Only in their combination is kinetic meaning constituted. In this way, the temporal dimension establishes a third articulation.

Eco not only proposes three articulations in film, he also identifies these articulations as codes, each one building upon the other. The cinematographic code has as its significant units the syntagms of the preceding code, i.e. the photographic code, and the latter has as its significant units the syntagms of its preceding code, i.e. the perceptive code.

The problem with this approach is its perceptive justification. In normal viewing circumstances, the kinetic figures cannot be discreted by human perception. Eco claims that the kinetic figures form a system of oppositions with differential values, i.e. that they have a distinctive function and negative, but no positive, value (Eco 1972: 227). However, this system only exists within the camera or the projector but plays no role in the communicative event of the film. For a unit to be able to be pertinent in a communicative event, it must be possible to discrete it as a unit, even if merely with a distinctive function. This does not mean that these figures have to be concrete (although they are in Eco's approach). Pertinent units can be abstract. But to be able to abstract them, the viewers need a concrete given from which to abstract. If these concrete givens exist below the threshold of perception, they cannot function as pertinent units in communication. In this case, the succession of images in film is manifestly designed so as to prevent recognition of these units, instead of enhancing them, and it is successful at that. We can conclude that physical discreteness does not necessarily have semiotic relevance (see also section 3).

Another problem with Eco's classification concerns the first two articulations. He claims that iconic figures do not form part of the signified of iconic signs. However, these figures always have referential meaning: they always refer to a part or feature of a referent. The signified is constituted of what are supposed to be pertinent features of a referent, which can be visual, functional, etc. (see section 2.3). The fact that it is possible for minimal parts of the signifier to refer to features or parts of the referent is due to iconicity. In an iconic sign, a relation of similarity is established between referent and signifier, and thus between signifier and signified. This is precisely its difference from arbitrariness: in an arbitrary system, there is an arbitrary relation between signifier and signified. The reason for this is that there is no relation of similarity between signifier and referent, so that elementary parts of the signifier have no relation of similarity with parts of the referent. As a result, there is no relation of similarity between the signifier and the signified. In iconic systems, the iconic relation between signifier and referent establishes iconic relations between the features of signifier and referent so that the features of the signifier realize features of the referential meaning of the sign. In other words, iconic relations are established between the signifier and the signified. Of course, the signifier only refers to a limited part of the meaning of the sign, viz. its visual referential meaning, and not to its functional or its non-referential meanings (see section 2.3 for an elaboration on the relation between iconic signifier, signified and referent).

2.3 Groupe m

Groupe m bases its discussion of minimal units on visual perception and cognition. They hold that perception is important in the study of semiotic systems because it is the first step in the understanding of texts. Every text has to pass through perception in order to become a text, and the perceptive medium imposes constraints on how the selection of pertinent elements takes place. Thus, it is important in the establishment of the semiotic system. For example, the linearity of the spoken medium imposes an important constraint on the establishment of the linguistic system. In the visual medium, the way we perceive colours and forms is important for the way in which we assign meaning to them. Perception, cognition and the semiotic process are all based on, and connected through, a continually progressive selection and categorization.

Groupe µ identifies three basic terms involved in the constitution of iconic signs: types [1] , signifiers and referents (see figure 6). Types are mental representations of visual percepts. They originate from the classification process in visual perception and cognition, which groups visual percepts in classes by abstracting their pertinent features and stabilizing them. The stabilization of these classes can be based on universal models (i.e. based on biological or universal experiential factors) but is also often culturally determined. Types are conceptual classes which can be described not only by visual characteristics but also by functional and other characteristics. For example, a <hat> [2] is not only determined by its visual features but also by the way it is used.

Figure 6. Model of the iconic sign (Groupe µ 1992: 136)

The function of these types is "to guarantee the equivalence (or transformed identity) of referent and signifier, an equivalence which is never just due to the relation of transformation" (Groupe m 1992: 137; my translation). Referent and signifier thus come into a relation of "cotypie". However, the relationship between referent and signifier is less indirect than in the usual semiotic triangles; it is a transformational relationship. That is why referent and signifier are not connected through a dotted line but through a full line.

Types manifest themselves in signifiers which can be articulated into units of a lower rank in two ways. They can be decomposed into sub-entities, which correspond to subtypes, and they can be decomposed into their formal characteristics, or marques [3] . For example, the signifier /head/ can be decomposed into its sub-entities as a particular configuration of signifiers /eyes/, /mouth/, /nose/, and it can be decomposed into its formal characteristics, like /circularity/, /curvature/, /superposition/. Other possible spatial marques are /elongation/, /relative size/, etc., but marques can also be based on colour and texture (movement is not included in this account of still images). Sub-entities can again be decomposed into sub-entities of a lower rank and into marques (for a definition of "rank" see section 1.2).

The analysis of types into subtypes (and, conversely, the constitution of supra-types) is in principle unlimited but in practice limited by recognition: below or above a certain threshold it is very difficult to recognize a type. Also, there may be widely differing levels of redundancy according to the level of abstraction of the iconic sign. A photograph will have a higher level of redundancy than a pen drawing. Therefore, lists of determinants of what constitutes a particular iconic sign may vary widely.

Groupe µ argue against an identification of their model with the phonological model in which entities would be seen as meaningful units, corresponding to types (first articulation), and marques as merely distinctive units (second articulation). One of the reasons is that linguistic analysis is supposed to occur successively along the two articulations (analysis of meaningful units until the smallest meaningful unit is reached, then analysis into distinctive units), whereas iconic analysis can always occur simultaneously (analysis into marques can occur immediately without waiting for analysis into sub-entities). Another difference is that marques do not have a fixed value: the type <face> can be realized by /straight lines/ or /curves/, by /pink/ or /white/. In other words, the replacement of /straight/ by /curved/ does not change the nature of the type <face>. However, this is not a pure phono-stylistical variation, as the same replacement can produce a change in the nature of other types. Therefore, in iconic signifiers, as opposed to linguistic signifiers, the "continuum of realizations is [not] segmented in a stable manner by theoretical classes of units" (Groupe µ 1993: 436, my translation). [4]

It seems to me that the concept of prototypicality would be very helpful here. It is not just a matter of free variation how types like <face> are realized. A prototypical realistic Caucasian <face> will be realized by /curves/ and /light pink/. Prototypical types are not defined by an exhaustive list of determinants to which all instances of that type must conform. Instead, they display a set of characteristic determinants to which instances can conform to a greater or lesser degree. The non-exhaustiveness of the lists of determinants (see above) is made acceptable by prototypicality as a semiotic mechanism.

There can be different prototypical types of each type according to what social semiotics calls "coding orientation" (see section 1.2) and according to the medium. In different media, there are different conceptions of what constitutes realism, for example, realism is different in a pen drawing or a photograph. These conceptions are also socially and historically determined. And, the naturalistic coding orientation is not the only coding orientation. The modality or truth value of an image can also be judged according to the abstract, the sensory, or the technological coding orientations. According to its coding orientation, the modality of an image depends on generality (abstract coding orientation), pleasure and affective meaning (sensory coding orientation), or effectiveness (technological coding orientation).

To come back to the comparison between marques and phonemes, another important difference is that marques are not just distinctive. As explained above, I think marques, as opposed to phonemes, have referential meaning.

Groupe µ makes a distinction between iconic signs and plastic signs. Plastic signs are related with non-figurative visual meaning whereas iconic signs involve the figurative. Groupe µ identifies semiotic systems for three "families" of plastic signs: colour, texture, and form. These systems are based on an analysis into visual parameters, 'chromemes', 'texturemes', and 'formemes'. The systematizations of the signifier in the three families are linked to systematizations of the signified. For example, form is established by the three parameters or "formemes": /position/, /dimension/, and /orientation/. On the level of the signified, /position/ is linked with the semantic axe of <repulsion>. Or, more specifically, the opposition /central/-/peripheral/ is linked with the opposition <stable> and <strong>-<unstable> and <weak>.

Although the same three families are used in the plastic sign and in the marques of the iconic sign, Groupe µ emphasizes that marques are not plastic units. Even though their physical manifestations can be described in the same way, the repartition of an énoncé, or text, into marques does not always coincide with its repartition into plastic units. For example, the sub-entity /hair/ of the entity /head/ constitutes only one chromatic marque, while it can constitute two or three plastic units.

Groupe µ's division of the semiotic field into iconic and plastic signs is a means to escape the traditional emphasis on referential meaning. However, it seems to be quite a radical solution, minimizing relations between referential and non-referential meaning making. This is typical of the modernist fragmentary point of view of traditional semiotics. In addition, the hegemony of the sign in traditional semiotics is reinforced, at the expense of grammatical relations. This is where social semiotics' metafunctional levels are helpful. As mentioned above, they imply that meaning is realized simultaneously on three different levels: the referential, the interpersonal, and the compositional level. In fact, a large part of Groupe µ's "plastic meaning" corresponds to the interpersonal and compositional metafunctions (e.g., /dimension/ is linked to <dominance> or <salience>, which is treated on the compositional level in social semiotics; see section 1.2). The advantage of the social semiotic approach is that it allows for variety without absolutizing the differences. In traditional semiotics, the postulation of a rigid system and the need to draw distinct boundaries may result in a black-and-white point of view, where differences are either minimized (to keep an element in a clearly defined category) or maximized (when the element differs too much from the central members of the category). In such a point of view, the study of similarities between contrasting elements is, I think, easily inhibited. The radical separation of the plastic sign and the iconic sign may be the result of such a viewpoint. Indeed, in my opinion, the differing repartition of the iconic and the plastic sign is not an argument for their complete separation. As we will see in section 5, differing repartitions of different systems are possible in linguistics too, like in the phonological system and the system of grammar, or on the different metafunctional levels, and this does not mean that the different systems, although differing, are not integrated (see section 5 for a more detailed explanation). As a result, the parallelisms between "iconic" and "plastic" meaning making can be explored. For example, the systematization of the plastic sign is much more worked out by Groupe µ than the systematization of the iconic marques, but it seems possible to apply the same general systematization to the iconic marques. Groupe µ's examples of iconic marques, like /elongation/, /superposition/, etc., show that they are based on the same parameters as plastic signs, like, for example, the formemes /dimension/, /position/ and /orientation/.

Groupe µ emphasizes that the actual repartitions in the signifier and the assignment of their semantic values are realized in the syntagmatic relations of the text instead of being pre-established in the system. We can say that the system functions as a meaning resource, its gradational nature offering a large number of possible "relative meanings" (meanings realized by their relative position on a scale), instead of a more limited number of "absolute meanings" (meanings realized by a pre-established systemic opposition). In section 3 we will elaborate on the difference between gradational systems and "absolutive" systems. We will see, firstly, that verbal language is based on both systems, and secondly, that the two systems do not diverge as much as often assumed. Gradational systems are not as relative as claimed by Groupe µ and others, and the functional approaches in linguistics have shown that "absolutive" systems are not as absolute as claimed by structural linguistics.

I consider Groupe µ's analysis a very useful approach to the analysis of minimal units of the still image. Of course, they work within the realm of traditional semiotics, with their emphasis on the sign instead of semiotic processes and the separation of signified and signifier (rather than just distinguishing between the two levels, as in social semiotics). This does not mean, however, that their insights cannot form a useful addition to the social semiotic approach. In the latter, not much attention has been given so far to minimal units. Also, they discuss some meaning relations that are not well worked out in the social semiotic approach. For example, <balance>, realized by /orientation/, is not discussed by Kress and Van Leeuwen. They only discuss balance resulting from the combination of figures with different degrees of salience (which is not the same as the "gravitational" <balance> discussed by Groupe µ) [5] . The insights of Groupe µ could be adapted to, and incorporated in, a social semiotic approach of the still image.

As regards the filmic image, Groupe µ's approach is helpful for the analysis of the spatial relations in the cinematographic system. However, the cinematographic system is not only characterized by spatial relations but also, and importantly, by temporal relations. These are established by two features: movement and editing (or the succession of images). These features are not just "added onto" the spatial features; they profoundly affect the relations between the other factors of form, colour, and texture. In an application of Groupe µ's approach to the filmic image, these relations would have to be reconsidered. I will not elaborate here on such an application. However, some suggestions can be made regarding a first basic analysis of movement and editing. Movement can be described according to the parameters of /speed/, /acceleration/, /duration/, and /direction/ (although /direction/ may also be treated as a formal parameter corresponding to /orientation/). Editing is a larger-scale phenomenon; however, it can be analysed in terms of rhythm and framing. In terms of rhythm, editing can be analysed according to the parameters of /duration/ and /acceleration/. In terms of framing, editing can be analysed according to the parameter of /continuity/. But this is only a starting point, and I will not pursue it here as it lies outside of the scope of this paper.

In the present discussion of Groupe µ's analysis I have formulated some criticisms, most of which concerned their traditional semiotic approach. I would like to add one more criticism, concerning the level of abstraction of the iconic signifier, and the latter's relation to the iconic type. On the one hand, the signifier is presented as "a modelised set of visual stimuli" (Groupe µ 1992: 137; my translation and italics). Modelization implies abstraction, so this definition could imply an abstract interpretation of the signifier as element of the system. This interpretation is supported by the marques given as example, which are described by abstract terms like /superposition/, /circularity/, etc. However, the signifier is put in a relation of cotypie with the referent. This could be interpreted as if signifier and referent were both instances of the same type. Thus, signifier and referent are put on the same concrete level, as opposed to the abstract level of the type. These conflicting interpretations of Groupe µ's model, i.e. the iconic signifier as an abstract entity vs. the iconic signifier as a concrete entity, are probably due to the difficulty of differentiating the signifier from the type on an abstract level, at least as far as referential meaning is concerned. For example, marques like /superposition/ and /circularity/ are, at first sight, not just part of the signifier /human head/ but also of the type <human head>. I will explain this further. The meaning of signs, both visual and linguistic signs, is partly constituted by visual characteristics. By abstracting visual features from their concrete physical manifestations, as in the abstraction of a signifier from a concrete instance, it seems that abstract concepts are created which cannot be separated from their referential meaning because they constitute their referential meaning. We could say, then, that referential signifiers constitute a sort of subtype of the general referential type, viz. they would group the visual characteristics into a visual type which would refer to, and be incorporated in, the general referential type.

However, the merger of signifier and type is not as total as that: it is still possible, through the concept of transformations, to make a difference between referential signifier and type as such. Transformations are defined by Groupe µ as the relations of commensurability or homologation between iconic referent and signifier. The most important transformation in the filmic image is the transformation of the three-dimensional referent into a two-dimensional image (for other transformational relations, see Groupe µ: 1992). We could assume the same transformational relations between referential signifier and type. The signifier would then group the two-dimensional visual characteristics while the type would have transformed these characteristics into three-dimensional concepts and thus given meaning to them. Would this imply that visual referential signifiers are meaningless as their meaning comes from their relation to a three-dimensional correlate? In one sense, yes, as they cannot be meaningful separately from their correlate. On the other hand, the relations between the two-dimensional signifier and its three-dimensional correlate act like "rules", affecting large groups of relevant elements, thus realizing a regular relationship between all the elements of the signifier and all the elements of the type. As a result, parts of the signifier correspond to parts of the type, so that we can say that these parts of the signifier are meaningful. In this it differs from verbal language, where words are not constituted by regular relationships between signifier and signified (on the level of lexicology). The arbitrary relation means that there is no rule in the correlation between signifier and signified. As a result, there are no meaningful relations between parts of the signifier and parts of the signified.

The postulation of a transformational relation between referential signifier and type is a theoretical and methodological modelization. In a cognitive-psychological framework, these transformations would be necessary to construct meaning from the perception of an image but there would be no constitution of two parallel units like signifier and type. However, in semiotic terms, it is a useful modelization of the elements necessary for the realization of meaning and their relations.

To conclude, Groupe µ's approach, adapted to the social semiotic perspective, constitutes a useful framework for the analysis of the filmic image into minimal units through the postulation of a limited number of visual parameters. It allows us to explore the abstractive and discretive aspect of visual meaning realization. At the same time, it takes into account the gradational nature of visual meaning systems (see further below). Of course, as mentioned before, Groupe µ's approach is focused on still images instead of the filmic image. Still, it gives an indication as to how an analysis of minimal units could be conceived in the filmic image, and it is still relevant for the spatial relations in the filmic image.

Bibliography

De Grauwe, Sophie. 1999. A systemic-functional analysis of the multimodal text of film. David Lynch's Lost Highway. (Unpublished M.A. thesis, University of Leuven).

De Grauwe, Sophie. 2000. "The cognitivist approach to film in the light of systemic-functional theory: a changing of the guards?". In: Image & Narrative, No. 1. http://www.imageandnarrative.be/narratology/sofiedegrauwe.htm

Eco, Umberto. 1972. La structure absente. Introduction à la recherche sémiotique. Paris: Mercure de France.

Eco, Umberto. 1976. A Theory of Semiotics. Bloomington/Indiana: Indiana University Press.

Elkins, James. 1998. On Pictures and the Words That Fail Them. Cambridge: Cambridge University Press.

Genette, Gérard. 1976. Mimologiques. Voyage en Cratylie. Paris: Editions du Seuil.

Groupe µ. 1992. Traité du signe visuel. Paris: Editions du Seuil.

Haiman, John. 1985. Natural Syntax. Cambridge: Cambridge University Press.

Halliday, M.A.K. 1994. An Introduction to Functional Grammar. London: Edward Arnold.

Iedema, R. 1998. "Culture-specific constructions of conflict: Bonnie & Clyde (US) and The Runner (Iran)". (Unpublished paper, University of New South Wales).

Kress, G. and Theo Van Leeuwen. 1996. Reading Images. The Grammar of Visual Design. London: Routledge.

Manovich, Lev. 2001. The Language of New Media. Cambridge/Mass.: The MIT Press.

Metz, Christian. 1971. Langage et cinéma. Paris: Larousse.

Metz, Christian. 1971b. Essais sur la signification au cinéma (Tome I). Paris: Editions Klincksieck.

Metz, Christian. 1974. Film Language. New York: Oxford University Press.

Putseys, Y. 1995. Sound Patterns in English and Dutch. Notes on Present-day English. Leuven: Acco.

Robins, R.H. 1971. General Linguistics. An Introductory Survey. London: Longman.

Salt, Barry. 1983. Film Style and Technology: History and Analysis. London: Starword.

Thibault, Paul J. 1997. Re-reading Saussure. The Dynamics of Signs in Social Life. London: Routledge.

Van Leeuwen, Theo. 1985. "Rhythmic Structure of the Film Text". In: T.A. Van Dijk (ed.). Discourse and Communication. New Approaches to the Analyses of Mass Media Discourse and Communication. Berlin: Walter de Gruyter.

Van Leeuwen, Theo and Carey Jewitt (ed.). 2001. Handbook of Visual Analysis. London: Sage.

Footnotes

[1] Not to be confused with types as defined in section 1.2, as the latter concern signified and signifier while Groupe m's types concern only the level of meaning. In the following sections, "type" will refer to types as defined in section 1.2, unless otherwise indicated.

[2] Groupe m marks types with "<…>", and signifiers with "/…/".

[3] I will continue to use the French form "marque" instead of its English translation "mark" to indicate the difference with the use of "mark" by, among others, James Elkins. Elkins uses it to designate concrete minimal features of paintings and drawings, like lines and brush strokes. Groupe m's marques, on the other hand, are abstract minimal features of images.

[4] One could add that the variation /straight/-/curved/ is also not comparable with non-stylistic allophonic variation, such as complementary distribution. When two allophones of a phoneme are in complementary distribution, they are "mutually exclusive and cannot occur in the same position to distinguish two utterances" (Putseys 1995: 9). For example, Dutch /d/ has [d] and [t] as allophones. The latter occurs in word-final position, the former in the other positions. However, Dutch also has the phoneme /t/. This means that the difference [d]-[t] in one case serves to distinguish two words, while in the other case [t] is a non-meaningful variation of [d]. Still, this is not the same as the relation between /straight/-/curved/. When these are in a "non-meaningful variation" with each other (in the sense that /straight/ can replace /curved/ without this producing a change in referential meaning), they can replace each other in the same position. This is not the case with [d] and [t] in non-meaningful variation: they cannot replace each other in the same place when they are in complementary distribution. Also, complementary distribution is strictly determined by the phonetic context, while the non-meaningful variation of /straight/-/curved/ involves coding orientation and medium, as explained further in the text.

[5] Paul Thibault, whose theoretical angle is related to systemic-functional/social semiotic theory, does discuss "gravitational" balance, but without integrating it in an overall system of the image.

This site is optimized for Netscape 6 and higher

site design: Sara Roegiers @ Maerlantcentrum