Navigation – Plan du site
Articles
Approches du discours scientifique

Image as Text. Aspects of the shared visual language of scientific conference participants

Elizabeth Rowley-Jolivet
p. 133-154

Résumés

La sémiotique visuelle joue un rôle important d’une manière générale en science et particulièrement dans le genre ésotérique de la communication orale de congrès. Compte tenu de la quantité impressionnante de documents visuels (diapositives et transparents) projetés lors des communications, il est impératif pour les membres de l’auditoire d’être des lecteurs experts du langage visuel de leur spécialité. Cet article examine six aspects, ou strates, des connaissances visuelles partagées de l’auditoire dans trois domaines (géologie, médecine, physique). Les données comprennent plus de 2 000 visuels projetés lors de quatre-vingt-dix communications dans des congrès internationaux récents. Il apparaît que, dans le contexte de communication du congrès scientifique international, ce langage visuel partagé permet aux chercheurs présents de lire les “images” comme un “texte”.

Haut de page

Texte intégral

Introduction

1The discourse genres of science are, in Lemke’s terms, ‘multimedia genres’ (Lemke 1998: 87) which call on multiple semiotic resources – linguistic, mathematical, visual-graphical, and actional-operational – in order to make meaning. Among these various semiotic resources, visualisation has received increasing attention in recent years from ethnographers, semioticians, and sociologists of scientific knowledge, and a significant body of work now exists on this topic. So far, the study of scientific visuals has focused on the inscription practices and interpretation of visual material in the laboratory (Lynch & Woolgar 1990; Amann & Knorr Cetina 1990; Latour 1985; Law 1985; Ochs & Jacoby 1997) and in fieldwork (Law & Lynch 1990; Goodwin 1994; Roth & Bowen 1999), or on pictorial representation in written scientific genres: the research article (Bastide 1990; Latour 1987; Lynch 1990; Lemke 1998; Miller 1998), textbooks (Kress & van Leeuwen 1996; Veel 1998) and the popular scientific press (Jacobi 1999; Jacobi & Schiele 1989; Miller 1998). Between the lab or field, and the written genres of science, however, lies the relatively unexplored spoken genre of the conference presentation (Rowley-Jolivet 1999 and in Ventola et al. 2002a). In the scientific conference presentation, whatever the discipline, the visual channel of communication is a major resource for meaning-making: visuals are omnipresent throughout the talks given, with slides or transparencies being continuously projected onto the screen during the speaker’s monologue. Any investigation of how the conference presentation genre makes and communicates meaning must therefore address its visual dimension.

  • 1  See Rowley-Jolivet 1998 and 2002b for details of the classification system used.

2As can be seen from Table 1, a great number of slides or transparencies are shown during scientific conference presentations, and the periodicity of projection is extremely rapid in certain fields: on average, a new slide every 40 seconds in medicine and geology (one every minute in physics), reaching, in certain presentations, a periodicity of one every 13 seconds, i.e. over 80 in a 20-minute paper. Slightly over 20% of the total are Scriptural, using the same linguistic semiotic as the speaker’s verbal commentary; 17.9% are expressed in the Mathematical semiotic, and the overwhelming majority (almost 60%) use the Visual semiotic (Figurative and Graphical images).1

  • 2  The data used here are composed of video recordings made in 1993-94 of five international conferen (...)

Table 1. Periodicity and semiotic types of visuals in conference presentations2

Geology

Medicine

Physics

Average

Periodicity (in seconds)

46.5

39.6

66.5

51

Semiotic type (in %):

Linguistic (Scriptural)

11

26.8

30.4

22.9

Mathematical (Numerical)

5.2

26.5

17.6

17.9

Visual (Figurative-Graphical)

83.8

46.7

52

59.2

Total (n = 2048)

100%

100%

100%

100%

3The obvious advantage of the two non-linguistic semiotics, in the international conference situation, is that they use universal languages of communication, unlike natural languages such as French, Russian, or English. The dilemma that having to use spoken English at international conferences creates for the non English-speaking scientist is well expressed by Guyon (1996: 3-4), when, commenting on the ‘limits’ of his bilingualism, he points out the usefulness of non-linguistic semiotics in this context:

Dans le cas d’un dispositif expérimental à décrire en anglais je suis un peu ‘coincé’ dans la mesure où je ne suis ni anglophone natif, ni technicien de laboratoire. Heureusement, nous sommes aidés par d’autres outils de description très robustes, tels que formules, diagrammes, schémas, photos, etc. L’utilisation de phrases courtes renvoyant, autant que possible, aux éléments iconographiques fait le reste.

4It is clear, however, given the wealth of specialised information communicated via the visual channel, and the rapidity with which it is shown, that conference participants in the audience need to be proficient in ‘reading’ these languages for them to function as an efficient mode of communication. The present paper therefore explores what is involved in proficiency in reading the predominant language of the two, namely the Visual one, or in other words, how conference presentation images can function as a text for the specialists present in the audience. It is of great interest and importance for the discourse analyst to attempt to grasp the extent or dimensions of this non-linguistic mode, as otherwise our understanding of the discourse will remain partial and incomplete.

5In order to address this question, the paper is organised as follows: section 1 examines in greater detail the term ‘visual language’ in order to bring out its full implications; the main body of the paper (section 2) then focuses on six different levels, or strata, of visual knowledge which, it is claimed, conference participants share. It is this shared knowledge which enables them to read and interpret the images rapidly and accurately in order to extract the relevant information. The paper concludes with a brief summary of the main points.

1. Visual Language: Culture - Code - Claims

6By using the term ‘visual language’ we do not wish to imply that visual expression is the same as linguistic expression. This point, and the distinction between the two, is made clearly by Kress and van Leeuwen:

The analogy with language does not imply (...) that visual structures are like linguistic structures. The relation is much more general. Visual structures realize meanings as linguistic structures do also, and thereby point to different interpretations of experience and different forms of social interaction. The meanings which can be realized in language and in visual communication overlap in part, that is, some things can be expressed both visually and verbally; and in part they diverge —some things can be ‘said’ only visually, others only verbally. But even when something can be ‘said’ both visually and verbally the way in which it will be said is different. For instance, what is expressed in language through the choice between different word classes and semantic structures, is, in visual communication, expressed through the choice between, for instance, different uses of colour, or different compositional structures. (1996: 2, their italics)

7Visualisation and language do, however, share the following three characteristics: both are culturally embedded (section 1.1), both constitute a code of communication (section 1.2), and both are used, in disciplinary discourse, to make claims (section 1.3). Each of these three features is examined below.

1.1. Visual language and culture

8Visual communication, like any other form of communication, is a social act: it is a social semiotic, embodying a social dimension.

Like verbal communication, pictures represent an understanding of the world acquired by members of a certain group, and thus the meaning readers construct from a given image may depend largely on knowledge they share with group members.(...) readers filter pictorial information through a social lens. (Kostelnick 1993: 244)

9The social group in question may be very large, comprising all those living within similar cultures at a particular period or time, or all the members of a particular country or society. This general cultural context which shapes how a given culture visualises the world around it often remains invisible to the members themselves: their culture’s forms of visualisation are taken for granted as being the only natural ones conceivable. As Gould points out concerning the standard images (which he calls ‘canonical icons’) attached to key concepts of our social and intellectual lives:

Nothing is more unconscious, and therefore more influential through its subliminal effect, than a standard and widely used picture for a subject that could, in theory, be rendered visually in a hundred different ways, some with strikingly different philosophical implications. (Gould 1997: 41)

10When sufficient distance has been created between the viewer and the image, however —distance in time, place, or cultural norms— then the culture-specificity of the images, their dominant paradigm, becomes clear. Medieval world maps, for instance, when viewed in the light of modern cartography, appear ‘unscientific’ and imbued with religious symbolism, but were taken at the time to be true representations of God’s world order (Mukerji 1985). Pictorial representations from periods even closer to the present time can also be seen to carry a rich cultural load and to be profoundly embedded in a particular historical, artistic and intellectual context: the visual design of the 18th century English landscape garden can only be fully comprehended by the viewer who has acquired knowledge of the art and poetry of the period, its canons of taste, the political, economic and philosophical climate, the technology of the age, with its interest in optics, changing perspectives, and hydraulics... all resonating with Biblical echoes of a lost Eden (Hunt & Willis 1975). While familiarity with our own culture’s norms means that we do not encounter many problems reading the everyday images that surround us (though we may not always be aware of their ideological import), as the above examples from other periods make clear, reading images involves in all cases a process of enculturation. In Kostelnick’s words:

Reading pictures involves not only what we see but what we know, and hence learning the visual language of pictures entails a process of initiation. (1993: 244-245).

11The importance of enculturation becomes even clearer when one considers the pictorial practice not of the wide social group we all belong to, but of the narrower groups formed by esoteric disciplinary cultures such as the different sciences. Each of these has developed its own “professional vision, which consists of socially organized ways of seeing and understanding events that are answerable to the distinctive interests of a particular social group” (Goodwin 1994: 606, his italics ; see also Rudwick 1976). Reading disciplinary visuals is therefore a learned response. This implies not only that for outsiders to the field – such as linguists – the visuals remain alien and largely incomprehensible, but also that for novices in a given discipline – such as students or doctoral researchers – learning to read and to construct the accepted forms of visualisation goes hand in hand with learning the methods, aims, and warrants of the field itself. It is only when the novice has acquired sufficient shared disciplinary knowledge that s/he will be able to communicate visually in a competent and unambiguous manner. As Larkin & Simon (1987: 71) point out when discussing visual recognition:

Because a representation is useful only if one has the productions that can use it, we can readily understand the common complaint of physics professors that students ‘refuse to draw diagrams’ or ‘don’t appreciate their value’. If the students lack productions for making physics inferences from diagrams, they may not only fail to ‘appreciate’ the value of diagrams, but will find them largely useless.

12There is therefore a visual learning curve, discussed in greater detail below (section 2.3).

1.2. Visual language as Code

13The second implication of the term ‘visual language’ is that visuals, like language, though using different tools, constitute a code of communication, with their own lexicogrammar. Analysis of contemporary Western visual communication – in advertising, textbooks, art and science – by systemicists such as Kress, van Leeuwen and O’Toole among others has demonstrated how the compositional structures, patterns and implicit rules for visual design that have become established in our contemporary culture are used by image-makers to produce meaning. They have focused in particular on the lexicogrammar of visuals, in other words how the various elements in images are combined into meaningful wholes or visual statements of varying degrees of complexity, just as the words of a language are combined by the grammar and lexis into sentences and discourse. The semiotic landscape is changing, they claim, and in our increasingly visual culture “Not being ‘visually literate’ will begin to attract social sanctions. ‘Visual literacy’ will begin to be a matter of survival, especially in the workplace.” (Kress & van Leeuwen 1996: 3)

14The notion of ostracism or exclusion adumbrated in the above quotation concerning general visual literacy becomes even more marked when one turns to the visual codes developed by scientific disciplines, for there is a sharp boundary between insiders and outsiders here: knowing the code signifies inclusion in the esoteric circle, while ignorance entails exclusion. As Kostelnick puts it:

Discipline-specific conventions define audiences by performing a gatekeeping function that allows some readers access to information while restricting other readers. (1993: 250)

15The heterogeneity of practice of visual representation in different scientific disciplines highlights the close link between visual conventions and the epistemology of a field. As Rudwick shows in the case of the emergence of geology as an independent scientific discipline:

...an essential part of this complex historical process was the construction of a visual language that was appropriate to the subject-matter of the science, and which could complement verbal descriptions and theories by communicating observations and ideas that could not be expressed in words (...) this increasing formalization of the visual language of geology involved —and reflects— the development of successive sets of cognitive goals. (1976: 177-8)

16Goodwin likewise stresses the tight anchoring of visual representation within disciplinary activity in science when he comments that:

All vision is perspectival and lodged within endogenous communities of practice (...) An event being seen, a relevant object of knowledge, emerges though the interplay between a domain of scrutiny (...) and a set of discursive practices (...) being deployed within a specific activity. (606, his italics)

1.3. Visual language and claim-making

17The third implication of the term ‘visual language’ is that the aim of disciplinary visual languages is, like the use of language in general in science, not just to inform, but to persuade, to argue a point, to make and support claims: “The most important use of the visuals in academic texts is to support the argument. ” (Miller 1998: 32; see also Latour 1987; Bastide 1990; Amann & Knorr Cetina 1990; Myers 1990; Johns 1998). The rhetorical function of visuals has been particularly stressed by the constructivists, who have highlighted the role of visuals in scientific claim-making. This rhetorical power of visuals to make or support arguments resides not only in the way that scientific image-makers filter, select and channel meaning (Lynch 1990; Bastide 1990) by using visual coding schemes which create order and salience out of perceptually complex, ‘noisy’ natural data (Goodwin 1994), but also in the fundamental features of visual perception itself: operations such as “active exploration, selection, grasping of essentials, simplification, abstraction, analysis and synthesis, completion, correction, comparison, problem solving, as well as combining, separating, putting in context” (Arnheim 1969: 13) are not the privilege of language alone but are also constitutive of visual perception, allowing Arnheim to claim that “Visual perception is visual thinking” (1969: 14).

18What is more, features specific to the visual semiotic endow it with an enhanced cognitive strength for certain functions compared to the linguistic semiotic: patterns, for example,  can be more easily perceived when presented in visual form rather than in a table of numerical data or in text (Krohn 1991); language, argues Lemke, while being a highly efficient tool for expressing typological meaning, is much poorer than the visual semiotic in resources for formulating topological relationships (which he defines as “degree, quantity, gradation, continuous change, continuous co-variation, non-integer ratios, varying proportionality, complex topological relations of relative nearness or connectedness, or non-linear relationships and dynamical emergence”, 1998: 87); diagrams are more effective in terms of mental processing time than text because they allow multiple attributes to be searched simultaneously:

The advantages of diagrams [...] are computational. That is diagrams can be better representations not because they contain more information, but because the indexing of this information can support extremely useful and efficient computational processes. But this means that diagrams are useful only to those who know the appropriate computational processes for taking advantage of them. (Larkin & Simon 1987: 99)

19All these features make visual language a powerful - and indeed essential - semiotic in scientific claim-making.

20This section has discussed three implications of the term ‘visual language’: how professional visualisation is deeply anchored in a discipline-specific culture ; how the disciplinary languages are constituted by visual codes which have great advantages in terms of mental processing time – a highly relevant consideration in the conference context where a large number of visuals are shown in rapid succession – provided that the participant in the audience possesses the necessary knowledge to read the code ; and how the argumentative power of the visual semiotic can be used to support and further research claims. This discussion sets the framework for the following section, which examines the various constituents which, it is claimed, make up the visual knowledge of the expert conference participant.

2. The six strata of visual knowledge

21For clarity’s sake, the different components of scientists’ shared visual knowledge will be presented as six distinct categories. It should be noted, however, that these categories do not operate sequentially but that several may be drawn upon simultaneously by the expert ‘reader’ in the process of interpreting visual data.

Table 2. The strata of visual knowledge

Stratum

Type of visual knowledge

Function

1

      Technical

      Purpose

2

      Compositional

      Coding schemes

3

      Interpretational

      Saliency

4

      Spatial

      Logical relations

5

      Sequential

      Structure & genre

6

      Rhetorical

      Claim-making

22This ‘stratigraphy’ can be interpreted, very broadly speaking, with the usual meaning given to stratigraphies in geology, where greater depth signifies greater length of time (the deeper the layer is, the further back in time it was probably deposited), in that here, greater depth corresponds to an ever greater degree of disciplinary visual knowledge. The top layers need to be mastered first in order to access the deeper layers. For reasons of lack of space, only a limited number of examples of each stratum will be given.

2.1. Stratum 1: Technical knowledge

23Scientists in a given field or speciality learn how to use certain field-specific techniques of visualisation. These are, naturally, linked to the objects of study, the methods of investigation, and the epistemology of the field: geology uses, to mention but a few, reflectance images, back-scattered light images or transmission electron microscopy (TEM) ; physics produces spectra, graphs, block diagrams, CCD camera images ; in chemistry, chromatograms, and 2D or 3D models of molecules are common ; medicine uses ultrasound and X-ray imaging, CAT scans, scintigraphy and MRI images. Technical visual knowledge means not only that scientists viewing an image can recognise the technique that has been used, but also that they know the type of information that each technique can —or cannot— provide ; in other words, technical knowledge implies knowing the purpose of each technique, what aspect is being investigated when such or such a technique is used. For specialists, this information is implicit in the image itself by nature of the technique used: it does not, therefore, need to be verbalised for the esoteric audience and becomes part of the ‘silential relations’ at work in the discourse community (Becker 1995), albeit with one proviso: according to the specialist informants consulted, technical visual knowledge is to some extent more speciality-specific than field-specific —within the broad realm of geology, for instance, a specialist in one sub-field attending a presentation in another geological speciality may encounter some difficulty recognising the visualisation techniques used.

24To illustrate the type of knowledge covered by this stratum, an example has been taken from petrology (Figs. 1 to 4).

Figs. 1 - 4

Figs. 1 - 4

In Bourdon 1999, reproduced with the kind permission of F. Laggoun-Défarge

25The visual knowledge shared by specialists in the field about these types of images is of three kinds:

  • The scale or resolution which each technique permits, and therefore the extent of information that each can, or cannot, provide. Figures 1 and 2 are images produced by photonic microscopy in transmitted light. Fig. 1 shows material from the superficial layers of the peat, while Fig. 2 shows material from lower down in the peat layer. As can be seen, the material in Fig. 2 has undergone various processes of biological transformation (the clear striations in Fig. 1 are no longer apparent). Photonic microscopy, however, only enables a magnification of up to 1000 times, which is insufficient to detect the transformational processes at work, since the material in Fig. 2 appears, under the microscope, as an amorphous mass or blob. Transmission electron microscopy (Figs. 3 and 4), which enables magnification of up to 100,000 times, has therefore to be used. With TEM, the degradation of the tissue by fungi (Fig. 3) and by bacteria (Fig. 4, in which the bacterial cell wall can be clearly seen) can be observed, and ultrastructure at the nanometric scale can be examined. TEM enables mineral inclusions to be seen (how they are placed in relation to the organic matter, if any organisation can be discerned, whether traces of the sources of the matter subsist, etc.) and structural analysis to be carried out, unlike photonic microscopy.

  • The type of investigation that each technique makes possible. Certain techniques cannot be used on very fresh or living material (i.e. containing a lot of water) because the method of preparing the sample involves eliminating the water, and as a result the water-bearing structures are then collapsed and cannot be visualised. In this case, the cryogenic technique will be used: samples are preserved in liquid nitrogen to transform the water into amorphous ice so that the sample is fixed both mechanically and chemically. A thin section is then cut and heated from -240° up to -70° in order to sublimate a few nanometres of tissue; the water-bearing tissues can then be observed with a special microscope.

  • The outward aspect of the final image. Reflectance images (not shown here) are in colour, while  photonic microscopy in transmitted light images, and TEM + cryogeny, for instance, will produce grey-level images only.

26This implicit technical knowledge forms an unvoiced sub-text to the conference presentation, as significant items of information (about the methodology, purpose, ...) in the discourse are not expressed verbally - but are nevertheless there for all the specialists in the audience to ‘see’.

2.2. Stratum 2: Compositional knowledge

27Knowledge of visual composition entails understanding how the diagram or graph has been constructed. This type of knowledge refers not to photographic techniques (the main focus of Stratum 1), but to what Bertin (1973) calls ‘graphemes’, i.e., monosemic constructed visuals whose conventions have been developed within the field or speciality and in which each element has a predefined, unambiguous meaning. Each field possesses its own conventions or coding schemes for constructing graphemes to ensure that they can be read off with no risk of misinterpretation. In each field one also finds a range of well-known graphemes, familiar to all researchers, which serve as references or yardsticks to position or to compare fresh data with established reference data. These yardsticks can be taken for granted as part of the necessary background knowledge of any researcher working in the area in question, and do not therefore need to be explicitly presented or described. In much the same way as a linguistic phrase such as “Chilly weather for the season, isn’t it?” presupposes that both speaker and listener know: a) which season it is, and b) what the average temperature for that season usually is, these visuals form part of specialists’ visual presuppositions.

Fig. 5. van Krevelen diagram

Fig. 5. van Krevelen diagram

28Figure 5 presents an example of such a well-known graph from geology, the van Krevelen diagram. The x-axis represents the ratio of oxygen to carbon, and the y-axis the ratio of hydrogen to carbon. The three lines labelled I, II, and III represent 3 different milieus (respectively, lagoon, marine, and terrestrial), each possessing different types of organic matter and different stages of maturation. The samples collected and analysed by the researcher presenting the paper are indicated by the labels ‘Tritrivakely peat’, ‘ Surface of the peat’, and ‘Cyperacae’; their position in relation to lines I, II, and III clearly shows with which type of milieu they have affinities. The participants’ shared knowledge, when viewing such a graph, includes the following elements:

  • The type of diagram. The name ‘van Krevelen diagram’ will not need to be mentioned, either in writing or verbally.

  • What the x- and y-axes refer to. This is also taken for granted and not commented on.

  • What I, II, and III refer to. The terms ‘lagoon’, ‘marine’, etc. will not be written on the graph ; in addition, each milieu defines a cluster of features which it is superfluous to mention. For a more specialised audience, the graph will often be truncated and show only one of the 3 lines.

29In addition to field-specific techniques, many different scientific fields share common methods of visualisation that are unfamiliar to the layman but which form part of all scientists’ background visual knowledge: objects may be displayed at certain angles to reveal information (top-down views, for instance), are conventionally shown in 2D (a flat plane) with no perspective (architectural plans, electric circuit diagrams, PCB layouts...), or use various types of cross-sections and hatching. The mental/visual agility required to ‘translate’ 3D objects into 2D and vice versa can only be acquired through practice and an often lengthy process of visual socialisation. The vast store of compositional knowledge shared by specialists appears to be a major factor in explaining how many diagrams and graphs containing a high density of information can, in the reality of the conference situation, be understood and assimilated in a matter of seconds by members of the audience, as they focus solely on the New – in the case of the van Krevelen diagram shown in Fig. 5, the rectangle (Tritrivakely peat) and the two small dots – and relate this new information instantaneously to their previous knowledge.

2.3. Stratum 3: Interpretational knowledge

30This stratum is to a large extent the direct consequence of the previous two strata, in that familiarity with the techniques of visualisation and with the conventions of composition used in a given field clearly sharpens one’s interpretative skills in reading visuals. Just as the art critic accustomed to reading paintings and having acquired considerable knowledge about different schools of painting, styles, techniques, etc., will immediately pick out the originality —or on the contrary the derivative nature— of a particular painter’s style, so the experienced researcher will be able to locate quickly and unerringly the interesting or new item of information in a visual, or on the contrary its anomalous or weak spots. Thanks to interpretational knowledge, not only can the salient information be detected rapidly by the specialist, but in many cases s/he will be able to predict with a fair degree of accuracy what a particular visual should look like. If the measurements have been done correctly, for instance, then specialists expect to see a certain shape to the curve; if these expectations are not fulfilled – the curve has a different shape – the anomaly will be immediately spotted. It would appear that many of the mental operations of an efficient reader of text, and those of an efficient reader of visuals during conference presentations, are similar: the ability to formulate hypotheses about what to expect – confirmed or not by the subsequent discourse – is in both cases an important skill in increasing the reader’s speed and competence.

31There is, however, a learning curve. Novice researchers’ visual interpretational skills are much less highly developed than those of seasoned researchers. Indeed, one of the main roles of thesis supervisors in many areas of science is to help their doctoral students to hone their visual literacy in order to see what the discipline considers to be salient information, using its visual categories to extract the relevant evidence from data. This visual apprenticeship is nicely illustrated by Goodwin when he describes how a researcher in archaeology teaches her student during fieldwork to ‘see’ and encode distinctions that have relevance for the discipline (subtle shades of colour in dirt) –distinctions that the student knows in theory but has not sufficient visual experience to perceive (Goodwin 1994: 611-615). Two examples from the data will illustrate the reality, and the difficulty, of interpretational knowledge.

Fig. 6

Fig. 6

Reproduced with the kind permission of F. Laggoun-Défarge

32Figure 6 presents a second TEM image from geology ; void spaces appear as white. Interpreting an image such as this involves distinguishing subtle shades of grey, but the ultrathin (30 nm) sections are difficult to read. Because it is so thin, the section has to be placed on a support or grid (usually carbon) to be examined; this carbon layer, however, has its own grey-level signature in the image. The sample is preserved in resin, which also has its own grey-level signature. It is only when all these ‘interferences’ due to the sample preparation have been abstracted that one can be sure of seeing the structure of the organic matter itself and not of the support or medium used to preserve it. The danger of being misled by instrumental artefacts is therefore very real, and it is only with practice that the scientist achieves reliable interpretational skills. Doctoral students need help and initiation in order to read complex images correctly, particularly with the strong trend towards the pixellisation of scientific imagery: visual resolution is becoming finer and finer, and hence human observational skills are being tried to their limit as more and more sophisticated visual techniques, well beyond the microscale, are developed to observe ultrastructures very closely.

33The difficulty of interpreting images is not confined to novices in the field, however. Any new visualisation technique will, when first introduced into the field, cause problems of interpretation even for experienced researchers, as the quotation below from a medical conference presentation shows. The specialist admits that his interpretation of ultrasound images, was, at the outset, less than optimal:

... in eighty-seven it was the beginning of the ultrasound in Lyon —next slide— and the specialist say ‘I see a small nodes’. I say ‘Okay’ and I tried to find it with my finger, I felt nothing and I say ‘Well, it’s a new technique, they don’t know exactly what they are saying’ and I didn’t believe that this thing [pointing to a barely perceptible circular structure on the slide] was a node, and during three years the patient was in good condition, local control, and three years later I could feel a node, and the ultrasound showed that node.

34These first three strata of visual knowledge – technical, compositional, and interpretational – apply equally well to reading visuals in laboratory work, in research articles and in conference presentations, though the knowledge needs to be activated instantaneously by members of the audience at a conference because of the real-time delivery by the speaker. The next two strata to be examined (sections 2.5 to 2.6) entail layers of visual knowledge that are specific to the conference presentation. Before addressing these strata, however, it is necessary to define rapidly some parameters of the conference presentation context.

2.4. Visual-verbal relationships  in the conference presentation

35Figure 7 is a schematic representation of the time and space relations operating during scientific conference presentations.

Fig. 7. Time & space relations in scientific conference presentations

Fig. 7. Time & space relations in scientific conference presentations

36In the time dimension, the dominant relationship between the visual and verbal channels is one of synchronicity: the information is presented in successive units or packets, each of which includes a verbal and a visual component. The co-existence of the two channels of communication creates a single textual space in which all references to visuals should be considered as endophoric (Young 1990: 91), and which has to be processed as an integrated whole by the audience - in other words, unlike the reader of a scientific article who can process the information selectively, in a non-linear fashion, dissociating if he so wishes the visuals from the text (Bazerman 1988), the researcher attending a conference paper is obliged to follow the linear progression and semiotic mix imposed by the speaker, who is likewise constrained to ensure that his verbal commentary is synchronised with the visual channel. This visual-cum-verbal linear progression is, however, combined with non-linear or spatial relationships specific to the visual channel.

37In the space dimension, in papers which make use of double-screen projection, one visual is projected onto the left screen, and the other on the right. We will call this phenomenon chirality, a term borrowed from stereochemistry, where it refers to the property of molecules to exist in a left-handed and a right-handed form (or levogyrous and dextrogyrous forms). Chirality creates interaction, represented by the horizontal double-headed arrow, between the two visuals themselves, which bypasses the oral channel altogether. In other words, the relationship set up visually between left and right can have an autonomous existence and convey relational information independently of the oral channel

38These distinctive features open up a wide range of meaning-making strategies for the visual channel, unavailable in the research article, which are exploited to structure the discourse and to express a multitude of cognitive or logical relations. As a result, the visual channel carries a heavy organisational, interactional and ideational burden (Rowley-Jolivet 1998, 2002b). These time and space relationships are also dimensions which form part of the shared visual knowledge of the specialist audience, as detailed in the following two sections (2.5 and 2.6).

2.5. Stratum 4: Spatial knowledge (Chirality)

39Chirality only concerns presentations which use double-screen projection (about one-quarter of the presentations at the conferences filmed). Chirality implies a left-right hierarchy: our reading habits (from left to right), the conventional representation of temporal sequence (from left to right along the horizontal axis), and the preferred forms of scientific communication (from the known to the unknown), combine to give the left visual an antecedent status. The viewer, who has acquired, through experience and cultural habit, a certain visual literacy, therefore comes to a given layout with visual presuppositions as to the implicit meaning of chirality. Kress and van Leeuwen (1996), analysing the information value of left and right in systemic terms, see the governing relationship as being that of Given and New and make out a convincing case for this in magazine advertisements, school textbooks, etc. In the case of scientific research presentations, however, it seems more relevant to interpret left-right relations as primarily logical or argumentative relations. Chirality is a local means of expressing such relations: the juxtaposition of the two visuals in space expresses in fact a logical relation, but without recourse to linguistic means. By analogy with the findings of clause-relational analysis in scientific discourse (Hoey 1983, 1994, Winter 1994), Left-Right can be understood as expressing visual clause relations.

40Knowing the common, expected or logical chiral relations that are used in their field gives conference participants an efficient reading strategy for talks using double-screen projection, as it enables them to comprehend twice the amount of visual material at a negligible cost in terms of cognitive processing time: their ability to perceive the logical link, or visual clause relation, between Left and Right visuals means that the two visuals are not processed separately (which would result in doubling the processing time required), but together, in a mutually-supportive or dialogic relationship. Such ‘pairing’ of visuals has been well analysed by Lynch (1991) in the specific case of photo-diagram pairs in research articles, but this phenomenon extends, we claim, to many other kinds of logical visual pairs in the case of the conference presentation. This point has been developed in greater detail elsewhere (Rowley-Jolivet 1998, 2002b), so will just be briefly summarised here. The main types of chiral relations observed in the data are as follows (Tables 3 to 7).

LEFT

RIGHT

Map of Brazil

Map of one province in Brazil

10 µm scanning electron microscope photo of laser-drilled mouse zygote

1 µm close-up of the hole drilled in the photo shown left

List of randomised trials in oncology

Details of one of the trials listed left

LEFT

RIGHT

Tumour before treatment

Tumour regression after treatment

Patch of skin with tattoo

Same patch of skin after laser ablation (no tattoo)

LEFT

RIGHT

Laser flash photograph (50 nanoseconds) showing tissue ablation above the surface of a real sample of tissue

Schematic diagram of the process photographed on the left, entitled ‘Cavitation-induced ablation driven by expansion of subsurface bubbles’, showing how the process operates

Photo of grain of mineral

Schematic computer-produced outline of the same grain, with labels and arrows

LEFT

RIGHT

Experimental trace (spectrum) of electrical current using a MILO

Computer simulation image of the same process

Ternary plot of a sulphur complex

Ternary plot of an arsenite complex

LEFT

RIGHT

Stratigraphy of 3 boreholes

(this image is left on the left-hand screen throughout the projection of the data on the right)

1. schematic map

2. mineral deposits low down in borehole 1

3. mineral deposits higher up in borehole 1

4. mineral deposits in borehole 2

5. mineral deposits in borehole 3

6. photo (thin section) of grains of mineral

7. photo (thin section) of grains

8. photo (thin section) of grains

41The logical links to be read between the two visuals appear to be self-evident for an esoteric audience in a given field or speciality. Chirality enables the visual channel to communicate logical and structural elements of the discourse autonomously, if need be without recourse to the verbal channel, by drawing on the shared visual knowledge of the audience.

2.6. Stratum 5: Sequential knowledge

42Chiral relations exist only in presentations that use double-screen projection, and are far more common in some fields than in others (half of geological presentations use two screens, but one finds a much smaller proportion in medicine, and practically none in physics). Sequences, however, can occur in all types of presentations and in all fields. A sequence is a stereotyped series of visuals which forms a kind of ‘visual routine’. It comprises a succession of visuals in a fixed order, is clearly bounded on the left, but can be of variable length. Some sequences are recursive, occurring in cycles The projection of the first member of a sequence generates expectations in the audience as to what is to follow, and is therefore cataphoric (Fig. 8).

Fig. 8. Sequential knowledge

Fig. 8. Sequential knowledge

43Acquired knowledge about the visual sequences regularly encountered in their domain enables participants to ‘chunk’ the flow of discourse during a presentation, as a given sequence of visuals will be perceived as forming a clearly bounded whole whose internal organisation is often highly predictable, which again reduces the mental processing cost for the audience.

44Observation of the data in geology, medicine and physics shows that each field possesses a certain number of visual sequences, which can be recognised thanks to their specific visual formats: in medicine, common sequences are Patient and tumour characteristics, Tumour Staging, Case-histories, Operating procedure, and Statistical series; in physics, Mathematical demonstrations, and Presentation of R&D products; and in geology, Zooms and description of Samples (see Rowley-Jolivet 2002b). One of these sequences, the Zoom in geology, is presented  below (Table 8).

Table 8. Spatial Zoom in geology

Visual #

Content

Zoom

1

map of country

establishing shot

2

map of region

full shot

3

photo of landscape

medium shot

4

photo of outcrop

medium close shot

5

photo of rock

close-up

6

photo of grain within the rock

big close-up

45In this case, the sequence is co-extensive with a clearly defined section of the paper, or sub-genre, dubbed ‘the geological setting’ by Dressen (1998). In other words, this sequence of images is the visual correlate of a section of the paper. If certain pre-formatted sequences of visuals are regularly or preferentially associated with a particular section of the talk (Spatial Zooms always occur in the Introduction to geology talks, Patient and Tumour Characteristics are always found in the Material and Methods section of medical presentations, for instance), then it becomes much easier for the audience to follow the argument, as there is a strong element of visual expectations. Even in fields such as physics where the highly conventionalised visual sequences described above are much less frequent than in medicine or geology, shared knowledge of the expected visual sequence plays an important role and enables members of the audience to anticipate the upcoming discourse, or, if they have missed information conveyed by the verbal channel, to follow the development of the argument nevertheless. Thus, for example, if the speaker in plasma physics has specified that the dust has been collected during an experiment, the audience will then expect a TEM picture showing the dust; if the speaker has also mentioned that the phenomenon evolves in time, the audience will then expect a graph showing the kinetics of the phenomenon ( a series of images showing measurement in time). The absence of these expected visuals will weaken the verbal claim, as it remains unsupported by data.

46On a higher level of generality than these clearly bounded sequences, specialists in the field also share genre knowledge about the overall structure of different types of presentations and about the types of visuals generally associated with each section. The topic is too vast to be fully developed here, as there are many different types of presentation within each speciality and therefore several different types of organisation. To give just one example, Table 9 shows a schematic of a ‘classical’ progression, and the associated visuals, in two types of geology presentations.

Table 9. Typical association between section and visual type (geology)

Section

Associated Visuals

1

Introduction

Literature data to establish the context and the gap

2

Material & Method

Type 1 presentation: Sample Analysis

Type 2 presentation: Experimental

Identity card of the sample + Figurative visuals (photographic images of samples collected)

Diagram of experimental set-up + flowchart of experimental technique

3

Results & Discussion

Graphemes (graphs)

4

Conclusion

One synthetic visual, or summary in verbal form (Scriptural)

2.7. Stratum 6: Rhetorical knowledge

47The final stratum of visual knowledge, briefly presented here, concerns the rhetoric of the images, or in other words,  how the way in which data or concepts are visualised makes a claim and militates in favour of a particular interpretation or hypothesis. As Claverie (1993) argues in the case of images in molecular biology:

Les schémas, loin d’être neutres dans la façon dont ils résument et synthétisent les résultats scientifiques, assument également un rôle militant dans la lutte pour les idées, la confrontation des modèles, l’établissement des paradigmes : chaque schéma concurrent aspire à devenir le schéma standard. [...] [L]eurs différences sont anodines pour le profane, alors qu’elles évoqueront subtilement tous les points de contentions des modèles concurrents pour le spécialiste. [...] [U]ne barre plus épaisse ici ou là, une surface de contact plus ou moins grande, seront comprises des initiés comme une prise de position plus ou moins claire dans les querelles du jour, l’appartenance à une chapelle, l’adhésion à un courant d’opinion.

48A similar phenomenon observed in our data is shown in Fig. 9, taken from a presentation on plasma processes.

Fig. 9.  Schematic illustration of particle growth in dusty plasma

Fig. 9.  Schematic illustration of particle growth in dusty plasma

Reproduced with the kind permission of L. Boufendi & A. Bouchoule

49The schematic shows how clusters of particles nucleate, grow and coagulate in the plasma. The initial stage in this 4-step scenario (‘Formation of primary clusters’) represents the process as being triggered by the negative ions which are trapped. This is in fact a controversial claim, not shared by another team of researchers working in the same field, but perceiving its value as a rhetorical claim requires considerable visual knowledge about other possible, or competing, representations of the same process. Another frequently encountered form of visual rhetoric concerns simplification or filtering of the data. Fig. 10 (scales and axes not represented here) shows the original data points corresponding to measurements of the ratio of hydrocarbon to organic carbon in a drillcore. The data are ‘noisy’, with a zig-zag shape to the graph and several outlying points, making it difficult to discern a clear trend. When presenting data such as this, the researcher will prefer to draw a single line which gives the best fit to the data points, thereby choosing to ignore the ‘anomalies’ or outlying points, but this is clearly a rhetorical decision, and the precise shape of the line chosen will be that which best supports the claim that the researcher is making.

Fig. 10 . ‘Noisy’ graph

Fig. 10 . ‘Noisy’ graph

In Conclusion

50This paper has investigated various strata or levels of visual knowledge which scientific conference participants draw on when following conference presentations. Technical visual knowledge enables them to perceive the purpose and limits of the methodology; compositional knowledge forms a rich store of coding strategies which speed up comprehension when faced with the high density of visual data shown ; interpretational knowledge enables expert readers to locate rapidly the salient, new or anomalous information; chiral, or spatial, knowledge involves visual perception of logical links and relations, at little extra cost in processing time ; sequential knowledge, a form of genre knowledge, means that the discourse can be ‘chunked’ efficiently by the audience and its progression anticipated; finally, rhetorical knowledge – an advanced manifestation of interpretational knowledge – involves perception of the claim that is made visually. The extent of this knowledge and the variety of aspects which it comprises not only appear to explain how scientific conference participants ‘manage’, conceptually speaking, the vast amount of visual material shown during presentations, but also, it is argued here, signifies that the visual semiotic operates as a true language of communication in the conference context. This ‘language’ is unvoiced, and largely invisible to the non-specialist, but for the esoteric audience, the images constitute a text.

I would like to express my thanks to F. Laggoun-Défarge, A. Bouchoule and L. Boufendi for providing the scientific visual material reproduced in this paper, and for their time and patient explanations.

Haut de page

Bibliographie

Amann, K. & K. Knorr Cetina. 1990. “The Fixation of (visual) evidence”. In Lynch, M. & S. Woolgar (eds.), 85-121.

Arnheim, R. 1969. Visual Thinking. Berkeley, CA: University of California Press.

Bastide, Françoise. 1990. “The iconography of scientific texts: Principles of analysis”. In Lynch, M. & S. Woolgar (eds.), 187-229.

Bazerman, Charles. 1988. Shaping Written Knowledge. Madison, WI: University of Wisconsin Press.

Becker, A.L. 1995. Beyond Translation. Ann Arbor, MI: University of Michigan Press.

Bertin, Jacques. 1973. Sémiologie graphique. Paris: Gauthier-Villars.

Bourdon, Sonia. 1999. “Approches micromorphologiques et moléculaires de la diagenèse précoce de la matière organique dans une tourbe à Cypéracées en milieu tropical (Tritrivakely, Madagascar)”. Thèse de l’Université d’Orléans.

Claverie, Jean-Michel. 1993. “Images et concepts. Point de vue d’un biologiste.” Alliage 15, 35-49.

Dressen, Dacia. 1998. “Le ‘cadre géologique’ dans l’article de géologie en français et en anglais : crédibilité et retenue sur le terrain”. ASp 19-22, 51-66.

Goodwin, Charles. 1994. “Professional vision”. American Anthropologist96/3, 606-633.

Gould, Stephen Jay. 1997. “Ladders and cones: Constraining evolution by canonical icons”. In Silvers R. B. (ed.), Hidden histories of science. London: Granta, 37-67.

Guyon, E. and A. Guyon. 1996. “Anglais de specialité et plurilinguisme”. ASp 11-14, 1-12.

Hoey, Michael. 1983. “The place of clause-relational analysis in linguistic description”. ELR Journal 4, 1-32.

Hoey, Michael. 1994. “Signalling in discourse: A functional analysis of a common discourse pattern in written and spoken English”. In Coulthard, M. (ed.), Advances in Written Text Analysis, London: Routledge, 26-45.

Hunt, J. Dixon & Peter Willis (eds.). 1975. The Genius of the Place: The English landscape garden 1620-1820. London: Paul Elek.

Jacobi, Daniel. 1999. La Communication scientifique : discours, figures, modèles, Grenoble: Presses Universitaires de Grenoble.

Jacobi, Daniel & B. Schiele. 1989. “Scientific imagery and popularized imagery: Differences and similarities in the photographic portraits of scientists”. Social Studies of Science 19, 731-753.

Johns, A. M. 1998. “The Visual and the verbal: A case study in macro economics”. English for Specific Purposes 17/2, 183-197.

Kostelnick, Charles. 1993. “Viewing functional pictures in context”. In Roundy N., Blyler & C. Thralls (eds), Professional Communication, The Social Perspective. London: Sage, 243-256.

Kress, Gunther & Theo van Leeuwen. 1996. Reading Images: A grammar of visual design. London: Routledge.

Krohn, Roger. 1991. “Why are graphs so central in science?”. Biology and Philosophy 6, 181-203.

Larkin, Jill H. & Herbert A. Simon. 1987. “Why a diagram is (sometimes) worth ten thousand words”. Cognitive Science 11, 65-99.

Latour, Bruno. 1985. “Les ‘vues’ de l’esprit : une introduction à l’anthropologie des sciences et des techniques”. Culture technique 14, 5-29.

Latour, Bruno. 1987. Science in Action, Cambridge, MA: Harvard University Press.

Law, John. 1985. “Les textes et leurs alliés”. Culture technique 14, 59-69.

Law, John, & Michael Lynch. 1990. “Lists, field guides and the descriptive organization of seeing”. In Lynch, M. & S. Woolgar (eds.), 267-299.

Lemke, Jay. 1998. “Multiplying meaning: Visual and verbal semiotics in scientific text”. In Martin, J. R. & Robert Veel (eds.), Reading Science. London: Routledge, 87-113.

Lynch, Michael. 1990. “The externalized retina: Selection and mathematization in the visual documentation of objects in the life sciences”. In Lynch, M. & S. Woolgar (eds.), 156-186.

Lynch, Michael. 1991. “Science in the age of mechanical reproduction: Moral and epistemic relations between diagrams and photographs”. Biology and Philosophy 6, 205-226.

Lynch, Michael & S. Woolgar (eds.). 1990. Representation in Scientific Practice. Cambridge, MA: MIT Press.

Miller, Thomas. 1998. “Visual persuasion: A comparison of visuals in academic texts and the popular press”. English for Specific Purposes 17/1, 29-46.

Mukerji, Shandra. 1985. “Voir le pouvoir : la cartographie au début de l’Europe moderne”. Culture Technique 14, 209-223.

Myers, Greg. 1990. Writing Biology. Texts in the Social Construction of Scientific Knowledge. Madison, WI: University of Wisconsin Press.

Ochs, Elinor & Sally Jacoby. 1997. “Down to the wire: The cultural clock of physicists and the discourse of consensus”. Language in Society 26, 479-505.

O’Toole, M. 1994. The Language of Displayed Art. Leicester: Leicester University Press.

Roth, Wolff-Michael & G. Michael Bowen. 1999. “Digitizing lizards: The topology of ‘vision’ in ecological fieldwork”. Social Studies of Science 29/5, 719-764.

Rowley-Jolivet, Elizabeth. 1998. La Communication scientifique orale : Étude des caractéristiques linguistiques et discursives d’un genre. Thèse de Doctorat, Université  Bordeaux 2.

Rowley-Jolivet, Elizabeth. 1999. “The pivotal role of conference papers in the network of scientific communication”. ASp 23-26, 179-196.

Rowley-Jolivet, Elizabeth. 2002a. “Science in the making: Scientific conference presentations and the construction of facts”. In Ventola E., S. Thompson, & C. Shalom (eds.), English as a Conference Language in Europe. Frankfurt: Peter Lang.

Rowley-Jolivet, Elizabeth. 2002b. “Visual discourse in scientific conference papers. A genre-based study”. English for Specific Purposes 21/1, 19-40.

Rudwick, Martin. 1976. “The emergence of a visual language for geological science 1760-1840”. History of Science xiv, 149-195.

Veel, Robert. 1998. “The greening of school science: Ecogenesis in secondary classrooms”. In Martin, J. R. & Robert Veel (eds.). Reading Science. London: Routledge, 114-151.

Winter, Eugene. 1994. “Clause relations as information structure: two basic text structures in English”. In Coulthard, M. (ed.) Advances in Written Text Analysis. London: Routledge, 46-68.

Young, Lynne. 1990. Language as Behaviour, Language as Code. Amsterdam: John Benjamins.

Haut de page

Annexe

Appendix 1 - Scientific conferences filmed

A. Geology
1. EUG VII: European Union of Geosciences, 4th-8th April 1993, Strasbourg.
2. International Symposium on Mineralization related to Mafic and Ultramafic Rocks, 1st-3rd Sept. 1993, Orleans.

B. Medicine
3. First International Symposium on Conservative Treatment in Oncology, 17th-19th June 1993, Lyon.
4. First Annual European-American Conference on Gastrointestinal Oncology: Cancers of the Lower Gastrointestinal Tract, 22nd-24th Sept. 1994, Bordeaux.

C. Physics
5. Euro ElectroMagnetics: International Symposium on Electromagnetic Environments and Consequences, 30th May-4th June 1994, Bordeaux.

Haut de page

Notes

1  See Rowley-Jolivet 1998 and 2002b for details of the classification system used.

2  The data used here are composed of video recordings made in 1993-94 of five international conferences in geology (mining geology and sedimentary geology), medicine (oncology), and physics (electromagnetism); 90 presentations (30 from each field) were analysed, comprising 2048 visuals. See Appendix 1 for details of the conferences filmed.

Haut de page

Table des illustrations

Titre Figs. 1 - 4
Crédits In Bourdon 1999, reproduced with the kind permission of F. Laggoun-Défarge
URL http://asp.revues.org/docannexe/image/2093/img-1.png
Fichier image/png, 1,0M
Titre Fig. 5. van Krevelen diagram
URL http://asp.revues.org/docannexe/image/2093/img-2.png
Fichier image/png, 882k
Titre Fig. 6
Crédits Reproduced with the kind permission of F. Laggoun-Défarge
URL http://asp.revues.org/docannexe/image/2093/img-3.png
Fichier image/png, 914k
Titre Fig. 7. Time & space relations in scientific conference presentations
URL http://asp.revues.org/docannexe/image/2093/img-4.png
Fichier image/png, 629k
URL http://asp.revues.org/docannexe/image/2093/img-5.png
Fichier image/png, 5,9k
URL http://asp.revues.org/docannexe/image/2093/img-6.png
Fichier image/png, 5,9k
URL http://asp.revues.org/docannexe/image/2093/img-7.png
Fichier image/png, 5,9k
URL http://asp.revues.org/docannexe/image/2093/img-8.png
Fichier image/png, 6,0k
URL http://asp.revues.org/docannexe/image/2093/img-9.png
Fichier image/png, 6,0k
Titre Fig. 8. Sequential knowledge
URL http://asp.revues.org/docannexe/image/2093/img-10.png
Fichier image/png, 410k
Titre Fig. 9.  Schematic illustration of particle growth in dusty plasma
Crédits Reproduced with the kind permission of L. Boufendi & A. Bouchoule
URL http://asp.revues.org/docannexe/image/2093/img-11.png
Fichier image/png, 1,1M
Titre Fig. 10 . ‘Noisy’ graph
URL http://asp.revues.org/docannexe/image/2093/img-12.png
Fichier image/png, 1,3M
Haut de page

Pour citer cet article

Référence papier

Elizabeth Rowley-Jolivet, « Image as Text. Aspects of the shared visual language of scientific conference participants », ASp, 27-30 | 2000, 133-154.

Référence électronique

Elizabeth Rowley-Jolivet, « Image as Text. Aspects of the shared visual language of scientific conference participants », ASp [En ligne], 27-30 | 2000, mis en ligne le 27 janvier 2011, consulté le 19 août 2017. URL : http://asp.revues.org/2093 ; DOI : 10.4000/asp.2093

Haut de page

Auteur

Elizabeth Rowley-Jolivet

Elizabeth Rowley-Jolivet est maître de conférences à l'Université d'Orléans. Ses domaines de recherche sont la communication scientifique orale et le passage oral-écrit, la sémiotique visuelle et la sociologie de la science. Elle est membre du groupe de recherche FRE 2203 du CNRS 'OSTERLITS' (Oral/Oural : Systèmes en transit et recherches linguistiques sur la traduction du sens). Elizabeth.Jolivet@univ-orleans.fr

Articles du même auteur

Haut de page

Droits d’auteur

Tous droits réservés

Haut de page
  • Logo GERAS -Groupe d'Etude et de Recherches en Anglais de Spécialité
  • Revues.org