1I am not infatuated or obsessed with high-tech tools to a fault, and not a cyberpunk or eccentric utopian visionary yet, so don’t expect this exposé to be about smart drugs, psychedelics or a piece of futuristic surrealism. Yet, since I have just been allowed my first steps into the 21st century, I would be happy to take you to Cyberspace, or on America’s electronic “data highways” like the huge spidery Internet, the globe-circling interconnected telephone network that is the conduit for billions of voice, fax and computer-to-computer communications.
2However, in order to reassure the most doubtful, I shall stick to the most traditional academic and classical French 3-part dissertation —and manage to quote Descartes, and Shakespeare— to surf the dark edges of the computer age. Of course, if necessary, I could even go back as far as Plato, since in his Allegory of the Cave in The Republic we are never allowed to see the real world, but an illusion based on reality, a virtual world.
3After these oratorical precautions, I shall introduce the issues at stake. In my first part, I shall attempt to define VR and consider how its characteristics or components meet the needs of language teaching. In my second part, I shall describe the achievements of VR and present developments to differentiate it from science fiction, and to show how VR communities, networks, entertainment and realisations could be used for language work. And finally, more specifically, I shall in my third part analyse its developments in medicine, from the point of view of English for Specific Purposes (ESP) language training because the existing possibilities in this field are already immense.
- 1 “I confess that it had never occurred to me that VR could be a good tool for teaching a foreign lan (...)
- 2 In 1990, the president of Nintendo Corporation announced in Kyoto a $3 million grant for Papert’s c (...)
- 3 Jaron Lanier, 1989, Interview in Whole Earth Review, Fall, Sausalito, CA, USA.
4I shall start, in fact, where Howard Rheingold’s state of the art publication – Virtual Reality – more or less stops. He wrote to me that he had not thought, in particular, of using VR to learn a foreign language,1 and yet, in his last but one chapter, he approached the general question of education, considered in the light of Papert’s theories about human learning.2 I do not mean the failed “computer-assisted instruction” concept of the 1960s. VR is not just another powerful educational tool to be identified with TV. “It is the very opposite of television,” as Jaron Lanier, one of the founders of VR, explained3 as early as 1989. And as Karl Sims put it during his last conference in Paris: “VR is an interactive environment... You can manipulate it, you are free to create your own version of it while television, which is still the reference, displays programs you can only receive.” (PIXEL 19) By shifting the barrier separating concrete and formal processes it is possible to amplify and extend first-hand experience, to enrich perception and understanding as well as the effectiveness of the hands-on approach.
5I shall therefore consider the opportunity to create ‘learning worlds’ after showing how the components of VR answer the needs of language work.
Of course, as Rheingold notes:
...much design and research remain to be done before anyone knows whether VR could furnish a magical sandbox for children and teachers to directly experience simulated worlds of history or biology, but Frederick Brooks pointed back to the American educational pioneer John Dewey when he summed up the educational potential of VR in the words ‘learning by doing.’ What if educational micro worlds could be designed in the form of the cyberspace playhouses Jaron Lanier and Randal Walser suggest? For those who wonder why kids love video games and hate school, and wonder if there’s something to be learned there, the future of the commercial electronic education industry will be instructive. (Rheingold 1991: 377)
6Can technology and commerce succeed, where state run school systems have failed? If you have to attend deteriorated schools and your parents can’t afford Nintendo’s educational supplements, will you end up “with the other informationally disenfranchised earthlings of the data age?” (Rheingold 1991: 377) Are we to leave Fujitsu’s plans to create educational VR simulations and decide about education and its contents, the key obstacle to getting an education in the future being simply the lack of access to direct-experience education technologies?
- 4 A figure with a thousand sides.
7We must keep in mind that we are actually on the brink of acquiring the power to create almost any experience we desire —even build on screen the ‘chiliogone’4 Descartes described in his Sixth Meditation as impossible not only to represent but to imagine! Therefore we have to decide fairly soon what kind of partnership we want to develop with our information tools $: the question is no longer what we can do, but what we want to do, what we want to change into and what we do not want to change into, and how we get the power to make the decisions as University researchers, before we have imposed on us a de facto silicon partnership by the media. And when I use the expression ‘we are on the brink of’ this is to insist on the immediacy of the phenomenon. As Philippe Quéau, responsible for IMAGINA programs and Directeur de recherche at INA writes in Le Virtuel, vertus et vertiges:
Les mondes virtuels doivent être pris au sérieux. Ce ne sont ni des gadgets coûteux réservés aux pilotes de chasse et aux astronautes ni un épiphénomène amusant, un délassement futile, une retombée mineure de la considérable révolution infographique...
Les systèmes de simulation virtuelle d’entrée de gamme ne coûtent que quelques dizaines de milliers de francs seulement, en attendant des chutes de prix considérables, accompagnant l’essor de la technologie RISC, qui mettra rapidement la synthèse d’images en temps réel (et donc les mondes virtuels) à la portée des ordinateurs personnels... Avant la fin de la décennie, le casque virtuel sera un équipement courant et confortable... (Quéau 1993: 14-15)
8First and foremost:
What is Virtual Reality?
9Of course there is no standard definition for this experience in some sort of willing suspension of disbelief...
10If computer graphics was the buzzword of the 1980’s, virtual reality is the buzzword of the 1990’s. But is it so new? After all for over a decade now we have had virtual computing machines, virtual memory, and virtual environments, and opticians a century ago discussed virtual images. (Rosebush 1994: 47)
Is VR something more than this or is it just hype?
11It has been argued that this oxymoron should be better termed ‘cyberspace’ but apparently William Gibson in his Neuromancer had claimed the coining and trademark of that word, so ‘artificial reality’ or ‘VR,’ apparently first used by Jaron Lanier, was adopted. Now, “...to choose one old-fashioned word to describe the general category of what this new thing might be, ‘simulator’ would be my candidate.” (Rheingold 1993: 16) I would personally add the idea of ‘immersion’ —using stereoscopy to create the illusion of being inside a computer-generated scene, and the idea of ‘navigation’ or ‘surrogate travelling’ —creating a computer model of a molecule or a city and enabling the user to move around, as if inside it:
Imagine, a wraparound television with three-dimensional programs, including three-dimensional sound, and solid objects that you can pick up and manipulate, even feel with your fingers and hands. Imagine immersing yourself in an artificial world and actively exploring it... Imagine you are the creator as well as the consumer of your artificial experience, with the power to use a gesture or word to remold the world you see and hear and feel. That part is no fiction. The head-mounted displays (HMDs) and three-dimensional computer graphics, input/output devices, computer models that constitute a VR system make it possible, today, to immerse yourself in an artificial world and to reach in and reshape it. (Rheingold 1993: 16)
12VR technology resembles, and is partially derived from, the flight simulators used by the Air Force and commercial airlines to train pilots without leaving the ground, by practising with a replica of aeroplane controls. The entire simulated cockpit is mounted on a motion platform that moves in accord with the movements of the simulated aeroplane. VR is also a simulator, but instead of looking at a flat, two-dimensional screen and operating a joystick, the person who experiences VR is surrounded by a three-dimensional computer-generated representation, and is able to move around in the virtual world and see it from different angles, to reach into it, grab it, and reshape it. The operator is no longer a passive observer of this environment —as a kid glued to TV, or as students ‘watching’ a video; “...immersion and navigation constitute the elements of a new kind of beast, ‘a personal simulator’,” as Rheingold dubbed it (Rheingold 1993: 100), able to clone himself even and send his proxy image throughout the world.
- 5 Voir également Le Figaro 31 août, 1 et 2 septembre 1993, “La réalité virtuelle, nouvelle reine de l (...)
13At the moment it is still necessary to put a high-tech helmet on your head, or a pair of electronic-shutter glasses, and slide a special glove on your hand to experience that world. (Science & Vie 1993b: 82-83)5 Miniature display screens in the helmet, linked with a device tracking the position of the head, create the illusion that the screen surrounds you on every side. The reality engine updates the way you see the world when you move your head or your gaze. You can look behind computer-generated objects you see, pick them up, examine them, manipulate them and walk around them. (See illustrations 1,2,3, Fisher et al.1986).
- 6 For example the Biomuse system, an interface which records and transmits to a computer the neuromus (...)
14In the future, less intrusive technologies will be used to create the same experience: indeed eye-tracking and gloveless gesture-sensing technologies will simplify the user’s costume or data suit, and his access to virtual reality.6 As Derrick de Kerckhove’s talk at the Centre Pompidou, in June this year illustrated, the act of looking becomes, as Thomas of Aquinas suggested, ‘a very subtle form of touch.’
15Already in 1965, Ivan Sutherland wrote in The Ultimate Display:
The computer can easily sense the positions of almost any of our body muscles. So far only the muscles of the hands and arms have been used for computer control. There is no reason why these should be the only ones...Machines to sense and interpret eye motion data can and will be built...An interesting experiment will be to make the display presentation depend on where we look. (Rheingold 1991: 224)
- 7 ATR opened in Kansai in Spring 1989.
- 8 Dossiers de l’audiovisuel, 40 1991. La Documentation Française, INA, 48.
16Now the gaze is as powerful as the hand: for instance, an infrared sensor can determine precisely the direction of the gaze by detecting movements at the back of the retina. It was first operational in 1982 (Furness 1978), to activate simple commands like ‘on’/’off’ or ‘go there,’ ‘ stop here’ in a cockpit. Tiny air pads in the gloves signalled to the pilot that a button had actually been pushed. In 1992, at Scott Fisher’s virtual reality exhibition at the Centre Pompidou, you could modify a picture while just looking at it. In the VR lab at ATR (The Advanced Telecommunications Research Institute International)7 in Japan’s Kansai Science City, the goal seems to be mass market “wireless VR,” 3D graphics without the use of head-mounted displays, coupled to remotely sensed gestural input that uses cameras and scene interpretation software rather than gloves and suits to track posture, gesture, and direction of gaze. More sophisticated still, the interactive Brain Wave Visual Analyzer (IBVA), allows transmission of brain wave data and the Brain Response Interface directly reads the cerebral activity corresponding to a given visual stimulus in the case of major paralysis and motor problems with the implantation of electrodes in the brain (Actes Imagina 1993: 156).8 Tools for Thought indeed... (Rheingold 1985)
17To sum up, the characteristics of VR can be defined according to Laurin Herr and Judson Rosebush as the following four Pillars of VR (Rosebush 1994):
- Photorealistic 3D environments
- Actions created by dynamic models
- Multisensory interactions
- Shared with others via networks.
- 9 Steve Tice was chairman of the 1992 SIGGRAPH Tomorrow’s Realities gallery.
18Although the ‘four pillars’ represent a meaningful model, they are not the only structural decomposition. Steve Tice suggested four interesting axes of VR. Though his classifications on sensors and generators are quite insightful, and perhaps do not encompass the rich variety of all elements combined in VR, I chose his representation which particularly fits the requirements in the field of language work: (See illustrations 4,5,6 from Steve Tice et al., Tomorrow’s Realities, Siggraph9 ’91.)
19- Immersion level
- Participation level
- Interaction level
- Reality level
20So we can now turn to the experiences of VR proper to make sure they are not science fiction and that “... CAD, entertainment, and education [are] the three markets Fujitsu [plans] to address.” (Rheingold 1991: 287)
21We have all seen synthesised pictures but we must clearly differentiate them from VR and this is why I am now going to comment briefly on the two short video extracts that were shown during my talk in order to make the difference visible:
- 10 Slides provided by the courtesy of IBM.
- 11 Cluny Abbey was resuscitated by Arts et Métiers engineers with the technical support of IBM. This y (...)
22First, the superb images of the paintings in the Grottes de Lascaux. Second, the first ever virtual meeting in a non-real Cluny Abbey10 between participants actually in Paris and Monte-Carlo, which took place at IMAGINA 1993.11 Look up and you will see the vaults of the choir, turn round to discover the nave and round again for the transept, and yet, the Abbey was destroyed more than two centuries ago. The illusion is not yet quite perfect and too quick a movement of your head will panic the computer, but with better software and more calculating power, there is no real obstacle to a truer to life virtual reality.
23While I have borrowed further examples from the medical field, to concentrate here on the teaching of English for Specific Purposes, we could just as well have chosen a theatrical experience: imagine acting the part of, say, Hamlet, on a virtual stage, according to the rules of dramatic interaction set down by Brenda Laurel in her thesis Computers as Theatre (she is now working at Carnegie-Mellon University) to create the ‘personalities’ of artificial characters.
How these characters might act and respond is easily modeled and tested in text; and as anyone who has ever played a text-only adventure game can attest, ample mimesis can be triggered by interaction with a well-constructed narrative. Those theories of drama should be testable by trying to build the dramatic structure of a do-it-yourself computer-based fantasy world. (Rheingold 1991: 308)
24Indeed, already, we must be aware that a kind of text-based cyberspace phenomenon has broken out all over the Internet: the latest fad adventure cyberspaces being known as “MUDs.” MUD stands for “multi-user dimension.” A player logs onto a computer and using the right passwords, creates and incarnates a character. There are public and private MUDS where the wizards —or system administrators— decide who can enter or not. When goggles and gloves and protocols for transmission of presence make it possible to jump right into a graphic MUD, there will be a population of thousands of sophisticated architects/players. Of course, the idea of 3D movies that play along with your actions in real time is not something we can expect to happen this year or next, but maybe “...we ought to think where this is all leading, before we find that we’ve been led somewhere by the technology’s economic imperative, the way we were swept away by commercial network television.”12 “If you don’t control technology it will control you” (Elmer-Dewitt 1993).
25And here indeed is where our research can come in: first by not always being left behind by purely ignoring what is happening in the Japanese and American research labs or pushing it so far away into a distant future that we will never be concerned with it instead of taming it and tailoring it to our needs. “Seeing is believing, and hearing something that syncs with what you are seeing is better yet, but feeling something with your hands entails a deep, visceral kind of ‘knowing’ that is normally associated with only the most “rock-solid” realities... and this is where, to me, the field of language teaching could be successfully explored. At the heart of VR is an ‘experience’...” (Rheingold 1991: 46). The great educational power of direct sensory experience to propel understanding to a new level had been underlined a generation before by Heilig:
A basic concept in teaching is that a person will have a greater efficiency of learning if he can actually experience a situation as compared with merely reading about it or listening to a lecture.
26And in 1971 he added bitterly, because his wild ideas had not been taken seriously:
For far less than the price of a bomber, we could have distributed marvellous learning environments to our major universities. (Heilig, Morton, Sensorama Simulator, US Patent #3,050,870, 1962)
27Apparently it seems to be a proven fact that a person remembers 25% of what s/he hears, 45% of what s/he both hears and sees and 70% of what s/he actually does (herself) (Dumond 1992: 85). Therefore Digital Interactive Video is already used in numerous training sessions reducing training time by 40% to 60%. It is a commonplace that, in general, memory deals with images much more efficiently than with language: the child who sees an elephant never forgets. If it is described to him as a large grey animal with a long nose, there is nothing keeping him from confusing it with a rhinoceros, which fits the same description. This is even more true when it is a question of highly sophisticated information: since there exist molecules with an identical formula but a variety of structures. A specialist in macromolecular biology for example, might, metaphorically, confuse a rhinoceros with an elephant if the description of the molecular bond is in the smallest degree incomplete.
28As an illustration, the video of an operation thanks to the facial mask will make my point much more convincing than my long speech!
29One of the most spectacular applications in the medical field is as an aid to diagnosis in dermatology: considering the fact that a lot of diagnoses are erroneous or too slow because the physician has no visual evidence of symptoms, the Derminfo program allows you to speed up medical diagnosis dramatically by providing adequate visual data —a real medical memory.
30With VR you could just as easily discuss the evolution of a cancerous cell and the various options of treatments with a colleague based in Tokyo. Or with Fujitsu we can use neuronal networks for gesture recognition and communication, so as to develop aid for physically disabled people - with just the same technique as is used by Matsushita to commercialise virtual kitchens allowing housewives to visit them virtually and optimise the organisation of the usually small surface of their flats.
31Other medical applications —they are too numerous to enumerate— consist in the functional exploration of molecules. These can be represented virtually in the form of coloured spheres, ribbons of aminated acids or sticks simulating the atom linkage. Thus their chemical reactivity can be studied: indeed the changes in the electronic envelopes of the molecules become visible when they come into contact with other molecules. Numerous such projects are being developed at the University of North Carolina, in particular to improve cancer therapy through radiation: a 3D model of the patient is visualised and the irradiation process can then be precisely elaborated taking into account the complex shapes of tumours.
- 13 Minsky coined the term ‘telepresence’ in 1979. In 1954, Ralph Mosher at General Electric developed (...)
32Remote presence can also have the delicacy of a surgeon: a person wears a jacket lined with sensors and muscle-like motors. Each motion of arm, hand, and finger is reproduced at another place by mobile, mechanical hands. Light, dextrous and strong, those hands have their own sensors, through which the operator sees and feels what is happening. Using such an instrument you can ‘work’ in another room, another city, or another country: “telepresence”13 or televirtuality is another vector of VR:
What you don’t realise until you do it, is that telepresence is a form of out-of-the-body experience... I began to accept the odd sensation that accompanied the act of transporting my point of view to that of a machine —until I swivelled my head and looked at myself and realised how odd it seems to be in two places in the same time... (Rheingold 1991: 255-256)
The strangest moment was when Dr. Tachi told me to look to my right. There was a guy in a dark blue suit and light blue painted shoes reclining in a dentist’s chair. He was looking to his right, so I could see the bald spot on the back of his head. He looked like me, and abstractedly I understood that he was me, but I know who me is, and me is here. He, on the other hand, was there. (Rheingold 1991: 264)
33Now not only can you see yourself through the eyes of the machine, but you can of course adopt any personality, as the famous waltz with a lobster in cyberspace illustrated.
34The tactile component or haptic perception —the relatively low-energy surface-level perceptions concentrated on the fingertips, or the feel of reality— added to tele-existence, can be improved by a kind of 4D experience if you sit in a moving, rolling, bumping, leaning, and shaking rocking chair which follows the rhythm of the action, the roughness and breathtaking quality of the experienced encounter being of course the criterion of excellence in most VR videogames. The Ex Machina company has completed its Ocean Shuttle, a spectacular work of computer-generated imagery, for which each seat becomes an independent mini-simulator sharing a single large screen with the other seats. The fifth dimension of olfactory data bases with the release of scents and appropriate odours to match the experience is already on the way and ‘bionic’ noses are not science fiction but operational devices in numerous food and perfume industries. As early as 1962 Morton Heilig’s primitive Sensorama combined all five senses for a wild race on a motorbike through the streets of New York.
35In Japan, the Communication with Realistic Sensations project is where overt VR is to be found. The bulk of the VR effort is focused on the enabling technologies for ‘virtual meeting spaces’ that could link teleconferencing with remote presence technologies. Telephones and computer-mediated communications do not convey the subtleties of body language and the small rapid changes in facial expression. And this is why ‘Communication with realistic sensations’ is now the ultimate purpose:
At times when physical proximity is not feasible, the experience of a simulated shared environment would be helpful for communication.
For people located far apart, if distance and time can be overcome by such a shared environment, creativity will be enhanced and understanding of intentions will be possible. Communication will indeed be enriched (Rheingold 1991: 226).
- 14 “The word ‘dildonics’ was coined in 1974 by that zany computer visionary Theodor Nelson (inventor o (...)
36This is how the head of ATR’s Artificial Intelligence Department reacts in Japan and leads the development of research, in the footsteps of Lanier’s RB2 (Reality Built for Two), a device allowing two users to connect to the same data base and to meet. One of the more provocative implications of full-body tactile feedback, the infinite variety of cutaneous patterns is explored in ‘Teledildonics’ (simulated sex at a distance), probably14 the only experience the media have decided to develop widely, as the picture on the front page of Science et Vie dedicated to VR illustrates, giving full details on the adequate remote control sensing quality of your exoskeletal contraptions or datasuit with sensors for heartbeats, brainwaves, facial expressions and eye positions.
Every nook and protuberance, every plane and valley and knob of your body’s surface, will require its own processor... The engineering problems in building the transducers, the parts of the system that communicate in a form that people can squeeze and scratch, stroke and probe, may be formidable, but they are already the subject of focused effort on three continents... One of the most sophisticated approaches –‘Smart Skin’– is being developed at the University of Pisa by Italian engineer Danilo De Rossi, who has closely modelled an artificial skin. (Rheingold 1991: 347)
37Of course we could conclude that among the dramatic choices which will have to be made regarding the possible uses of VR, it would be safer and preferable to make love and not war with any partner around the world connected to your system —getting at the same time the opportunity to learn a little of his/her language.
From remotely operated machine guns to systems for liberating quadriplegics from the prisons of their bodies, teleoperator technology exemplifies the full range of potential applications for VR interfaces, from the darker to the higher sides of human capabilities. (Rheingold 1991: 265)
38Of course, if I have pointed to the field of medicine as a privileged one to start working on, it is because, as you can see, the examples of applied uses of VR are too numerous already to quote them all —whether we consider aid to diagnosis, surgery practice and training, biology, cancer research, aid to the visually impaired with the Guide Dog Robot called MELDOG, or aid to quadriplegics... From this year onward new students in medicine in French universities will be taught English, a mere sign of the importance of the issues at stake. From the rudiments which they could visualise and manipulate, experiencing all the possible movements of muscles and bones, to a teleoperation or a teleconference with the USA, whatever their level, we could produce invaluable training for them in language through the use of VR. For years we have heard about the total failure of language teaching because immersion or bains de langue, communication, actual situations, interactive experience, games, role-play were deemed to be THE ideal and the only positive ways of teaching a language successfully. I suppose these do correspond exactly to our definition of VR. If we added the personal freedom to explore and create any experience, we could even conclude that VR communication is superior —if this did not sound too much like QT 1’s demonstration in Asimov’s short story Reason on the superiority of robots compared to human beings!
39“Whereas film is used to show a reality to an audience, cyberspace is used to give a virtual body, and a role, to everyone in the audience...” (Walser 1990)
If All the world’s a stage...as Shakespeare declared, and if all the world is in the process of being wired into a global electronic media space as McLuhan prophesied, and if mass media reality networks of the future might enable anybody anywhere to experience directly any role in any cyberspace they can create or afford...would it surprise you that the education industry of tomorrow might be revolutionised by MIT ...and Nintendo? (Rheingold 1991: 287).
Furness, T.A. 1978. “Visually coupled information system”. ARPA Conference on Biocybernetic Applications for military Systems, Chicago.
I should like to express all my thanks to IBM France, Medialab, Howard Rheingold, Director of the Whole Earth Review and Derrick de Kerckhove, Director of the McLuhan Program, for their invaluable help in preparing this paper.