Toward Multimedia Writing
Learning from Godard and Computer Programming
In my attempts to deal with these questions I was inspired by certain filmmakers who appear to be obsessed not simply with using cinema as a medium to convey ideas and arguments (which is what conventional documentaries are supposed to be doing) but rather as a medium capable of presenting the very process of thinking. Among these filmmakers I would single out Eisenstein, Marker and Godard.
|
|
The first tried to formulate the notion of intellectual montage and planned to film Marx's Capital. Already at the end of the 1920s he was predicting that in the future philosophy and history will be presented as films:
The proclamation that I'm going to make a movie of Marx's Das Kapital is not a publicity stunt. I believe that the films of the future will be found going in this direction (or else they'll be filming things like The Idea of Christianity from the bourgeois point of view!) In any case, they will have to do with philosophy...the field is absolutely untouched. Tabula rasa.
(See also)
Marker showed that cinema can be used to construct intellectual essays, the essays which associatively move from idea to image, and an image to another idea (for instance, in his recent "Level 5"). And Godard has already explored computer multimedia's new language in his films from the 1960s onward by systematically juxtaposing moving images, non-realistic sound and graphically presented texts; more recently he created a true multimedia masterpiece, an essay which takes us along his thinking process while using every multimedia code available: pages of text, still images, moving images, voice and sound ("JLG by JLG. Self-Portrait in November").
Not only to convey complex ideas through multimedia, but to take the reader along the process of thinking -- this is the challenge of multimedia writing. The use of a computer as writer's tool can only be justified if we can evolve more nuanced, more complex and more precise ways of conveying what it means to think, of how it feels to move from one association to the next, from one memory to another, from one insight to the next. Only when we will give justice to the common view of a computer, which accompanied it from its very beginnings half a century ago, as a model of a human brain. A machine which has memory, which can store words and images, which can search and match, which, most importantly, can link, i.e. associate -- even if it is not a human mind, it has most of the functions we, humans, perform when we think. Therefore, we should be able to use a computer to portray human thinking in a more precise and engaging way than literature and cinema have already done. To do this is the challenge of multimedia writing.
Earlier in this century, Proust, Nabokov, Joyce and other modernist giants came up with new techniques to represent our mental life: thinking and remembering, forgetting and repressing, formulating concepts, moving between the sensorial world outside and the mental world inside. Literature became the best mirror for the modern psyche, achieving highest fidelity in relation to our inner world. But other arts did not match its achievement. Cinema, multimedia's main precursor, is particularly disappointing in this respect. By and large, its language followed 19th century novel, rather than trying to match -- and go beyond -- the microscopic view of human inner experience recorded by Joyce, Nabokov and other moderns. This is particularly surprising, given that the first theoretical text on cinema -- Hugo Munsterberg's The Film: A Psychological Study (1916) -- proclaimed that the essence of the new medium lies in its ability to reproduce, or "objectify" various mental functions on the screen.
According to Munsterberg, "The photoplay obeys the laws of the mind rather than those of the outer world." In a provocative analysis, Munsterberg correlated the main cinematic techniques to different mental functions such as attention and memory, one-to-one. For example, in the close-up, "everything which our mind wants to disregard has been suddenly banished from our sight and has disappeared," analogous to how our attention selects a particular object from the environment. Similarly, the "cut-back" technique objectifies the mental function of memory.
Yet, despite this promising analysis made by Munsterberg early on, cinema hardly took up the challenge of being a mirror of mind's operations. In my view, the only real systematic attempt in cinema to do this has been Godard's recent work, such as already mentioned "JLG by JLG" and also the majestic and monumental "Histories of Cinema" (which currently can be seen in Documenta X in Kassel). In the latter, Godard uses new cinematic techniques of his own invention in order to portray thinking process more accurately. For instance, he often superimposes 2, 3 or more images which gradually fade in and out, but never dissappear completely, staying on the screen for a few minutes at a time. It is as though these are ideas or mental images floating around in our minds, coming in and out of mental focus. Another technique involves replacing one image by another not through a cut or a dissolve but through a repeated oscillation, with two images flickering back and forth over and over, until the second image finally replaces the first. This technique can be interpreted as an attempt to represent mind's movement from one concept, mental image or memory to another -- the attempt, in other words, to represent what, according to Locke and other associationist philosophers, is the basis of our mental life -- forming associations.
Yet, along with these innovative techniques which would certainly please Munsterberg (who, accidentally, was a professor of Psychology at Harvard University) by being visual equivalents of mental operations (or shall we say video-temporal equivalents, since time obviously plays a crucial role in Godard's techniques) Godard often "capitulates" to cinema's more conventional way of narration: showing reality (here, a person thinking) from a third person point of view, i.e. from the outside. In "Histories of Cinema," we repeatedly see close-ups of a book page, or Godard himself, standing next to a bookshelf, getting a book, reading a sentence or two; or, finally, Godard sitting at a table and typing or writing. Perhaps these can be thought of as being equivalents of "establishing shots" in traditional cinematic narration: Godard's shows us his location (i.e., his mind) from the outside, so to speak, before taking us inside. Perhaps these images can be also interpreted as challenges to the conventional cinema and its extension, computer multimedia -- lets focus on intellectual life, on human mind rather than external actions and stories.
So far a computer, despite his persistent association with a human mind, has served as even worse artistic mirror for our mind than cinema. This is strange given the fact that while only Munsterberg and few others made a connection between human mind and cinema language, in the case of a computer making similar connections became the research focus of a number of new fields, enormously successful, fields which keep expanding more and more: artificial intelligence, cognitive psychology, neuroscience -- in short, a whole set of disciplines grouped together under the name cognitive science, whose ultimate purpose is to map the language of the mind and the language of a computer one into another. While the attempts of artificial intelligence to simulate human mind have met with some limited success in such areas as parsing human speech, understanding stories, planning actions and interpreting images, the reverse problem -- the cultural problem -- using a computer to represent human mind in all its complexity and specificity (i.e., modeling not just the rational-computational part, as in artificial intelligence, but the phenomenological whole), pushing beyond what arts has accomplished so far -- was hardly even raised. Obviously, current language of multimedia -- presenting a user with a page containing a small number of links leading to other pages -- is hardly an adequate mirror of our mental life, or how we think, remember, plan, make connections.
At present, software tools themselves are more revolutionary than multimedia applications they are used to design. They are better artistic visions of our inner life. Relational databases; pointers; control structures ("if... than," "case," etc.); object-oriented programming -- these and other programming concepts point towards potentially complex, dynamic and rich cultural representations of human mind. Even such seemingly trivial concept as a hierarchical file system is already more suggestive than the typical pages with hyperlinks which are being served to us in the 1990s under the slogan of "new media." Whatever it may involve, human thinking is certainly more like a computer program under execution (which involves translating between a hierarchy of computer languages, writing and reading data, keeping track of a current place in a program, clearing space in memory for new data and so on) than a set of pages linked by hyperlinks.
To bring this new level of complexity, already achieved in software design, into the realm of cultural representation -- this is the challenge of multimedia writing. To do this, we need to be looking both at best cultural achievements in "mind modeling" -- Proust and Nabokov, Joyce and Godard -- and at the concepts of computer science, at the structure of computer hardware and software. Only when our multimedia texts will do justice both to the complexity of the machines used to compose and distribute these texts -- computers -- as well as to the complexity of what it feels to be a human being today: to think, to reflect, to carry the burden of human cultural history and of never before available amount of information and news from around the world, to interact with artificial minds of computers and with minds of other humans -- and also, as always, still to respond to the sunlight shining through a window shade, the green of grass, the movement of wind in the trees. In short, to be human, to reflect and to exist, to be inside and to outside at the same time. To represent this uniquely human, embodied thinking -- this is the challenge of multimedia writing.
My own experiments in multimedia writing started in 1992. From that year on, I would turn each of my finished papers into a multimedia presentation, using images and QuickTime movies. Other times I would put together few images whose juxtaposition, I thought, summed up my argument, and then I would develop text around them. For instance, my Ph.D. dissertation begins with a montage of three images which sums up its historical thesis about the engineering of vision: El Lissitsky's photomontage "Tatlin at Work" (1922), a photograph of SAGE operator (early 1950s) and a photograph of a VR user (late 1980s).
Next, in 1995, I begun a project which I called "Little Movies": a set of short (under 1 minute each) QuickTime films, each film acting as an entry in a diary, presenting a record of an idea, a piece of a thought, an observation. Noticing how much time I was spending to perfect each short movie, I realized that rather than being diary entries, they are rather like chapters in a book. In 1997 I "released" Volume I of this "multimedia book" as a Web site (http://jupiter.ucsd.edu/~manovich/little-movies) and a videotape. Volume II is planned for Winter 1997 release. These computer films are very, very far from the ideal I evoke here (see above) -- but I will keep trying.
Related URLs:
"little movies"
homepage
Darstellungsbreite ändern
Da bei großen Monitoren im Fullscreen-Modus die Zeilen teils unleserlich lang werden, können Sie hier die Breite auf das Minimum zurücksetzen. Die einmal gewählte Einstellung wird durch ein Cookie fortgesetzt, sofern Sie dieses akzeptieren.
