ABSTRACT

We consider multi-user game engines such as Epic Megagames' UNREAL engine to be extremely useful tools for the design of knowledge spaces.

For a collaborative project with 10 Viennese museums [1] we developed a semantic matrix for a cross-disciplinary exhibition showing items from different collections (Sigmund Freud Museum, Jewish Museum, Museum of Natural History...). The content provided by these museums had to be made accessible and comprehensible to users of different ages, educational backgrounds and computer literacy. We developed a system of connotations amongst the objects, which then was translated into a spatial structure of rooms, corridors and places of different sizes, shapes, remotenesses or proximities. The viewer/listener of our knowledge space explores a semantic structure by navigating virtual spaces with the topics being contained in these rooms. The connecting architecture between these rooms resembles staircases, passages, elevators, hidden doors or portals, each of them referring to the nature of the connotation. Quite contrary to web-based databases and hypertext structures, the links therefore possess a quality of their own, carrying much more information than just "is connected with".

Categories and Subject Descriptors [Game Design]:  Semiotic aspects of game design

General Terms Design, Experimentation, Human Factors, Theory

Keywords Knowledge Space, Mnemosyne, Game Culture, First Person Shooter

 

1.     History

The concept for our computer-aided knowledge space is related to techniques of Mnemosyne, used by Greek "singers" (Simonides of Cheos) and philosophers as well as Renaissance scholars. [1] This form of mnemotechnique, called loci or place method, was widely used by orators to memorize complete speeches. The orator picked a building and learned every nook and cranny very intensely until he was able to move about the building in his memory. As a preparation for the speech a plethora of items of different complexity and amount of detail could be placed in the memorized rooms, e.g. a scale for justice etc. While delivering the speech the orator wandered from room to room and collected the hints while the speech unfolded.

Figure 1: Mnemotic system with abbey
(Johannes Romberch: Congestorium Artificiose Memorie, Venice 1533)

Figure 2: Items used within the mnemotic system of Romberch

Obviously the urban structure of the mnemotic system helped in keeping a sense of orientation amongst the rather abstract items under investigation. That is why urban metaphors and architectural metaphors were so popular with medieval mnemotechniques.

In another respect our computer aided knowledge space also adopts techniques developed by Aby Warburg [2] for his research on the visual codes of Renaissance art. Warburg's scientific method consisted of connecting seemingly unrelated imagery to gain insight into visual similarities and connotations, which he called Pathosformeln [3][4]. In our knowledge space the multiple coding of meanings contained with the exhibited objects is made transparent by the spatial relation superimposed upon the objects. (A technical drawing of a prosthesis, e.g., is positioned close to Freud's Prothesengott quote and therefore connected to Freud's theory from "Das Unbehagen in der Kultur". The latter might lead to beautifully painted transportation vehicles from Pakistan which have been supplied to us by the Museum of Anthropology.)

 

2.     Interface

To navigate the spaces of different content the users have to keep moving. They can walk, run, climb, jump, crouch, swim or fly according to the spatial situation. The Frankfurt based cultural scientist Manfred Fassler has mentioned in his recent publication that the etymological root of the German word for experience (Erfahrung) stems from fahren, i.e. "to move". We consider the process of actively exploring a quasi-spatial structure as the key mechanism for creating a semantic structure that is neither linear nor hierarchical. We consider the technology of a computer game as a helpful tool for the mediation of complex content. We also consider the freedom of the user to go his or her own way in the virtual environment as an important feature that allows for individually shaped relational networks inside a complex field of knowledge.

3.     Developing a Structure for Knowledge Representation in Regards to a Concept of Spatial Organisation

Our approach of putting items containing multiply coded meaning into rooms which then are connected by other spatial structures poses a few questions which we tried to answer from a scientific standpoint as well as an artist's viewpoint:

1. How does the appearance of the rooms contribute to the meaning of the objects contained in these rooms?

2. How does the appearance of the connecting rooms contribute in signifying the relations between objects?

3. How does the appearance of the learning subject contribute to the learning process and the possible modes of acquiring knowledge?

4. How do modes of movement contribute to emphasize certain topics in the field of knowledge?

5. Is it essential for users to have a bird's eyes view of the semantic terrain they are investigating by cruising it?

6. What types of interaction can there be between multiple users exploring the same field of knowledge?

  Even though these questions might have been approached at a rather pragmatic level there is a semantic question lying underneath. If we are going to map semantic fields (semantische Felder) into fancy rooms, the signified (Signifikate) into topics, and the signifiers (Signifikanten) into 3D-objects or sound objects, does the game we are about to be immersed with still correspond to the semiotic structure we started with? If we suppose that we had a clear concept of a semantic structure, e.g. in the form of a Quillian model [5] or a Katz and Fodor structure [6], could we rediscover the model or structure in our game then? In other words: is there an isomorphism of the semantic structure of the object under investigation and the spatial structure of the "level"? [2] We don't think so.

At least in the case of our approach there is a large amount of deliberate distortion of the semantic structure underway when we build the levels. We tend to concentrate on information which is feasible to be displayed in the game engine's range of possibilities. We also have to chop off branches of the semantic trees the technological platform we work with would not allow at present. Furthermore we often add connotations in the process of construction for the reason of subjective choice and media-specific deliberation. We do not intend to rebuild the museums or the structure of knowledge contained in the museums in electronic space. Our main goal is to find an artistic form of communicating topics which have traditionally been attributed to the museums. For that reason our work should be considered experimental.

 

Figure 4: Another user in front of the Virtual Technical Museum (Virtueller Wissensraum by fuchs-eckermann)

  The game engine we decided to use was not developed before 1999; therefore there are not many implementations of the kind of game we are interested in. Nevertheless, a few projects have been carried through which differ from the primitive look and feel of the original game. We are aware of the fact that our investigation is based on a very small sample of persons we could watch using the game. Our second knowledge space ”Expositur” is at this moment only in a prototype state and still has to be considerably extended. Nonetheless there are a few hypothetical propositions we would like to make:

3.1      Appearance of Rooms

It seems of extreme importance for the creation of meaning how the environment the information is contained in, looks like and sounds: Within UNREAL we can create 3D-shapes for the rooms of a complexity of about 10,000 polygons. We recognized that the richness of the architectural forms decreases the stress a person feels confronted with, when navigating through this space. Former versions of UNREAL allowed for just a few hundred polygons and therefore favoured boxy levels creating a higher aggressiveness and a feeling of discomfort for certain users. On the other hand we recognized that very complex environments often create a feeling of "being lost" and of nausea. UNREAL Tournament at present allows using 8 sound slots at the same time, i.e. a number of 8 sound sources that may be set at any location in the rooms and can be heard simultaneously. The function of sounds accompanying the objects exhibited in the rooms can be analyzed in analogy to the function of sounds in movies:

  3.1.1     Emotional Support

The sounds can deliberately be used to value objects as dangerous, hilarious, important, historically significant or other. We use sound in this respect to add ethical standpoints to objects we have to show, like armour, prostheses or extinct animals.

3.1.2     Additional Information about Visible Object

We use recorded sound material to tell about the material qualities of objects.

3.1.3     Continuity Music

We recognized that turning off the background music of the rooms in a computer game results often in a much shorter playing time. Also the speed of the investigation, the restlessness and the carefulness of exploring a specific room can be manipulated via music played in the background.

3.1.4     Subtext

This is an important function for content creation of an ambiguous character or for the creation of content which can be interpreted in different ways. (In other words, this is important for any content.) For our game of knowledge we used the method of acoustical subtext in the room showing the collection of technical prostheses. The collection intends to throw a critical glance upon the dark sides of technological progress. In 1917 a workshop for prostheses was set up for workers having lost limbs when working on dangerous machinery. Of course in 1917 the number of persons being wounded in World War I was very high. The soundtrack accompanying the prostheses hints that capitalist production and warfare can result in the same sad results for the victims of either. This information, when put forward as a text, would sound quite banal and not lead to an intensive experience for most of the users. However, the soundscapes of heavy machinery mixed with heavy artillery and superimposed with composed rhythmical patterns does.

3.2     Appearance of Links

It has been mentioned earlier that we use staircases, passages, elevators, hidden doors and portals to signify the nature of the connotation, and in a second step: the nature of the relation between objects and topics. As Peter Kivy points out in his book "The Corded Shell" [7], there are "contour" features of visual objects and of acoustic objects, which are connoted with emotions. A rising line is usually considered to be positive or optimistic; a falling line to be negative, disappointing or dangerous. These features have to be carefully observed when laying out the spatial structure of the knowledge building. Also the size of passages can invite users to select the larger corridor for the further way instead of the smaller one. This is however not always the case. We recognized that frequent players often prefer the less obvious way instead of the more obvious one. This might have to do with an anticipation of tricks the game designer might have had in mind. It could also be that all users do not always enjoy the easiest way.

 

3.3     Appearance of the Learning Subject

UNREAL allows for the user to select "skins" from a set of stereotypes while running the game. We provide the same possibility for our knowledge space. After the avatar [3] is visible when exploring the knowledge space, it influences the behaviour of the player. There are role models connected to certain "skins" which are very hard to predict. We found out that it is not necessarily the case that boys select male avatars and girls select females. It might be that the choice of an avatar has to do with what Denis Smalley calls levels of identification [8]. We suppose that certain users identify themselves strongly with the avatar whereas others find themselves outside the virtual space directing or leading a puppet through a simulated environment.

3.4     Modes of Movement

We consider the selection of a mode of movement by the player/user to be essential for the way the user structures his/her personal system of relevance. By freely selecting the speed, the user can declare topics as being relevant to him or of lesser importance. As opposed to e-learning systems based on video footage, the user of the knowledge space can find out his or her own pace in acquiring knowledge and can build his personal map of the field of knowledge. This is not always the case with computer aided systems. As Dave Campbell correctly observes, "software functionality like pre-defined paths of movement and pre-defined view-points of key spaces diminish a participant`s ability to construct an accurate cognitive map of the space" [9] We found out that different player types like to keep running all the time, or relax and just look around. UNREAL includes possibilities to force the user to move at certain points. The technical term for such objects is "movers", "teleporters" and "translocators". Movers are objects that can change location or move players when triggered. Movers might be used for situations resembling a daydream or a gentle shift of focus. Teleporters instantly move from one room to another. In terms of the knowledge represented by different objects, this means that you are taken over by a spontaneous idea or an abrupt change of mind. Translocators on the other hand are devices allowing the player to go from one point to another deliberately and quickly. In some way translocators can be compared to the rhetorical formula "Let's change the topic."

 

Figure 5: Running Avatar (Virtueller Wissensraum by fuchs-eckermann)

 

Another way of influencing the reception of content by changing the mode of movement of the player is changing the physical properties of the location or room the player is in. The designer of the knowledge space is not only able to change the gravity of the room but also to set the friction of the player on the ground. In this way certain topics can be presented on a ground which feels slippery or sticky. The player can be given the feeling of moving on thin ice or of being in a situation which makes it hard for him/her to move. It is obvious that this must have an effect on the care the player takes in acquiring the appropriate information. It probably also effects the level of satisfaction/frustration in gaining information.

3.5     Indexes, Survey

We are not sure whether a survey on the knowledge space is useful or desirable for the user. UNREAL provides the possibility to rise up into the sky (the process is called ghost mode) and take a look down upon the architectural structure of the knowledge space. The survey might increase the chance to go on once one is lost or caught in a labyrinth, at the same time it probably takes away the incentive for trying very hard to find out what one not yet knows.

3.6     Types of interaction

The modes of interaction UNREAL offers are not really useful for knowledge space design with multiple users. (They are with slight differences based on the imperative "Shoot them before they shoot you!") What we found most useful for real-time multi-user situations is the possibility for one knowledge seeker to show others their way by guiding them through the rooms. Another useful feature is the possibility to exchange messages via written or spoken word. This feature makes knowledge spaces of the kind described feasible for geographically spread networks of students, scholars and players. It is essential however to implement individual sets of voices and not to rely on the default voices provided by the manufacturer.

4.     Implications

Multi-user games will play an important part in the field of e-learning and the visualisation of knowledge. This fact is complained about by one side and advocated by the other. The discussion of the pros and cons of an audio-visual mode of knowledge representation versus a text-based one mirrors old discussions about the value, truth and appropriateness of images [10]. We would like to suggest however, that audio-visual virtual environments will gain importance during the next decade. The fact that the gaming industry gains ground compared to elder forms of edutainment like movies, video and printed text is an argument at least for not ignoring computer games. We think that these games - however simple they still might appear at present - contain possibilities for knowledge spaces of a delicate nature - if they are thoughtfully conceived, carefully designed and joyfully experienced.

  (Mathias Fuchs and Sylvia Eckermann, e-mail: fuchs-eckermann@t0.or.at)

 

5.     ACKNOWLEDGMENTS

Thanks to Manfred Fassler for critical discussions and to Renée Gadsden for proofreading the paper. The project has been carried out with the support of Austrian Bundesministerium für Verkehr, Innovation und Technologie and Bundeskanzleramt Sektion für Kunstangelegenheiten.

 

6.     REFERENCES

[1] Yates, Frances A.: The Art of Memory. 1966.

[2] Gombrich, Ernst H.: Aby Warburg. Eine intellektuelle Biographie, Frankfurt, 1981.

[3] Roeck, Bernd: Psychohistorie im Zeichen Saturns. Aby Warburgs Denksystem und die moderne Kulturgeschichte In: Hardtwig, W. /Wehler, H.-U. (eds.), Kulturgeschichte Heute (Geschichte und Gesellschaft, Sonderheft 16), p. 231-254.

[4] Wuttke, Dieter (ed.): Aby M. Warburg. Ausgewählte Schriften und Würdigungen, Baden-Baden, 1980.

[5] Quillian, Ross M.: “Semantic Memory”. In: Minsky, Marvin (ed.): Semantic Information Processing. MIT Press, Cambridge, 1968.

[6] Katz, J. J. and Fodor, J. A. (eds.): The Structure of Language. Prentice-Hall, Englewood Cliffs, 1964.

[7] Kivy, Peter: The Corded Shell. Princeton University Press, Princeton N. J., 1980.

[8] Smalley, Denis: Spectromorphology: Explaining Sound-Shapes. In: Organised Sound. Vol. 2(2), Cambridge University Press, 1997.

[9] Campbell, Dace: Design in Virtual Environments Using Architectural Metaphor, and also Daniela Bertol: Designing Digital Space. An Architect`s Guide to Virtual Reality, New York 1995, p.64 f. and pp. 88.

[10] Reck, Hans Ulrich: Konstruktion des Erinnerns: Transitorische Turbulenzen  3. Mnemo-Topologien: Die Abwesenheit Gottes. Kunstforum Cologne July - September 1994.