spatial objects and intelligent agents in a virtual environment

Automation in Construction 5 (1996) 141-150

Spatial objects and intelligent agents in a virtual environment 1

Maia Engeli * , David Kurmann

Swiss Federal Institute of Technology, ETH Zurich, Switzerland

Abstract

Many CAD software tools are available today for architectural design. They are useful for drafting, but tools that support design development in an early stage are still missing. In a conceptual phase of the design aspects other than precision and measurements become important. With today’s knowledge and technological possibilities new ways of interaction, different data structures and intelligent support tools can be implemented. This article describes our research on new ways to support the design development in an early stage. The concept of modelling spaces, the virtual modelling tool SCULPTOR and the integration of intelligent agents are described.

Keywords: Conceptual design in architecture; Virtual reality: Void modelling; Artificial intelligence; Intelligent agents

1. Introduction

A CAD tool should not only support the designer in modelling buildings of any level of complexity, but it should also support the development of conceptual design ideas. In this paper we describe our research towards a design environment, within which different aspects of designing can be combined, elab- orated and controlled. New hardware equipment is combined with recent developments in graphics and artificial intelligence programming to develop appropriate computer based tools and allow new design techniques. The core of this design environment comprises spatial ob.jects that can exhibit different behaviours drawn from Artificial Intelligence (AI) principles.

The main motivation of our work is to look for solutions to problems like: “How would designers like to work?“, “How would a computer-supported design environment look like?“, and “How can the design process and result be enhanced thanks to computers?“. This leads to the vision of a new kind of design environment that should cover many aspects of designing, and for which the different aspects of designing are re-evaluated and tools refor- mulated. Future design environments should take advantage of the new technologies that computers offer, and, if successful, they will be followed by new theories for their application and discussions on their implications on design.

* Corresponding author. E-mail: [engeli, kurmann]@arch.ethz.

ch. WWWz http://caad.arch.ethz.ch/

’ Discussion is open until March 1997 (please submit your

discussion paper to the Editor of Architecture, Y.E. Kalay).

First and very promising steps have been realised with the development of SCULPTOR [l], an experi- mental computer tool that allows direct specification and manipulation of objects and scenes in 3D. It is not any one single feature in SCULFTOR but a judi- cious combination of many separate features that makes it possible to generate and explore 3D models in a very fluid and visually engaging way. This

0926-5805/96/$15.00 Copyright 0 1996 Elsevier Science B.V. All rights reserved.

PII SO926-5805(96)00150-I

142 M. Engeli, D. Kurmann/Automation in Construction 5 (1996) 141-150

includes embedding SCULPTOR into a Virtual Reality (VR) environment, which augments the immediacy of the design project and process to the designer [2].

Intelligent software agents were developed, which

assist the designer when interacting with and navi- gating in the designed building. They should help to

create an optimised environment, where the com-

plexity of the design task can be reduced thanks to the intelligent support from the machine.

2. Enhancing the design process and environment

“To improve the design process and therefore the quality of the result” is the primary goal we want to

reach with the new design studio. This involves further developments on different levels: interface, interaction, data structure, functionality of the tools

and even experiments with new approaches to the

design process.

There are different phases in the design process

and different sorts of interaction are needed during the various stages of a design. The conceptual, abstract phase at the beginning of a design process is the phase, where conceptual decisions are made and major constraints are established. In this phase most of the architects use paper and pencil to sketch. Later

on, the sketches are transferred into the computer, by using a CAD tool. These tools are very powerful in

managing exact numbers and measurements, but not

for outlining vague ideas.

We envision computer aided design to happen in Virtual Reality, in a set-up where one or more

designers can participate, while they are physically in the same or different locations.

Virtual Reality means that there will be new

possibilities to experience the architectural design. In addition to plan views and model views in different scales, the architect gets the possibility to virtually enter the design (see Fig. 11, to experience the spaces that get created. This experience can be enhanced by simulating different lighting situations. The addition of sound will help to get more of the designers perceptive senses involved. This can lead to a higher appreciation of the task and easier concentration.

A Virtual Reality set-up allows the use of spatial input devices, like 3D joysticks or data gloves. The interaction can happen directly in 3D so that it feels

Fig. 1. A designer (wearing 3D-glasses) is going inside the virtual

project (projected onto a large screen with a video projector).

almost like modelling with physical building blocks or clay (specially if the left and the right hand hold

an input device), which enables the user to formulate design ideas in the 3-dimensional space. Such a set-up allows the motor skills of the whole body to be involved in the process of designing. As Laseau points out; the combination of motor actions, with the visual perception, and the activities of the brain

stimulate the process of creative graphical thinking

[31. Virtual Reality environments can show things that

are not real. Non realistic features can be used to display additional information. Walls can become

transparent to display the plumbing. Parts can start blinking to draw the attention to an unresolved or conflicting situation. Non existing elements can show up to illustrate other activities in the design studio.

Surfaces can display texts or movies to give additional information. Further there is the possibility to change the scale and linearity of time for simula- tions, which allows to study a project’s qualities over time and consider different possible influences during its whole lifetime.

A major factor for modelling in virtual reality is interactivity. The possibility to change an object or a scene in real time adds a new dimensionality to the design. Being able to recompose a design at the same speed as we rearrange a composition of wooden

M. Engeli, D. Kurmann/Automation in Construction 5 (1996) 141-150 143

blocks on a table supports the design thinking pro-

cess and it allows the designer to focus on spatial arrangements. Directness is a key factor to allow a certain freedom while designing, which is desired especially in the early, conceptual design phase.

3. Modelling the space

3.1. The significance of space in architecture

One of the important factors in architecture - if

not the most important one - is space. Is it not this

nothing, this void, what interests us the most? The volume of a living room, or a bed room? Equally

important as designing the form and facades of a building, is the design of the spaces within. A design task is usually described by the size and number of

different spaces needed. The importance of space in architecture is described more into depth in [4,5]. Solid modelers and surface modelers have been inte-

grated in CAD tools, But which ones allow to design space? To support me creation of spaces a different

data structure, a new description of objects needs to

be implemented for CAD. The possibility to model

with spaces, will be influential on the approach an

architect can take to design a building.

3.2. Implementing negative volumes

The idea is to use volumes in a different way than

solid modelers do. Instead of using volumes and logic operations like subtraction, union, or differ-

ence, two types of volumes, positive (solid) and negative (space) volumes are used (see Fig. 2). The

rule is that a negative volume always carves out

space. Where it intersects with a positive volume this

becomes visible, but there is no effect where it intersects with another negative volume.

In our implementation the calculation of objects that are composed of positive and negative volumes is based on four steps:

finding out which faces get intersected, cutting each face according to the intersection

lines, deciding, whether the trimmed face is shown or

not, and finding out, in which direction the face is ori-

ented.

Fig. 2. Positive and negative volumes: A sequence of interactively moving two voids into one solid. The colour of the cut surface is defined

by the colour attributes elf the voids which are drawn by a wire frame. One solved special case: entirely separated face of one positive

volume. The face normal:; of the resulting object are indicated on one object.

Fig. 3. Different architectural models built by using voids and solids in SCUL~OR: a cave, a rough reconstruction of the ‘Casa Cavagli’ by

Luigi Snozzi and a light simulation rendered with RADIANCE.

Technically, the principle used is a surface modelling There are some principles that make this concept

technique, used by other authors in a similar way, for easier to realise. A possible face of any composed

example in [6,7]. object will always be a part of an original face of the

Fig. 4. A sequence of how to compose a room (= group of a void and a solid). The rule also allows hierarchical ‘room in room’ buildings.

composing objects. Therefore it does not matter what shape an object has *or how it is oriented in space, as long it is described by a closed surface. The intersection of an object with another object’s face will give the points and lines that define the shape of the

resulting face. This trimming process is the most time consuming pari of the calculation. The normals

of the resulting faces are defined easily: a solid

volume will always create faces looking outside, faces looking inside are produced by a void. The

fact, that the void Idefines the corresponding faces

makes it possible, that the attribute of a void defines

colour and texture of the faces it cuts out. Two representations, the way an object is com-

posed and the final composition, are stored. This dual representation is needed for two reasons: The

final composition is kept because the face calculations should be done as few times as possible. To

gain performance, a re-calculation is done only when

one of the objects changes. The description of how an object is composed is necessary to do this re-

calculation.

This representation of objects is implemented for

an orthogonal world with cubes as the only possible voids (see Fig. 3). The algorithm is generally appli- cable for every closed object freely oriented in space, but is much faster if it can be reduced to orthogonal cases. The orthogonal world makes interesting and complex objects possible and allows to work with spaces in an intuitive and direct way. With the

currently available computing speed, direct manipu-

lation would not be possible in the general case if there are a larger number of volumes in the scene.

3.3. Rooms, windows, doors, and sections

With this concept of negative volumes, the relevant objects and operations for an architect in an

abstract phase are defined. Windows and doors are negative volumes that connect other voids inside a positive one. A room can be seen as a composition

of a void within a solid. This group then can be intersected with other room groups interactively, a correct composition is always assured (see also Fig.

4). While resizing a room group (a pair of a solid

and a void) the thickness of a wall stays constant.

Fig. 5. Various kinds of scenes produced with S CULPTOR, including two which show the path of dynamic object (left side) and one of urban

design modelled on an image of the site.

146 M. En&i, D. Kurmann/Automntion in Construction 5 (19961 141-150

Using the negative volume concept, sections through buildings are possible by using a large void and intersecting it with the building.

Besides the ease of manipulation, the introduction of rooms offers new possibilities in the analysis of

buildings. By defining rooms that compose a build-

ing (instead of the walls that create the rooms), it is

easy to retrieve data for additional support such as:

navigation in a building, spatial sound effects, cost estimations, or energy consumption calculations. This

will be emphasised later in this article in the description of some intelligent agents.

4. The design tool: SCULPTOR

In order to test and demonstrate these ideas, SCULFTOR was created. It is a computer tool for

virtual design in architecture that has been developed

by David Kurmann over the last four years. It com- bines the novel features and functions described

above with general concepts of modelling in 3D

(Fig. 5). A key factor in the program is the human machine

interaction. SCULITOR allows very direct, intuitive and immersive access to three dimensional design models. Through interactive modelling in a virtual space and the introduction of positive and negative

volumes, an easy way of generating and manipulat- ing architectural models is made possible. Interactive

parameter specification of objects, and models with

attributes like form, geometry, colour, or material are

supported. Objects can be grouped together hierar-

chically. Objects, groups and virtual worlds can be changed in real time by scaling, resizing, rotating, reshaping and moving them in space. Different points of view can be chosen as well as functions invoked for walking and flying through 3D space. All manip- ulations happen immediately by moving the mouse or one of the possible 3D input devices and the scenes are changed and rendered in real time. Multi- ple windows, complex text input and sliders or but- tons are avoided. The interface is almost widget-less.

Besides assisting the users when creating the three-dimensional geometry of objects, SCIJLPTOR

also supports models of behaviour based on principles of mechanics and dynamics. One is collision detection while objects are changed in size or moved

in the scene. Another is the concept of gravity: an object will fall down if it is not supported by another

object or standing on the ground. Using these constraints, the interaction with objects in the virtual

worlds is enhanced since users experience in a direct

way the act of moving objects to valid positions or

combining them by following physical principles.

Different sorts of feedback are an important feature

of virtual reality tools to understand complex scenes in space.

SCULPTOR can encode different knowledge into objects, knowledge about the object itself, about its behaviour, and about the environment. This allows

the definition of intelligent objects with a certain quality and behaviour [3]. Models may represent

physical objects such as building elements or fumi- ture, as well as objects with purely functional and

behavioural characteristics. Such possibilities are

useful for participatory design in which different

building partners access the design and make decisions according to their field of expertise.

5. Intelligent agents that support the designer

Our vision of a design studio is an optimised human-machine collaborative environment, where it is a joy for the human to work in. Research about

design automation is advancing, but we believe that in the near future the human experience, common

sense and emotions, will be needed to give a design

its final touch, enhance its unique qualities and its

expressive power.

Our research focuses on supporting the designer in coping with the increasing complexity of design tasks and increasing challenges through new prod- ucts and new requirements. Intelligent software agents can be introduced to assist the designer in

aspects like: keeping the overview of the project, taking over tedious tasks, or looking up external information sources.

Besides their supportive function, agents may in- fluence the quality of human-computer collaboration. AI engenders the great danger, that its results can look more intelligent if the user does not understand well, how they get produced. Agents can be introduced to counteract this mystification, because they can have names, full2 specific tasks, take com-


mands, and learn how to perform best on the user’s

behalf. This helps the user in two ways. First, the AI tasks are distributed among known entities. Second, the user has the possibility to give positive or nega-

tive feedback and customise the agents.

Because the term ‘agent’ is used in a non-stan- dardised way, we defined that an agent should have

the following five characteristics. It contains knowl-

edge, it is designed to work on a specific task, it can work autonomously, it acts on behalf of the user, and

it has the ability to learn. Nicholas Negroponte had already formulated the

idea of employing agents in the interface in 1970 in his book The Architecture Machine: Towards a more

Human Environment [8]. It took some years for AI, behaviour based AI and agents research to mature, so

that proven principles (speech recognition, planning

algorithms, neural networks and learning through

reinforcement) can .be used for the implementation [9]. Our contribution to the field lies in the new way of using known principles and combining them for

appropriate behaviour. A particular aspect of our

research efforts, is to find ways to represent the

agents to the user, make their activities visible and let the user interact with them. In this respect, we

hope to make innovative contributions, specially with the new interface possibilities provided by a virtual

reality set-up. We can imagine two basic kinds of agents: Design assisting agents, that help the user by

providing information and executing background

tasks, and design generating agents, that interfere with the design and autonomously come up with new

solutions. For now we are working on design assisting agents, because we are focusing on improving the human-computer interaction.

Three prototypical agents have been implemented so far and will be described more in detail now: the

Navigator, the Sound Agent and the Cost Agent.

They are meant to be personal assistants, trained by each user to adapt to one’s individual preferences.

These three agents were implemented first because

they were the most feasible ones and the most effective ones to test our idea of agents in a virtual

environment. To gather experience with several ap-

Fig. 6. The graph built by the Navigator.

148 M. Engeli, D. Kumnn/Automution in Constmction 5 (1996) 141-150

proaches and their advantages or disadvantages, each of these agents is designed in a different way, using different AI approaches and different kinds of human computer interaction.

5.1. The Navigator

The Navigator acts like a guide in the virtual world. It can follow different kinds of instructions like: moving to a specified place, moving in a specific direction, or composing a tour. The Navigator gets commands through a voice interface. This kind of interface is very suitable to be used in a VR environment and very natural to work with. For our implementation, we started with a simple speech recognition software that recognises single words. The commands are composed of a pair of keys. The first key describes the action: go, show, go to, jump to, that describe the action to be taken. The second key either names what to show (this floor, the building), a place to go to (the kitchen, the entrance, the living room>, or tells the direction to be taken (left, right, up, down, forward, backward). The Navigator is built into SCULFTOR and takes advantage of the data structure which describes the openings and the different kinds of rooms as negative volumes with attributes. From this information the Navigator builds a graph. According to the command by the user the

Navigator can then search the right path in this graph (Fig. 6).

5.2. The Sound agent

The Sound agent is a companion of the Navigator. It will try to enhance the perception of a space by adding an auditory component to the visual impres- sion. There are four different things the Sound agent can do:

It can play ‘sound labels’, that means that when it enters a room, a voice announces the purpose of this room. It can generate the sound of footsteps, which are synchronised with the movement by the Naviga- tor. It can change the spatial sound effects, which can modify the synthetic footsteps or the user’s voice when using a microphone. It can play a sound track, in accordance to the purpose, shape and colour situation of different rooms. The Sound agent is a program that communicates

with SCULPTOR using the NCSA Data Transfer Mechanism (DTM) [lo]. Sound labels and footsteps can easily be generated from the information available from SCUL~OR. It describes the room the Navi- gator is in (the purpose, size, and colour situation)

Fig. 7. The actions triggered by entering a room: 1 - get the room attributes, 2 - figure out which node can handle this type of room, 3

- use the examples in memory to define the weights of the attributes of the sound, 4 - find the best matching sound in the library, 5 - calculate the level of confidence, 6 - play the sound and inform the user about the level of confidence.


and the current position within this room. Which spatial sound effect to use and which sound track to

play has to be learned first. The Sound agent does this through memory based learning. For each kind of room a number of examples get stored. If there

are enough examples new ones will override old ones, which allows lthe agent to adapt to changing preferences of the uslcr.

The available sounds are described using a number of weighted attributes [ 111, the agent defines the sound it is looking for by the weights of these

attributes. The sound that gets selected is the one that

is the closest to the description. The Sound agent can express its level of confi-

dence, a feature for agents introduced by Maes and

Kozierok in 1993 [ 1211. The level of confidence gets calculated by considering the number of examples in memory, how well the new case matches one of the examples and how close the available sound is to the description. If the level of confidence is not very high the sound gets played with a stuttering onset, if

it is underneath the ‘tell-me’ threshold [12], the

sound agent asks for confirmation or help. The Sound agent can be criticised and instructed

at any time and it will use the information to im-

prove or adapt to changed preferences or a new user. Fig. 7 illustrates the process triggered at the mo-

ment of entering a room and resulting in the playing

of the sound track.

5.3. The Cost agent

The Cost agent estimates the costs of the project and displays the result graphically (Fig. 8). The vertical bar represents the costs and turns from green

to red when the costs are getting too high. This small

interface can be used to define a cost limit, to indicate the desired level of luxury, to access a

database of projects and to look at a graph that

illustrates the calculation. Like the Sound agent the Cost agent is a separate

program that communicates with SCULPTOR using the NCSA DTM. Kind and size of each room are considered for the calculatioas (Fig. 9). Using these formulas windows and doors can be treated like rooms. This corresponds to the way the data is represented in %xJLPTOR.

The way the agent learns is implemented similar

Fig. 8. The interface of the Cost agent.

to the delta rule learning in neural networks. Instead of modulating weights in a network, the ‘init price’ and the different factors for each type of room get

changed. The agent will learn from every project that was designed with SCULPTOR and then realised. For the initial learning phase a database will be provided consisting of realised buildings that were recon-

structed with SCULPTOR.

5.4. Future plans

The next set of agents will be agents that connect to information sources that are outside of the actual design studio. Controlling agents will use outside information sources to check the validity of the current project, like figuring out if spaces are dimen-

Cost Calculation

For each room: (init price and factors are specific for each type of room)

I (init price * deluxefactorl) + (volume * roomfactor * deluxefactor )

Total: [j&it of ai Ey;;,, for th,s year / (buildlng init price * deluxefactorS))

Fig. 9. Formulas used by the Cost agent to estimate the cost of a

building.

150 M. Engeli, D. Kurmann/Automation in Construction 5 (1996) 141-150

sioned reasonably or checking if building laws have

been considered appropriately. Information agents can scan through different sources and, whenever they consider it to be appropriate or when they get asked, pass relevant information to the user. Exam-

ples are: - An information agent, that scans for information

24 hours a day.

- An HVAC agent, that calculates and controls

isolation as well as energy exchange. - An illumination agent, that can simulate different

lighting situations. Agents will get improved so that they are able to

help with designing. For example:

- The cost agent can make proposals on how to lower costs.

* The HVAC agent can help to optimise isolation and energy exchange, and help to layout installa- tions.

* The illumination agent can help to plan for an

optimised light situation in the rooms considering daylight and artificial illumination.

Later design agents, that generate and suggest design solutions, will be added. They will be based

on a more knowledge based and holistic approach than can be built into SCULPTOR’S intelligent objects.

6. Conclusions

Previous research and development in CAD has

often reflected the manual design reasoning and design development sequence. As described in this

paper we believe that there is a need to explore other possibilities to support design, especially those that derive directly from using space modelling, artificial

intelligence and virtual reality. Since the project implementation has led us to

challenge and question our own beliefs about architectural design, it is yet early to evaluate all the possible contributions of this project. It will take some time until we can comment on the real impact of this new environment on design. First experiments by students and demonstrations to other designers have generated encouraging feedback.

Acknowledgements

This paper shows some results from two projects

sponsored by the Swiss National Science Founda- tion, Priority Program Informatik SPP: M&i-Agent

Interaction in a Complex Virtual Design Enuiron-

ment [13] and Working Group in Model-Based De-

sign and Reasoning. We would like to thank Dr. Bharat Dave for his input on this paper and for

backing us up in various ways during our research.

References

HI

[21

131

141

[51

[61

[71

Kurmann, D., .!bJLFTOR - A Tool for Intuitive Archtectual

Design, in: CAAD Futures ‘95, Vol. 1, no. 5/24, Univ. of

Singapore, 1995.

Schmitt, G., Wenz, F., Kurmann D. and van der Mark, E.,

Towards Virtual Reality in Architecture: Concepts and Sce-

narios from the Architectural Space Lab, Presence, Vol. 4,

No. 3 (1995) 267-285.

Laseau, P., Graphic Thinking for Architects and Designers, Van Nostrand Reinhold, New York, NY (1989).

Tschumi, B., Questions of Space - Lectures on Architec- ture, Architectural Association, London (1990).

van de Ven, C., Space in Architecture: The Evolution of a New Idea in the Theory and History of the Modern Moue- ments Van Gorcum, Assen/Maastricht, The Netherlands

(1987).

Barzel, R., Physical-Based Model& for Computer Graph- ics, Academic Press, New York, NY (1992).

C.I. Yessios, The Computability of Void Architectural Mod-

eling, in: Y.E. Kalay (Ed.), Computability of Design - Principles of Computer-Aided Design, Wiley-Interscience,

New York, NY (1987) 141-172.

[8] Negroponte, N., The Architectural Machine: Towards a more Human Enuironment, MIT Press, Cambridge, MA (1970).

[9] Maes, P., Behaviour-Based Artificial Intelligence, in: Pro- ceedings of the 2nd Conference on Adaptive Behauiour, MIT

Press, Cambridge, MA (1993).

[lo] NCSA, Data Transfer Mechanism Programming Manual, ftp://ftp.ncsa.uiuc.edu/DTM/dtm.manual.ps, National

Center for Supercomputing Applications (1992).

[l 11 Engeli, M., Sound Library, VLWpapers, MIT Media Labora-

tory (1993) unpublished.

[12] Kozierok, R. and Maes, P., Learning interface agents, in:

Proceedings of the 11th National Conference on Artificial Intelligence, AAAI (1993).

[13] Schmitt, G., Engeli, M., Kurmann, D., Faltings B. and Frei,

C., Multi-agent interaction in a complex virtual design environment, in: Priority Programme Informatics Research Pro- ceedings: Information Conference, 15-16 December, 1994,

Swiss National Science Foundation.

spatial objects and intelligent agents in a virtual environment

Documents