international journal of ar chitectural computing … · international journal of ar chitectural...

17
Let There be Light! Knowledge-Based 3-D Sketching Design Tools Ellen Yi-Luen Do and Mark D. Gross 211 issue 02, volume 02 international journal of architectural computing

Upload: dangthien

Post on 09-Sep-2018

228 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

Let There be Light Knowledge-Based 3-DSketching Design ToolsEllen Yi-Luen Do and Mark D Gross

211issue 02 volume 02international journal of architectural computing

IJAC 2-2_3 proof 16-7-04 346 pm Page 211

Let There be Light Knowledge-Based 3-D Sketching Design ToolsEllen Yi-Luen Do and Mark D Gross

This paper presents a framework for 3D knowledge-based sketching tools for lighting design and twosoftware prototypes built to illustrate sketch-basedinteraction with intelligent systems in 3-D domainsSpot supports direct sunlight simulation andvisualization in a selected time period and Light Pensupports placement of electric lighting designs to lightan intended area in space In both examples a 3-Dsketching front-end is coupled with a back-endknowledge-based systemThis enables a designer topose a problem by drawing onto a 3-D model towhich the knowledge-based system offers a solution ndashin one case by providing quantitative data analysis inthe other by modifying the 3-D model Spot and LightPenrsquos specific domain of architectural lighting designexemplifies a more general class of 3-D interactionwith intelligent systems

212

IJAC 2-2_3 proof 16-7-04 346 pm Page 212

1 Introduction

11 Motivation ndash tools for lighting design

Lighting design is an important part of architectural designArchitectsconsider light as the ldquofundamental basis of architecturerdquo (Le Corbusier)ldquoweonly know the world as it is evoked by lightrdquo (Louis Kahn) light is part ofthe ldquostructure of the thinking of the architecturerdquo (Richard Meier) and thewall is a ldquoluminous vertical surfacerdquo (Carlo Scarpa) Light is a design variablethat affects the character of life and activities in the spaces people inhabitThe orientation form and scale of a building the arrangement of openingsand glazing as well as spatial configurations of rooms are strongly influencedby architectsrsquo approaches to light

Lighting design requires the consideration of human needs such ascomfort and aesthetics and the solutions for energy efficiency and costeffectivenessThe balancing act of the quantitative and qualitative concernsof lighting design remains a difficult challengeArchitects have traditionallydealt with lighting design in buildings qualitatively and intuitively Recentadvancements in computer graphics provide compelling rendering tools forphoto-realistic visualization to support qualitative assessment Research andsoftware applications for numerical calculation of diurnal and annual cyclesfor sunlight now provide methods for quantitative analysis and assessmentBoth are welcome developments However the question is not whetherquantitative or qualitative methods provide better support for lightingdesign Designers need to approach lighting both qualitatively andquantitativelyA good design tool should provide relevant information beintegrated into the design environment to better support quick iterativeexplorations essential to a design decision-making processTo address thisneed we propose a framework of a computer-aided 3D sketchingenvironment with knowledge-based lighting design systems

This paper presents two software prototypes developed under theframework that link intuitive user interfaces to algorithmic calculations thatoperate on a 3D modelTo address the essential pair ndash the quantitative andthe qualitative ndash we have chosen to implement software modules toseparately investigate daylight simulation and lighting fixture design usingsketching in three-dimensional space to manage the two modules Spotrenders daylight analysis and visualization over time on any surface in spaceLight Pen recommends lighting fixture placements from intended illuminatedsurfaces Below we briefly provide our rationale for building thesecomputational tools for lighting visualization analysis and design

12 Daylight analysis

Visualization and analysis of sunlight effects in buildings is essential fordaylight design and performance Sunlight has been considered an importantfeature for any successful space design as well as an energy source for

213Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 213

buildings [1] Louis Kahn stressed the importance of sunlight and arguedthat ldquoA room is not a room without natural lightrdquo Daylight design involvesconsiderations of the variation of lighting conditions and the degree ofpenetration into buildings Researchers have developed many methods tohelp quantitatively assess projected lighting levels Hand calculation methodssuch as Lumen methods daylight protractors and work sheets are usefultools for calculating physical parameters such as the size and shape of askylight Physical model and tilt tables also help analyze full load lightinghours and density However applying any of the methods above is laborintensive and involves several steps

Recently we have seen ray tracing and other computer graphicstechniques being employed (eg Radiance) to simulate the behavior of lightin the real environmentThese tools can simulate lighting effects of a spaceat a given moment in a viewpoint However they do not provide real timefeedback of lighting effects over timeA high-resolution image of outputfrom a simulation requires complex computation to decide the luminancevalue and usually takes time to generate During the early conceptual designstage designers need a simple-to-use simulation and visualization tool tohelp them understand and recognize lighting problems and opportunitiesRapid direct sunlight simulation in the 3D design environment would beuseful in early stage designTherefore we implemented the Spot system toenable architects to sketch in 3D to specify areas for quick previews andcalculations of the amount of direct sunlight projected over time

13 Lighting fixture design

A lighting designerrsquos main task is to select lighting fixtures and position themin the building appropriately for the activities that are to take placeThelocation size and orientation of windows and skylights must be consideredbut typically the architect has already made these building design decisionsTypically a lighting design includes three categories of illumination ambienttask and accent lightingAmbient illumination supports general activities inspace such as wayfinding orientation and movementTask illuminationsupports specific activities such as reading work eating and conversationAccent lighting highlights points of interest such as paintings photographsor architectural details Several alternatives exist for implementing each typeof lighting for example recessed ceiling lights fluorescent lamps indirectlyreflecting pendant lamps floor and table lamps and track lights Each hasspecific preconditions for installation advantages and drawbacksThe lightingdesigner must also specify the configuration and location of switches [2]

14 Structure of the paper

In the following sections we briefly introduce Space Pen a platform that webuilt earlier for browsing and sketch annotation in 3-D and explain how itled to Spot and Light Pen Next we demonstrate Spot and Light Pen and

214 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 214

outline the lighting design expertise that these programs embodyWe closewith a brief discussion about how 3D sketching interface enables easierinteraction in the specific domain of architectural lighting and directions forfuture work

2 Sketching as an interface to lighting design tools

21 Intelligent sketching systems

Generating 3-D geometry from 2-D sketches has been a lsquoholy grailrsquo researchtopic for some time yielding diverse approaches to pen-based interfaces forgenerating and editing three-dimensional models Sketch [3] like itscommercial cousin Sketchup recognizes gestures and generates modelingcommands rather than parsing and interpreting a line drawing whereasTeddy [4] uses a simple heuristic to inflate three dimensional curvilinearforms from a freehand drawingVR Sketchpad [5] combines extrusion andsymbol recognition to rapidly create VRML worlds Chateau [6] tries toanticipate the userrsquos intentions in generating a three-dimensional modeloffering alternative 3-D completions as the user draws a 2-D sketch DigitalClay [7] uses constraint propagation of concave and convex vertices togenerate three-dimensional models from a two-dimensional diagram Stilton[8] enables designers to draw into a three-dimensional scene represented inVRML the program parses the line drawing on the fly to add 3-D geometryto the scene

Beyond sketch-to-3-D geometry creation a quite different line ofsketching research explores using diagrams to interact with applications suchas simulation programs databases and other intelligent systemsThisapproach is especially appropriate in domains where drawings are commonrepresentations Diagrammatic interaction has been explored in physicaldomains such as mechanical engineering [9] and geography [10] as well asto support tasks such as interaction design [11] and military action planning[12] In this vein our earlier work developed a general system fordiagrammatic interaction based on an end-user programmable visuallanguage [13]

Diagram interfaces for intelligent systems have mostly supported two-dimensional drawing However in physical design domains such asmechanical engineering entertainment and architecture the artifacts thatdesigners manipulate are typically 3-D computer graphics models Designersoperate on these models with applications that process 3-D data anddisplay results in 3-D For example a mechanical engineer may performkinematics analyses of a mechanism in 3-D or finite element analysis of astructure a game designer may perform visual analysis of sight-lines and anarchitect may test a proposed building for emergency egress routesTheseanalysis tasks typically involve running a 3-D model through a stand-aloneapplication

We would like to bring applications that operate on 3-D models into

215Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 215

the designerrsquos working environment to seamlessly integrate design andanalysis Designers often begin work with informal sketches and diagramsThat is why we want to employ pen-based interfaces to interact with theintelligent systems that serve as computational assistants in three-dimensional design domainsWe built Spot and Light Pen to demonstratehow such an interface can be used in the domain of architectural lightingdesign

22 Space Pen

Our platform for 3-D sketching is Space Pen [14] software we built tosupport Web based design collaboration with annotation capabilities in 3-DThe Space Pen server converts any VRML model posted by the architectsinto a Java 3D model in a standard Web browser Collaborating teammembers can then browse and annotate by drawing on model surfaces Forexample a team member reviewing a proposed architectural design drawson a wall-graffiti style-to indicate a proposed location for a new windowSpace Pen also supports text annotation with threaded discussions linkedto Post-It(r) style tags left in the model Designers mark on existing modelsurfaces or on a temporary drawing plane to add geometry to the modelSpace Pen identifies figures such as arrows rectangles and circles which itcan then rectify as model geometry or interpret as commands In short theSpace Pen provides a platform for drawing onto and into 3-D models

23 Sketching with light

Our work on Light Pen began after we demonstrated Space Pen [15] to aprofessional lighting designer [16]We wondered how to improve SpacePenrsquos rendering capabilities to support lighting design tasks She commentedthat commercial lighting design software applications render excellently thevisual appearance of a proposed lighting design configuration Givenpositions of a set of lighting fixtures windows and architectural geometrythe software renders the resulting lighting effects at a given date and timeWhat lighting designers really do she explained is ldquopaint with lightrdquo-theyidentify desired lighting effects at specific locations then reason backward todetermine the selection and location of lighting fixtures that produce theseeffectsWe realized that the Space Pen could be an effective interface forlighting designers to specify design intentions

Inverse calculation of lighting effects working backward from lightingeffect to position and characteristics of lighting sources is not newTheappositely named Painting with Light system [17] computes color andintensity values for fixed theater lighting based on a lighting effect that thescene designer paints on a model of the stage surface Inverse calculation isalso used in other design domains For example mechanical engineers useinverse kinematics to determine the geometry of a mechanism needed toproduce a given set of motions

216 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 216

This idea-sketching on a 3-D model to identify desired lighting effects-sparked the development of the Light Pen More generally we saw thatsketching in 3-D could be a direct and natural means to interact withsystems that reason about and calculate on three-dimensional modelsWeproposed to use light-painting as input to an automated design assistant thatwould help an architect select and position lamps to produce desiredlighting effects

24 Sketching for daylight visualization

Our work on Spot began with the interest of providing sunlight visualizationto 3D space and to complement Light Penrsquos lighting fixture design featuresSpot follows the same framework using Space Penrsquos Java 3D sketchingnavigation and annotation platform as an interface to knowledge basedsystems Spot provides direct sunlight visualization in a navigable 3D space Itcomputes 3D geometry (spatial variables XY and Z) and also the sun anglevariations with the diurnal and annual cycles (temporal variables date andtime)

Multi-dimensional data is usually displayed on a single 2D visualizationpaneThe Space Series project [18] uses a focus-plus-context technique tosupport display of spatial and temporal data variations For lighting expertswith specialized knowledge this 2D display is sufficient However easyvisualization and interaction techniques would better support architectsrsquoqualitative assessment when designing in 3D Software like Ecotect [19]displays the pattern of multiple shadows projected during a period of timeon a single diagram but it does not quantify the amount of received light

To initiate lighting visualization in Spot designers first sketch a boundaryshape on the 3D model indicating the area for simulation Spot thengenerates a representation of the spatial distribution of the illuminationlevel on the selected surfaces over time Spot also enables designers tovisualize the light distribution over time for a given point For each pointtapped by pen (or clicked by mouse) on the 3D model Spot generates acalendar chart where the X and Y axis represent the months of the yearand the time of the dayThe color of each calendar cell corresponds to thecalculated amounts of light reaching the point the user tapped

3 Light Pen

The Light Pen system consists of a 3D sketching front end to a rule-basedelectrical lighting fixture advisor Below we briefly describe the systemarchitecture and the components of Light Pen and a use scenario todemonstrate how it supports lighting design More technical detail onimplementation can be found in (Jung Gross and Do 2003) Here weprovide a conceptual explanation of the work and the current extensions ofthe rules

217Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 217

31 System architecture

Figure 1 illustrates Light Penrsquos system architecture It consists of twocommunicating components the Space Pen 3-D drawing program (left) andthe Lux lighting advisor (right)

The designer interacts directly with Space Pen which provides tools for3-D browsing and sketchingAfter importing a three-dimensional (VRML)model the designer marks it up to indicate desired lighting effectsThemodel geometry and the designerrsquos lighting sketch marks are passed to theLux lighting design advisor

Lux is Light Penrsquos lsquoback endrsquo intelligent system coded in Java as a simpleset of lighting design decision rules Lux accepts the lighting sketch marksand building geometry as inputs Based on the desired lighting and themodel geometry Lux recommends solutions selecting fixtures based ontheir desired characteristics Finally it passes these recommendations backto Space Pen which adds the fixtures to the 3-D model to indicate Luxrsquosproposed design solution

32The Lux lighting design advisor

Lux is the lsquoback endrsquo intelligent system component of Light Pen Lux firstanalyzes the model geometry on and near the designerrsquos lighting sketch-mark It determines for example whether the area to be lit is a floor wallor work-surface and whether the area is large or small Next based on thisanalysis Lux determines whether the lighting category is task accent orambientThen the system selects a set of appropriate lighting fixtures basedon the lighting category and the architectural geometry For example it willsuggest track-lighting only if it finds an appropriate surface nearby formounting the lighting track Finally for each fixture that it deems appropriatefor the lighting category Lux identifies an appropriate locationThese stepsare described in the following sections

m Figure 1 Light Penrsquos two

components a 3-D sketch browser

(Space Pen) and a knowledge-based

advisor (Lux)

218 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 218

Analyze the model geometry

The architectural model is imported from a CAD program in VRML formatso it consists merely of a set of surfaces with no information about whatbuilding components or furniture the surfaces may represent Rather thantrying to classify all surfaces in the model a posteriori or requiring thedesigner to tag them a priori we chose to perform a local analysis on theparts of the model where the designer has sketched lighting marksThe Luxrules use simple predicates to reason about the local architectural geometryof the illumination problemThese predicates include tests for coplanarityvertical and horizontal surfaces as well as more specific tests to determinewhether a surface is a floor a work-height surface or near the ceiling

The designerrsquos marks on the 3-D model are passed as a query to theLux lighting design advisor which takes three actions

bull First it identifies the type of illumination needed based on the size ofthe light mark and the surface it is drawn on

bull Second it selects a lighting fixture suitable for the illumination typeeg track lights for task lighting a pendant lamp for ambientillumination

bull Third it proposes appropriate positions for mounting the lightingfixture

This information is conveyed back to the Space Pen which adds theproposed design elements in their positions to the model and displays thenew scene indicating with a cone of light the illumination effects theyprovide

Identify illumination task

Based on the analysis of architectural geometry and the size and location ofthe lighting sketch mark Lux then determines the desired category ofillumination It selects ldquotask lightingrdquo if the surfaces to be illuminated arehorizontal and the surface is close to the ceiling Otherwise if the area tobe lit is small it selects ldquoaccent lightingrdquo If the area to be lit is large or thesurfaces to be lit are both horizontal and vertical then Lux selectsldquoambientrdquo as the illumination category

Select Appropriate Fixtur

Next using a decision tree Lux selects a fixture or a set of fixturesappropriate to the illumination category that it determined in the previousstep If the illumination category is ldquotask lightingrdquo then the set of possiblefixtures includes several possibilities [spotlight desk lamp table lampfluorescent light long pendant light] If the illumination category is ldquoaccentrdquothen the only fixture it suggests is ldquospotlightrdquo If the illumination category is

219Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 219

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 2: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

Let There be Light Knowledge-Based 3-D Sketching Design ToolsEllen Yi-Luen Do and Mark D Gross

This paper presents a framework for 3D knowledge-based sketching tools for lighting design and twosoftware prototypes built to illustrate sketch-basedinteraction with intelligent systems in 3-D domainsSpot supports direct sunlight simulation andvisualization in a selected time period and Light Pensupports placement of electric lighting designs to lightan intended area in space In both examples a 3-Dsketching front-end is coupled with a back-endknowledge-based systemThis enables a designer topose a problem by drawing onto a 3-D model towhich the knowledge-based system offers a solution ndashin one case by providing quantitative data analysis inthe other by modifying the 3-D model Spot and LightPenrsquos specific domain of architectural lighting designexemplifies a more general class of 3-D interactionwith intelligent systems

212

IJAC 2-2_3 proof 16-7-04 346 pm Page 212

1 Introduction

11 Motivation ndash tools for lighting design

Lighting design is an important part of architectural designArchitectsconsider light as the ldquofundamental basis of architecturerdquo (Le Corbusier)ldquoweonly know the world as it is evoked by lightrdquo (Louis Kahn) light is part ofthe ldquostructure of the thinking of the architecturerdquo (Richard Meier) and thewall is a ldquoluminous vertical surfacerdquo (Carlo Scarpa) Light is a design variablethat affects the character of life and activities in the spaces people inhabitThe orientation form and scale of a building the arrangement of openingsand glazing as well as spatial configurations of rooms are strongly influencedby architectsrsquo approaches to light

Lighting design requires the consideration of human needs such ascomfort and aesthetics and the solutions for energy efficiency and costeffectivenessThe balancing act of the quantitative and qualitative concernsof lighting design remains a difficult challengeArchitects have traditionallydealt with lighting design in buildings qualitatively and intuitively Recentadvancements in computer graphics provide compelling rendering tools forphoto-realistic visualization to support qualitative assessment Research andsoftware applications for numerical calculation of diurnal and annual cyclesfor sunlight now provide methods for quantitative analysis and assessmentBoth are welcome developments However the question is not whetherquantitative or qualitative methods provide better support for lightingdesign Designers need to approach lighting both qualitatively andquantitativelyA good design tool should provide relevant information beintegrated into the design environment to better support quick iterativeexplorations essential to a design decision-making processTo address thisneed we propose a framework of a computer-aided 3D sketchingenvironment with knowledge-based lighting design systems

This paper presents two software prototypes developed under theframework that link intuitive user interfaces to algorithmic calculations thatoperate on a 3D modelTo address the essential pair ndash the quantitative andthe qualitative ndash we have chosen to implement software modules toseparately investigate daylight simulation and lighting fixture design usingsketching in three-dimensional space to manage the two modules Spotrenders daylight analysis and visualization over time on any surface in spaceLight Pen recommends lighting fixture placements from intended illuminatedsurfaces Below we briefly provide our rationale for building thesecomputational tools for lighting visualization analysis and design

12 Daylight analysis

Visualization and analysis of sunlight effects in buildings is essential fordaylight design and performance Sunlight has been considered an importantfeature for any successful space design as well as an energy source for

213Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 213

buildings [1] Louis Kahn stressed the importance of sunlight and arguedthat ldquoA room is not a room without natural lightrdquo Daylight design involvesconsiderations of the variation of lighting conditions and the degree ofpenetration into buildings Researchers have developed many methods tohelp quantitatively assess projected lighting levels Hand calculation methodssuch as Lumen methods daylight protractors and work sheets are usefultools for calculating physical parameters such as the size and shape of askylight Physical model and tilt tables also help analyze full load lightinghours and density However applying any of the methods above is laborintensive and involves several steps

Recently we have seen ray tracing and other computer graphicstechniques being employed (eg Radiance) to simulate the behavior of lightin the real environmentThese tools can simulate lighting effects of a spaceat a given moment in a viewpoint However they do not provide real timefeedback of lighting effects over timeA high-resolution image of outputfrom a simulation requires complex computation to decide the luminancevalue and usually takes time to generate During the early conceptual designstage designers need a simple-to-use simulation and visualization tool tohelp them understand and recognize lighting problems and opportunitiesRapid direct sunlight simulation in the 3D design environment would beuseful in early stage designTherefore we implemented the Spot system toenable architects to sketch in 3D to specify areas for quick previews andcalculations of the amount of direct sunlight projected over time

13 Lighting fixture design

A lighting designerrsquos main task is to select lighting fixtures and position themin the building appropriately for the activities that are to take placeThelocation size and orientation of windows and skylights must be consideredbut typically the architect has already made these building design decisionsTypically a lighting design includes three categories of illumination ambienttask and accent lightingAmbient illumination supports general activities inspace such as wayfinding orientation and movementTask illuminationsupports specific activities such as reading work eating and conversationAccent lighting highlights points of interest such as paintings photographsor architectural details Several alternatives exist for implementing each typeof lighting for example recessed ceiling lights fluorescent lamps indirectlyreflecting pendant lamps floor and table lamps and track lights Each hasspecific preconditions for installation advantages and drawbacksThe lightingdesigner must also specify the configuration and location of switches [2]

14 Structure of the paper

In the following sections we briefly introduce Space Pen a platform that webuilt earlier for browsing and sketch annotation in 3-D and explain how itled to Spot and Light Pen Next we demonstrate Spot and Light Pen and

214 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 214

outline the lighting design expertise that these programs embodyWe closewith a brief discussion about how 3D sketching interface enables easierinteraction in the specific domain of architectural lighting and directions forfuture work

2 Sketching as an interface to lighting design tools

21 Intelligent sketching systems

Generating 3-D geometry from 2-D sketches has been a lsquoholy grailrsquo researchtopic for some time yielding diverse approaches to pen-based interfaces forgenerating and editing three-dimensional models Sketch [3] like itscommercial cousin Sketchup recognizes gestures and generates modelingcommands rather than parsing and interpreting a line drawing whereasTeddy [4] uses a simple heuristic to inflate three dimensional curvilinearforms from a freehand drawingVR Sketchpad [5] combines extrusion andsymbol recognition to rapidly create VRML worlds Chateau [6] tries toanticipate the userrsquos intentions in generating a three-dimensional modeloffering alternative 3-D completions as the user draws a 2-D sketch DigitalClay [7] uses constraint propagation of concave and convex vertices togenerate three-dimensional models from a two-dimensional diagram Stilton[8] enables designers to draw into a three-dimensional scene represented inVRML the program parses the line drawing on the fly to add 3-D geometryto the scene

Beyond sketch-to-3-D geometry creation a quite different line ofsketching research explores using diagrams to interact with applications suchas simulation programs databases and other intelligent systemsThisapproach is especially appropriate in domains where drawings are commonrepresentations Diagrammatic interaction has been explored in physicaldomains such as mechanical engineering [9] and geography [10] as well asto support tasks such as interaction design [11] and military action planning[12] In this vein our earlier work developed a general system fordiagrammatic interaction based on an end-user programmable visuallanguage [13]

Diagram interfaces for intelligent systems have mostly supported two-dimensional drawing However in physical design domains such asmechanical engineering entertainment and architecture the artifacts thatdesigners manipulate are typically 3-D computer graphics models Designersoperate on these models with applications that process 3-D data anddisplay results in 3-D For example a mechanical engineer may performkinematics analyses of a mechanism in 3-D or finite element analysis of astructure a game designer may perform visual analysis of sight-lines and anarchitect may test a proposed building for emergency egress routesTheseanalysis tasks typically involve running a 3-D model through a stand-aloneapplication

We would like to bring applications that operate on 3-D models into

215Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 215

the designerrsquos working environment to seamlessly integrate design andanalysis Designers often begin work with informal sketches and diagramsThat is why we want to employ pen-based interfaces to interact with theintelligent systems that serve as computational assistants in three-dimensional design domainsWe built Spot and Light Pen to demonstratehow such an interface can be used in the domain of architectural lightingdesign

22 Space Pen

Our platform for 3-D sketching is Space Pen [14] software we built tosupport Web based design collaboration with annotation capabilities in 3-DThe Space Pen server converts any VRML model posted by the architectsinto a Java 3D model in a standard Web browser Collaborating teammembers can then browse and annotate by drawing on model surfaces Forexample a team member reviewing a proposed architectural design drawson a wall-graffiti style-to indicate a proposed location for a new windowSpace Pen also supports text annotation with threaded discussions linkedto Post-It(r) style tags left in the model Designers mark on existing modelsurfaces or on a temporary drawing plane to add geometry to the modelSpace Pen identifies figures such as arrows rectangles and circles which itcan then rectify as model geometry or interpret as commands In short theSpace Pen provides a platform for drawing onto and into 3-D models

23 Sketching with light

Our work on Light Pen began after we demonstrated Space Pen [15] to aprofessional lighting designer [16]We wondered how to improve SpacePenrsquos rendering capabilities to support lighting design tasks She commentedthat commercial lighting design software applications render excellently thevisual appearance of a proposed lighting design configuration Givenpositions of a set of lighting fixtures windows and architectural geometrythe software renders the resulting lighting effects at a given date and timeWhat lighting designers really do she explained is ldquopaint with lightrdquo-theyidentify desired lighting effects at specific locations then reason backward todetermine the selection and location of lighting fixtures that produce theseeffectsWe realized that the Space Pen could be an effective interface forlighting designers to specify design intentions

Inverse calculation of lighting effects working backward from lightingeffect to position and characteristics of lighting sources is not newTheappositely named Painting with Light system [17] computes color andintensity values for fixed theater lighting based on a lighting effect that thescene designer paints on a model of the stage surface Inverse calculation isalso used in other design domains For example mechanical engineers useinverse kinematics to determine the geometry of a mechanism needed toproduce a given set of motions

216 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 216

This idea-sketching on a 3-D model to identify desired lighting effects-sparked the development of the Light Pen More generally we saw thatsketching in 3-D could be a direct and natural means to interact withsystems that reason about and calculate on three-dimensional modelsWeproposed to use light-painting as input to an automated design assistant thatwould help an architect select and position lamps to produce desiredlighting effects

24 Sketching for daylight visualization

Our work on Spot began with the interest of providing sunlight visualizationto 3D space and to complement Light Penrsquos lighting fixture design featuresSpot follows the same framework using Space Penrsquos Java 3D sketchingnavigation and annotation platform as an interface to knowledge basedsystems Spot provides direct sunlight visualization in a navigable 3D space Itcomputes 3D geometry (spatial variables XY and Z) and also the sun anglevariations with the diurnal and annual cycles (temporal variables date andtime)

Multi-dimensional data is usually displayed on a single 2D visualizationpaneThe Space Series project [18] uses a focus-plus-context technique tosupport display of spatial and temporal data variations For lighting expertswith specialized knowledge this 2D display is sufficient However easyvisualization and interaction techniques would better support architectsrsquoqualitative assessment when designing in 3D Software like Ecotect [19]displays the pattern of multiple shadows projected during a period of timeon a single diagram but it does not quantify the amount of received light

To initiate lighting visualization in Spot designers first sketch a boundaryshape on the 3D model indicating the area for simulation Spot thengenerates a representation of the spatial distribution of the illuminationlevel on the selected surfaces over time Spot also enables designers tovisualize the light distribution over time for a given point For each pointtapped by pen (or clicked by mouse) on the 3D model Spot generates acalendar chart where the X and Y axis represent the months of the yearand the time of the dayThe color of each calendar cell corresponds to thecalculated amounts of light reaching the point the user tapped

3 Light Pen

The Light Pen system consists of a 3D sketching front end to a rule-basedelectrical lighting fixture advisor Below we briefly describe the systemarchitecture and the components of Light Pen and a use scenario todemonstrate how it supports lighting design More technical detail onimplementation can be found in (Jung Gross and Do 2003) Here weprovide a conceptual explanation of the work and the current extensions ofthe rules

217Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 217

31 System architecture

Figure 1 illustrates Light Penrsquos system architecture It consists of twocommunicating components the Space Pen 3-D drawing program (left) andthe Lux lighting advisor (right)

The designer interacts directly with Space Pen which provides tools for3-D browsing and sketchingAfter importing a three-dimensional (VRML)model the designer marks it up to indicate desired lighting effectsThemodel geometry and the designerrsquos lighting sketch marks are passed to theLux lighting design advisor

Lux is Light Penrsquos lsquoback endrsquo intelligent system coded in Java as a simpleset of lighting design decision rules Lux accepts the lighting sketch marksand building geometry as inputs Based on the desired lighting and themodel geometry Lux recommends solutions selecting fixtures based ontheir desired characteristics Finally it passes these recommendations backto Space Pen which adds the fixtures to the 3-D model to indicate Luxrsquosproposed design solution

32The Lux lighting design advisor

Lux is the lsquoback endrsquo intelligent system component of Light Pen Lux firstanalyzes the model geometry on and near the designerrsquos lighting sketch-mark It determines for example whether the area to be lit is a floor wallor work-surface and whether the area is large or small Next based on thisanalysis Lux determines whether the lighting category is task accent orambientThen the system selects a set of appropriate lighting fixtures basedon the lighting category and the architectural geometry For example it willsuggest track-lighting only if it finds an appropriate surface nearby formounting the lighting track Finally for each fixture that it deems appropriatefor the lighting category Lux identifies an appropriate locationThese stepsare described in the following sections

m Figure 1 Light Penrsquos two

components a 3-D sketch browser

(Space Pen) and a knowledge-based

advisor (Lux)

218 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 218

Analyze the model geometry

The architectural model is imported from a CAD program in VRML formatso it consists merely of a set of surfaces with no information about whatbuilding components or furniture the surfaces may represent Rather thantrying to classify all surfaces in the model a posteriori or requiring thedesigner to tag them a priori we chose to perform a local analysis on theparts of the model where the designer has sketched lighting marksThe Luxrules use simple predicates to reason about the local architectural geometryof the illumination problemThese predicates include tests for coplanarityvertical and horizontal surfaces as well as more specific tests to determinewhether a surface is a floor a work-height surface or near the ceiling

The designerrsquos marks on the 3-D model are passed as a query to theLux lighting design advisor which takes three actions

bull First it identifies the type of illumination needed based on the size ofthe light mark and the surface it is drawn on

bull Second it selects a lighting fixture suitable for the illumination typeeg track lights for task lighting a pendant lamp for ambientillumination

bull Third it proposes appropriate positions for mounting the lightingfixture

This information is conveyed back to the Space Pen which adds theproposed design elements in their positions to the model and displays thenew scene indicating with a cone of light the illumination effects theyprovide

Identify illumination task

Based on the analysis of architectural geometry and the size and location ofthe lighting sketch mark Lux then determines the desired category ofillumination It selects ldquotask lightingrdquo if the surfaces to be illuminated arehorizontal and the surface is close to the ceiling Otherwise if the area tobe lit is small it selects ldquoaccent lightingrdquo If the area to be lit is large or thesurfaces to be lit are both horizontal and vertical then Lux selectsldquoambientrdquo as the illumination category

Select Appropriate Fixtur

Next using a decision tree Lux selects a fixture or a set of fixturesappropriate to the illumination category that it determined in the previousstep If the illumination category is ldquotask lightingrdquo then the set of possiblefixtures includes several possibilities [spotlight desk lamp table lampfluorescent light long pendant light] If the illumination category is ldquoaccentrdquothen the only fixture it suggests is ldquospotlightrdquo If the illumination category is

219Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 219

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 3: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

1 Introduction

11 Motivation ndash tools for lighting design

Lighting design is an important part of architectural designArchitectsconsider light as the ldquofundamental basis of architecturerdquo (Le Corbusier)ldquoweonly know the world as it is evoked by lightrdquo (Louis Kahn) light is part ofthe ldquostructure of the thinking of the architecturerdquo (Richard Meier) and thewall is a ldquoluminous vertical surfacerdquo (Carlo Scarpa) Light is a design variablethat affects the character of life and activities in the spaces people inhabitThe orientation form and scale of a building the arrangement of openingsand glazing as well as spatial configurations of rooms are strongly influencedby architectsrsquo approaches to light

Lighting design requires the consideration of human needs such ascomfort and aesthetics and the solutions for energy efficiency and costeffectivenessThe balancing act of the quantitative and qualitative concernsof lighting design remains a difficult challengeArchitects have traditionallydealt with lighting design in buildings qualitatively and intuitively Recentadvancements in computer graphics provide compelling rendering tools forphoto-realistic visualization to support qualitative assessment Research andsoftware applications for numerical calculation of diurnal and annual cyclesfor sunlight now provide methods for quantitative analysis and assessmentBoth are welcome developments However the question is not whetherquantitative or qualitative methods provide better support for lightingdesign Designers need to approach lighting both qualitatively andquantitativelyA good design tool should provide relevant information beintegrated into the design environment to better support quick iterativeexplorations essential to a design decision-making processTo address thisneed we propose a framework of a computer-aided 3D sketchingenvironment with knowledge-based lighting design systems

This paper presents two software prototypes developed under theframework that link intuitive user interfaces to algorithmic calculations thatoperate on a 3D modelTo address the essential pair ndash the quantitative andthe qualitative ndash we have chosen to implement software modules toseparately investigate daylight simulation and lighting fixture design usingsketching in three-dimensional space to manage the two modules Spotrenders daylight analysis and visualization over time on any surface in spaceLight Pen recommends lighting fixture placements from intended illuminatedsurfaces Below we briefly provide our rationale for building thesecomputational tools for lighting visualization analysis and design

12 Daylight analysis

Visualization and analysis of sunlight effects in buildings is essential fordaylight design and performance Sunlight has been considered an importantfeature for any successful space design as well as an energy source for

213Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 213

buildings [1] Louis Kahn stressed the importance of sunlight and arguedthat ldquoA room is not a room without natural lightrdquo Daylight design involvesconsiderations of the variation of lighting conditions and the degree ofpenetration into buildings Researchers have developed many methods tohelp quantitatively assess projected lighting levels Hand calculation methodssuch as Lumen methods daylight protractors and work sheets are usefultools for calculating physical parameters such as the size and shape of askylight Physical model and tilt tables also help analyze full load lightinghours and density However applying any of the methods above is laborintensive and involves several steps

Recently we have seen ray tracing and other computer graphicstechniques being employed (eg Radiance) to simulate the behavior of lightin the real environmentThese tools can simulate lighting effects of a spaceat a given moment in a viewpoint However they do not provide real timefeedback of lighting effects over timeA high-resolution image of outputfrom a simulation requires complex computation to decide the luminancevalue and usually takes time to generate During the early conceptual designstage designers need a simple-to-use simulation and visualization tool tohelp them understand and recognize lighting problems and opportunitiesRapid direct sunlight simulation in the 3D design environment would beuseful in early stage designTherefore we implemented the Spot system toenable architects to sketch in 3D to specify areas for quick previews andcalculations of the amount of direct sunlight projected over time

13 Lighting fixture design

A lighting designerrsquos main task is to select lighting fixtures and position themin the building appropriately for the activities that are to take placeThelocation size and orientation of windows and skylights must be consideredbut typically the architect has already made these building design decisionsTypically a lighting design includes three categories of illumination ambienttask and accent lightingAmbient illumination supports general activities inspace such as wayfinding orientation and movementTask illuminationsupports specific activities such as reading work eating and conversationAccent lighting highlights points of interest such as paintings photographsor architectural details Several alternatives exist for implementing each typeof lighting for example recessed ceiling lights fluorescent lamps indirectlyreflecting pendant lamps floor and table lamps and track lights Each hasspecific preconditions for installation advantages and drawbacksThe lightingdesigner must also specify the configuration and location of switches [2]

14 Structure of the paper

In the following sections we briefly introduce Space Pen a platform that webuilt earlier for browsing and sketch annotation in 3-D and explain how itled to Spot and Light Pen Next we demonstrate Spot and Light Pen and

214 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 214

outline the lighting design expertise that these programs embodyWe closewith a brief discussion about how 3D sketching interface enables easierinteraction in the specific domain of architectural lighting and directions forfuture work

2 Sketching as an interface to lighting design tools

21 Intelligent sketching systems

Generating 3-D geometry from 2-D sketches has been a lsquoholy grailrsquo researchtopic for some time yielding diverse approaches to pen-based interfaces forgenerating and editing three-dimensional models Sketch [3] like itscommercial cousin Sketchup recognizes gestures and generates modelingcommands rather than parsing and interpreting a line drawing whereasTeddy [4] uses a simple heuristic to inflate three dimensional curvilinearforms from a freehand drawingVR Sketchpad [5] combines extrusion andsymbol recognition to rapidly create VRML worlds Chateau [6] tries toanticipate the userrsquos intentions in generating a three-dimensional modeloffering alternative 3-D completions as the user draws a 2-D sketch DigitalClay [7] uses constraint propagation of concave and convex vertices togenerate three-dimensional models from a two-dimensional diagram Stilton[8] enables designers to draw into a three-dimensional scene represented inVRML the program parses the line drawing on the fly to add 3-D geometryto the scene

Beyond sketch-to-3-D geometry creation a quite different line ofsketching research explores using diagrams to interact with applications suchas simulation programs databases and other intelligent systemsThisapproach is especially appropriate in domains where drawings are commonrepresentations Diagrammatic interaction has been explored in physicaldomains such as mechanical engineering [9] and geography [10] as well asto support tasks such as interaction design [11] and military action planning[12] In this vein our earlier work developed a general system fordiagrammatic interaction based on an end-user programmable visuallanguage [13]

Diagram interfaces for intelligent systems have mostly supported two-dimensional drawing However in physical design domains such asmechanical engineering entertainment and architecture the artifacts thatdesigners manipulate are typically 3-D computer graphics models Designersoperate on these models with applications that process 3-D data anddisplay results in 3-D For example a mechanical engineer may performkinematics analyses of a mechanism in 3-D or finite element analysis of astructure a game designer may perform visual analysis of sight-lines and anarchitect may test a proposed building for emergency egress routesTheseanalysis tasks typically involve running a 3-D model through a stand-aloneapplication

We would like to bring applications that operate on 3-D models into

215Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 215

the designerrsquos working environment to seamlessly integrate design andanalysis Designers often begin work with informal sketches and diagramsThat is why we want to employ pen-based interfaces to interact with theintelligent systems that serve as computational assistants in three-dimensional design domainsWe built Spot and Light Pen to demonstratehow such an interface can be used in the domain of architectural lightingdesign

22 Space Pen

Our platform for 3-D sketching is Space Pen [14] software we built tosupport Web based design collaboration with annotation capabilities in 3-DThe Space Pen server converts any VRML model posted by the architectsinto a Java 3D model in a standard Web browser Collaborating teammembers can then browse and annotate by drawing on model surfaces Forexample a team member reviewing a proposed architectural design drawson a wall-graffiti style-to indicate a proposed location for a new windowSpace Pen also supports text annotation with threaded discussions linkedto Post-It(r) style tags left in the model Designers mark on existing modelsurfaces or on a temporary drawing plane to add geometry to the modelSpace Pen identifies figures such as arrows rectangles and circles which itcan then rectify as model geometry or interpret as commands In short theSpace Pen provides a platform for drawing onto and into 3-D models

23 Sketching with light

Our work on Light Pen began after we demonstrated Space Pen [15] to aprofessional lighting designer [16]We wondered how to improve SpacePenrsquos rendering capabilities to support lighting design tasks She commentedthat commercial lighting design software applications render excellently thevisual appearance of a proposed lighting design configuration Givenpositions of a set of lighting fixtures windows and architectural geometrythe software renders the resulting lighting effects at a given date and timeWhat lighting designers really do she explained is ldquopaint with lightrdquo-theyidentify desired lighting effects at specific locations then reason backward todetermine the selection and location of lighting fixtures that produce theseeffectsWe realized that the Space Pen could be an effective interface forlighting designers to specify design intentions

Inverse calculation of lighting effects working backward from lightingeffect to position and characteristics of lighting sources is not newTheappositely named Painting with Light system [17] computes color andintensity values for fixed theater lighting based on a lighting effect that thescene designer paints on a model of the stage surface Inverse calculation isalso used in other design domains For example mechanical engineers useinverse kinematics to determine the geometry of a mechanism needed toproduce a given set of motions

216 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 216

This idea-sketching on a 3-D model to identify desired lighting effects-sparked the development of the Light Pen More generally we saw thatsketching in 3-D could be a direct and natural means to interact withsystems that reason about and calculate on three-dimensional modelsWeproposed to use light-painting as input to an automated design assistant thatwould help an architect select and position lamps to produce desiredlighting effects

24 Sketching for daylight visualization

Our work on Spot began with the interest of providing sunlight visualizationto 3D space and to complement Light Penrsquos lighting fixture design featuresSpot follows the same framework using Space Penrsquos Java 3D sketchingnavigation and annotation platform as an interface to knowledge basedsystems Spot provides direct sunlight visualization in a navigable 3D space Itcomputes 3D geometry (spatial variables XY and Z) and also the sun anglevariations with the diurnal and annual cycles (temporal variables date andtime)

Multi-dimensional data is usually displayed on a single 2D visualizationpaneThe Space Series project [18] uses a focus-plus-context technique tosupport display of spatial and temporal data variations For lighting expertswith specialized knowledge this 2D display is sufficient However easyvisualization and interaction techniques would better support architectsrsquoqualitative assessment when designing in 3D Software like Ecotect [19]displays the pattern of multiple shadows projected during a period of timeon a single diagram but it does not quantify the amount of received light

To initiate lighting visualization in Spot designers first sketch a boundaryshape on the 3D model indicating the area for simulation Spot thengenerates a representation of the spatial distribution of the illuminationlevel on the selected surfaces over time Spot also enables designers tovisualize the light distribution over time for a given point For each pointtapped by pen (or clicked by mouse) on the 3D model Spot generates acalendar chart where the X and Y axis represent the months of the yearand the time of the dayThe color of each calendar cell corresponds to thecalculated amounts of light reaching the point the user tapped

3 Light Pen

The Light Pen system consists of a 3D sketching front end to a rule-basedelectrical lighting fixture advisor Below we briefly describe the systemarchitecture and the components of Light Pen and a use scenario todemonstrate how it supports lighting design More technical detail onimplementation can be found in (Jung Gross and Do 2003) Here weprovide a conceptual explanation of the work and the current extensions ofthe rules

217Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 217

31 System architecture

Figure 1 illustrates Light Penrsquos system architecture It consists of twocommunicating components the Space Pen 3-D drawing program (left) andthe Lux lighting advisor (right)

The designer interacts directly with Space Pen which provides tools for3-D browsing and sketchingAfter importing a three-dimensional (VRML)model the designer marks it up to indicate desired lighting effectsThemodel geometry and the designerrsquos lighting sketch marks are passed to theLux lighting design advisor

Lux is Light Penrsquos lsquoback endrsquo intelligent system coded in Java as a simpleset of lighting design decision rules Lux accepts the lighting sketch marksand building geometry as inputs Based on the desired lighting and themodel geometry Lux recommends solutions selecting fixtures based ontheir desired characteristics Finally it passes these recommendations backto Space Pen which adds the fixtures to the 3-D model to indicate Luxrsquosproposed design solution

32The Lux lighting design advisor

Lux is the lsquoback endrsquo intelligent system component of Light Pen Lux firstanalyzes the model geometry on and near the designerrsquos lighting sketch-mark It determines for example whether the area to be lit is a floor wallor work-surface and whether the area is large or small Next based on thisanalysis Lux determines whether the lighting category is task accent orambientThen the system selects a set of appropriate lighting fixtures basedon the lighting category and the architectural geometry For example it willsuggest track-lighting only if it finds an appropriate surface nearby formounting the lighting track Finally for each fixture that it deems appropriatefor the lighting category Lux identifies an appropriate locationThese stepsare described in the following sections

m Figure 1 Light Penrsquos two

components a 3-D sketch browser

(Space Pen) and a knowledge-based

advisor (Lux)

218 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 218

Analyze the model geometry

The architectural model is imported from a CAD program in VRML formatso it consists merely of a set of surfaces with no information about whatbuilding components or furniture the surfaces may represent Rather thantrying to classify all surfaces in the model a posteriori or requiring thedesigner to tag them a priori we chose to perform a local analysis on theparts of the model where the designer has sketched lighting marksThe Luxrules use simple predicates to reason about the local architectural geometryof the illumination problemThese predicates include tests for coplanarityvertical and horizontal surfaces as well as more specific tests to determinewhether a surface is a floor a work-height surface or near the ceiling

The designerrsquos marks on the 3-D model are passed as a query to theLux lighting design advisor which takes three actions

bull First it identifies the type of illumination needed based on the size ofthe light mark and the surface it is drawn on

bull Second it selects a lighting fixture suitable for the illumination typeeg track lights for task lighting a pendant lamp for ambientillumination

bull Third it proposes appropriate positions for mounting the lightingfixture

This information is conveyed back to the Space Pen which adds theproposed design elements in their positions to the model and displays thenew scene indicating with a cone of light the illumination effects theyprovide

Identify illumination task

Based on the analysis of architectural geometry and the size and location ofthe lighting sketch mark Lux then determines the desired category ofillumination It selects ldquotask lightingrdquo if the surfaces to be illuminated arehorizontal and the surface is close to the ceiling Otherwise if the area tobe lit is small it selects ldquoaccent lightingrdquo If the area to be lit is large or thesurfaces to be lit are both horizontal and vertical then Lux selectsldquoambientrdquo as the illumination category

Select Appropriate Fixtur

Next using a decision tree Lux selects a fixture or a set of fixturesappropriate to the illumination category that it determined in the previousstep If the illumination category is ldquotask lightingrdquo then the set of possiblefixtures includes several possibilities [spotlight desk lamp table lampfluorescent light long pendant light] If the illumination category is ldquoaccentrdquothen the only fixture it suggests is ldquospotlightrdquo If the illumination category is

219Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 219

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 4: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

buildings [1] Louis Kahn stressed the importance of sunlight and arguedthat ldquoA room is not a room without natural lightrdquo Daylight design involvesconsiderations of the variation of lighting conditions and the degree ofpenetration into buildings Researchers have developed many methods tohelp quantitatively assess projected lighting levels Hand calculation methodssuch as Lumen methods daylight protractors and work sheets are usefultools for calculating physical parameters such as the size and shape of askylight Physical model and tilt tables also help analyze full load lightinghours and density However applying any of the methods above is laborintensive and involves several steps

Recently we have seen ray tracing and other computer graphicstechniques being employed (eg Radiance) to simulate the behavior of lightin the real environmentThese tools can simulate lighting effects of a spaceat a given moment in a viewpoint However they do not provide real timefeedback of lighting effects over timeA high-resolution image of outputfrom a simulation requires complex computation to decide the luminancevalue and usually takes time to generate During the early conceptual designstage designers need a simple-to-use simulation and visualization tool tohelp them understand and recognize lighting problems and opportunitiesRapid direct sunlight simulation in the 3D design environment would beuseful in early stage designTherefore we implemented the Spot system toenable architects to sketch in 3D to specify areas for quick previews andcalculations of the amount of direct sunlight projected over time

13 Lighting fixture design

A lighting designerrsquos main task is to select lighting fixtures and position themin the building appropriately for the activities that are to take placeThelocation size and orientation of windows and skylights must be consideredbut typically the architect has already made these building design decisionsTypically a lighting design includes three categories of illumination ambienttask and accent lightingAmbient illumination supports general activities inspace such as wayfinding orientation and movementTask illuminationsupports specific activities such as reading work eating and conversationAccent lighting highlights points of interest such as paintings photographsor architectural details Several alternatives exist for implementing each typeof lighting for example recessed ceiling lights fluorescent lamps indirectlyreflecting pendant lamps floor and table lamps and track lights Each hasspecific preconditions for installation advantages and drawbacksThe lightingdesigner must also specify the configuration and location of switches [2]

14 Structure of the paper

In the following sections we briefly introduce Space Pen a platform that webuilt earlier for browsing and sketch annotation in 3-D and explain how itled to Spot and Light Pen Next we demonstrate Spot and Light Pen and

214 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 214

outline the lighting design expertise that these programs embodyWe closewith a brief discussion about how 3D sketching interface enables easierinteraction in the specific domain of architectural lighting and directions forfuture work

2 Sketching as an interface to lighting design tools

21 Intelligent sketching systems

Generating 3-D geometry from 2-D sketches has been a lsquoholy grailrsquo researchtopic for some time yielding diverse approaches to pen-based interfaces forgenerating and editing three-dimensional models Sketch [3] like itscommercial cousin Sketchup recognizes gestures and generates modelingcommands rather than parsing and interpreting a line drawing whereasTeddy [4] uses a simple heuristic to inflate three dimensional curvilinearforms from a freehand drawingVR Sketchpad [5] combines extrusion andsymbol recognition to rapidly create VRML worlds Chateau [6] tries toanticipate the userrsquos intentions in generating a three-dimensional modeloffering alternative 3-D completions as the user draws a 2-D sketch DigitalClay [7] uses constraint propagation of concave and convex vertices togenerate three-dimensional models from a two-dimensional diagram Stilton[8] enables designers to draw into a three-dimensional scene represented inVRML the program parses the line drawing on the fly to add 3-D geometryto the scene

Beyond sketch-to-3-D geometry creation a quite different line ofsketching research explores using diagrams to interact with applications suchas simulation programs databases and other intelligent systemsThisapproach is especially appropriate in domains where drawings are commonrepresentations Diagrammatic interaction has been explored in physicaldomains such as mechanical engineering [9] and geography [10] as well asto support tasks such as interaction design [11] and military action planning[12] In this vein our earlier work developed a general system fordiagrammatic interaction based on an end-user programmable visuallanguage [13]

Diagram interfaces for intelligent systems have mostly supported two-dimensional drawing However in physical design domains such asmechanical engineering entertainment and architecture the artifacts thatdesigners manipulate are typically 3-D computer graphics models Designersoperate on these models with applications that process 3-D data anddisplay results in 3-D For example a mechanical engineer may performkinematics analyses of a mechanism in 3-D or finite element analysis of astructure a game designer may perform visual analysis of sight-lines and anarchitect may test a proposed building for emergency egress routesTheseanalysis tasks typically involve running a 3-D model through a stand-aloneapplication

We would like to bring applications that operate on 3-D models into

215Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 215

the designerrsquos working environment to seamlessly integrate design andanalysis Designers often begin work with informal sketches and diagramsThat is why we want to employ pen-based interfaces to interact with theintelligent systems that serve as computational assistants in three-dimensional design domainsWe built Spot and Light Pen to demonstratehow such an interface can be used in the domain of architectural lightingdesign

22 Space Pen

Our platform for 3-D sketching is Space Pen [14] software we built tosupport Web based design collaboration with annotation capabilities in 3-DThe Space Pen server converts any VRML model posted by the architectsinto a Java 3D model in a standard Web browser Collaborating teammembers can then browse and annotate by drawing on model surfaces Forexample a team member reviewing a proposed architectural design drawson a wall-graffiti style-to indicate a proposed location for a new windowSpace Pen also supports text annotation with threaded discussions linkedto Post-It(r) style tags left in the model Designers mark on existing modelsurfaces or on a temporary drawing plane to add geometry to the modelSpace Pen identifies figures such as arrows rectangles and circles which itcan then rectify as model geometry or interpret as commands In short theSpace Pen provides a platform for drawing onto and into 3-D models

23 Sketching with light

Our work on Light Pen began after we demonstrated Space Pen [15] to aprofessional lighting designer [16]We wondered how to improve SpacePenrsquos rendering capabilities to support lighting design tasks She commentedthat commercial lighting design software applications render excellently thevisual appearance of a proposed lighting design configuration Givenpositions of a set of lighting fixtures windows and architectural geometrythe software renders the resulting lighting effects at a given date and timeWhat lighting designers really do she explained is ldquopaint with lightrdquo-theyidentify desired lighting effects at specific locations then reason backward todetermine the selection and location of lighting fixtures that produce theseeffectsWe realized that the Space Pen could be an effective interface forlighting designers to specify design intentions

Inverse calculation of lighting effects working backward from lightingeffect to position and characteristics of lighting sources is not newTheappositely named Painting with Light system [17] computes color andintensity values for fixed theater lighting based on a lighting effect that thescene designer paints on a model of the stage surface Inverse calculation isalso used in other design domains For example mechanical engineers useinverse kinematics to determine the geometry of a mechanism needed toproduce a given set of motions

216 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 216

This idea-sketching on a 3-D model to identify desired lighting effects-sparked the development of the Light Pen More generally we saw thatsketching in 3-D could be a direct and natural means to interact withsystems that reason about and calculate on three-dimensional modelsWeproposed to use light-painting as input to an automated design assistant thatwould help an architect select and position lamps to produce desiredlighting effects

24 Sketching for daylight visualization

Our work on Spot began with the interest of providing sunlight visualizationto 3D space and to complement Light Penrsquos lighting fixture design featuresSpot follows the same framework using Space Penrsquos Java 3D sketchingnavigation and annotation platform as an interface to knowledge basedsystems Spot provides direct sunlight visualization in a navigable 3D space Itcomputes 3D geometry (spatial variables XY and Z) and also the sun anglevariations with the diurnal and annual cycles (temporal variables date andtime)

Multi-dimensional data is usually displayed on a single 2D visualizationpaneThe Space Series project [18] uses a focus-plus-context technique tosupport display of spatial and temporal data variations For lighting expertswith specialized knowledge this 2D display is sufficient However easyvisualization and interaction techniques would better support architectsrsquoqualitative assessment when designing in 3D Software like Ecotect [19]displays the pattern of multiple shadows projected during a period of timeon a single diagram but it does not quantify the amount of received light

To initiate lighting visualization in Spot designers first sketch a boundaryshape on the 3D model indicating the area for simulation Spot thengenerates a representation of the spatial distribution of the illuminationlevel on the selected surfaces over time Spot also enables designers tovisualize the light distribution over time for a given point For each pointtapped by pen (or clicked by mouse) on the 3D model Spot generates acalendar chart where the X and Y axis represent the months of the yearand the time of the dayThe color of each calendar cell corresponds to thecalculated amounts of light reaching the point the user tapped

3 Light Pen

The Light Pen system consists of a 3D sketching front end to a rule-basedelectrical lighting fixture advisor Below we briefly describe the systemarchitecture and the components of Light Pen and a use scenario todemonstrate how it supports lighting design More technical detail onimplementation can be found in (Jung Gross and Do 2003) Here weprovide a conceptual explanation of the work and the current extensions ofthe rules

217Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 217

31 System architecture

Figure 1 illustrates Light Penrsquos system architecture It consists of twocommunicating components the Space Pen 3-D drawing program (left) andthe Lux lighting advisor (right)

The designer interacts directly with Space Pen which provides tools for3-D browsing and sketchingAfter importing a three-dimensional (VRML)model the designer marks it up to indicate desired lighting effectsThemodel geometry and the designerrsquos lighting sketch marks are passed to theLux lighting design advisor

Lux is Light Penrsquos lsquoback endrsquo intelligent system coded in Java as a simpleset of lighting design decision rules Lux accepts the lighting sketch marksand building geometry as inputs Based on the desired lighting and themodel geometry Lux recommends solutions selecting fixtures based ontheir desired characteristics Finally it passes these recommendations backto Space Pen which adds the fixtures to the 3-D model to indicate Luxrsquosproposed design solution

32The Lux lighting design advisor

Lux is the lsquoback endrsquo intelligent system component of Light Pen Lux firstanalyzes the model geometry on and near the designerrsquos lighting sketch-mark It determines for example whether the area to be lit is a floor wallor work-surface and whether the area is large or small Next based on thisanalysis Lux determines whether the lighting category is task accent orambientThen the system selects a set of appropriate lighting fixtures basedon the lighting category and the architectural geometry For example it willsuggest track-lighting only if it finds an appropriate surface nearby formounting the lighting track Finally for each fixture that it deems appropriatefor the lighting category Lux identifies an appropriate locationThese stepsare described in the following sections

m Figure 1 Light Penrsquos two

components a 3-D sketch browser

(Space Pen) and a knowledge-based

advisor (Lux)

218 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 218

Analyze the model geometry

The architectural model is imported from a CAD program in VRML formatso it consists merely of a set of surfaces with no information about whatbuilding components or furniture the surfaces may represent Rather thantrying to classify all surfaces in the model a posteriori or requiring thedesigner to tag them a priori we chose to perform a local analysis on theparts of the model where the designer has sketched lighting marksThe Luxrules use simple predicates to reason about the local architectural geometryof the illumination problemThese predicates include tests for coplanarityvertical and horizontal surfaces as well as more specific tests to determinewhether a surface is a floor a work-height surface or near the ceiling

The designerrsquos marks on the 3-D model are passed as a query to theLux lighting design advisor which takes three actions

bull First it identifies the type of illumination needed based on the size ofthe light mark and the surface it is drawn on

bull Second it selects a lighting fixture suitable for the illumination typeeg track lights for task lighting a pendant lamp for ambientillumination

bull Third it proposes appropriate positions for mounting the lightingfixture

This information is conveyed back to the Space Pen which adds theproposed design elements in their positions to the model and displays thenew scene indicating with a cone of light the illumination effects theyprovide

Identify illumination task

Based on the analysis of architectural geometry and the size and location ofthe lighting sketch mark Lux then determines the desired category ofillumination It selects ldquotask lightingrdquo if the surfaces to be illuminated arehorizontal and the surface is close to the ceiling Otherwise if the area tobe lit is small it selects ldquoaccent lightingrdquo If the area to be lit is large or thesurfaces to be lit are both horizontal and vertical then Lux selectsldquoambientrdquo as the illumination category

Select Appropriate Fixtur

Next using a decision tree Lux selects a fixture or a set of fixturesappropriate to the illumination category that it determined in the previousstep If the illumination category is ldquotask lightingrdquo then the set of possiblefixtures includes several possibilities [spotlight desk lamp table lampfluorescent light long pendant light] If the illumination category is ldquoaccentrdquothen the only fixture it suggests is ldquospotlightrdquo If the illumination category is

219Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 219

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 5: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

outline the lighting design expertise that these programs embodyWe closewith a brief discussion about how 3D sketching interface enables easierinteraction in the specific domain of architectural lighting and directions forfuture work

2 Sketching as an interface to lighting design tools

21 Intelligent sketching systems

Generating 3-D geometry from 2-D sketches has been a lsquoholy grailrsquo researchtopic for some time yielding diverse approaches to pen-based interfaces forgenerating and editing three-dimensional models Sketch [3] like itscommercial cousin Sketchup recognizes gestures and generates modelingcommands rather than parsing and interpreting a line drawing whereasTeddy [4] uses a simple heuristic to inflate three dimensional curvilinearforms from a freehand drawingVR Sketchpad [5] combines extrusion andsymbol recognition to rapidly create VRML worlds Chateau [6] tries toanticipate the userrsquos intentions in generating a three-dimensional modeloffering alternative 3-D completions as the user draws a 2-D sketch DigitalClay [7] uses constraint propagation of concave and convex vertices togenerate three-dimensional models from a two-dimensional diagram Stilton[8] enables designers to draw into a three-dimensional scene represented inVRML the program parses the line drawing on the fly to add 3-D geometryto the scene

Beyond sketch-to-3-D geometry creation a quite different line ofsketching research explores using diagrams to interact with applications suchas simulation programs databases and other intelligent systemsThisapproach is especially appropriate in domains where drawings are commonrepresentations Diagrammatic interaction has been explored in physicaldomains such as mechanical engineering [9] and geography [10] as well asto support tasks such as interaction design [11] and military action planning[12] In this vein our earlier work developed a general system fordiagrammatic interaction based on an end-user programmable visuallanguage [13]

Diagram interfaces for intelligent systems have mostly supported two-dimensional drawing However in physical design domains such asmechanical engineering entertainment and architecture the artifacts thatdesigners manipulate are typically 3-D computer graphics models Designersoperate on these models with applications that process 3-D data anddisplay results in 3-D For example a mechanical engineer may performkinematics analyses of a mechanism in 3-D or finite element analysis of astructure a game designer may perform visual analysis of sight-lines and anarchitect may test a proposed building for emergency egress routesTheseanalysis tasks typically involve running a 3-D model through a stand-aloneapplication

We would like to bring applications that operate on 3-D models into

215Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 215

the designerrsquos working environment to seamlessly integrate design andanalysis Designers often begin work with informal sketches and diagramsThat is why we want to employ pen-based interfaces to interact with theintelligent systems that serve as computational assistants in three-dimensional design domainsWe built Spot and Light Pen to demonstratehow such an interface can be used in the domain of architectural lightingdesign

22 Space Pen

Our platform for 3-D sketching is Space Pen [14] software we built tosupport Web based design collaboration with annotation capabilities in 3-DThe Space Pen server converts any VRML model posted by the architectsinto a Java 3D model in a standard Web browser Collaborating teammembers can then browse and annotate by drawing on model surfaces Forexample a team member reviewing a proposed architectural design drawson a wall-graffiti style-to indicate a proposed location for a new windowSpace Pen also supports text annotation with threaded discussions linkedto Post-It(r) style tags left in the model Designers mark on existing modelsurfaces or on a temporary drawing plane to add geometry to the modelSpace Pen identifies figures such as arrows rectangles and circles which itcan then rectify as model geometry or interpret as commands In short theSpace Pen provides a platform for drawing onto and into 3-D models

23 Sketching with light

Our work on Light Pen began after we demonstrated Space Pen [15] to aprofessional lighting designer [16]We wondered how to improve SpacePenrsquos rendering capabilities to support lighting design tasks She commentedthat commercial lighting design software applications render excellently thevisual appearance of a proposed lighting design configuration Givenpositions of a set of lighting fixtures windows and architectural geometrythe software renders the resulting lighting effects at a given date and timeWhat lighting designers really do she explained is ldquopaint with lightrdquo-theyidentify desired lighting effects at specific locations then reason backward todetermine the selection and location of lighting fixtures that produce theseeffectsWe realized that the Space Pen could be an effective interface forlighting designers to specify design intentions

Inverse calculation of lighting effects working backward from lightingeffect to position and characteristics of lighting sources is not newTheappositely named Painting with Light system [17] computes color andintensity values for fixed theater lighting based on a lighting effect that thescene designer paints on a model of the stage surface Inverse calculation isalso used in other design domains For example mechanical engineers useinverse kinematics to determine the geometry of a mechanism needed toproduce a given set of motions

216 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 216

This idea-sketching on a 3-D model to identify desired lighting effects-sparked the development of the Light Pen More generally we saw thatsketching in 3-D could be a direct and natural means to interact withsystems that reason about and calculate on three-dimensional modelsWeproposed to use light-painting as input to an automated design assistant thatwould help an architect select and position lamps to produce desiredlighting effects

24 Sketching for daylight visualization

Our work on Spot began with the interest of providing sunlight visualizationto 3D space and to complement Light Penrsquos lighting fixture design featuresSpot follows the same framework using Space Penrsquos Java 3D sketchingnavigation and annotation platform as an interface to knowledge basedsystems Spot provides direct sunlight visualization in a navigable 3D space Itcomputes 3D geometry (spatial variables XY and Z) and also the sun anglevariations with the diurnal and annual cycles (temporal variables date andtime)

Multi-dimensional data is usually displayed on a single 2D visualizationpaneThe Space Series project [18] uses a focus-plus-context technique tosupport display of spatial and temporal data variations For lighting expertswith specialized knowledge this 2D display is sufficient However easyvisualization and interaction techniques would better support architectsrsquoqualitative assessment when designing in 3D Software like Ecotect [19]displays the pattern of multiple shadows projected during a period of timeon a single diagram but it does not quantify the amount of received light

To initiate lighting visualization in Spot designers first sketch a boundaryshape on the 3D model indicating the area for simulation Spot thengenerates a representation of the spatial distribution of the illuminationlevel on the selected surfaces over time Spot also enables designers tovisualize the light distribution over time for a given point For each pointtapped by pen (or clicked by mouse) on the 3D model Spot generates acalendar chart where the X and Y axis represent the months of the yearand the time of the dayThe color of each calendar cell corresponds to thecalculated amounts of light reaching the point the user tapped

3 Light Pen

The Light Pen system consists of a 3D sketching front end to a rule-basedelectrical lighting fixture advisor Below we briefly describe the systemarchitecture and the components of Light Pen and a use scenario todemonstrate how it supports lighting design More technical detail onimplementation can be found in (Jung Gross and Do 2003) Here weprovide a conceptual explanation of the work and the current extensions ofthe rules

217Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 217

31 System architecture

Figure 1 illustrates Light Penrsquos system architecture It consists of twocommunicating components the Space Pen 3-D drawing program (left) andthe Lux lighting advisor (right)

The designer interacts directly with Space Pen which provides tools for3-D browsing and sketchingAfter importing a three-dimensional (VRML)model the designer marks it up to indicate desired lighting effectsThemodel geometry and the designerrsquos lighting sketch marks are passed to theLux lighting design advisor

Lux is Light Penrsquos lsquoback endrsquo intelligent system coded in Java as a simpleset of lighting design decision rules Lux accepts the lighting sketch marksand building geometry as inputs Based on the desired lighting and themodel geometry Lux recommends solutions selecting fixtures based ontheir desired characteristics Finally it passes these recommendations backto Space Pen which adds the fixtures to the 3-D model to indicate Luxrsquosproposed design solution

32The Lux lighting design advisor

Lux is the lsquoback endrsquo intelligent system component of Light Pen Lux firstanalyzes the model geometry on and near the designerrsquos lighting sketch-mark It determines for example whether the area to be lit is a floor wallor work-surface and whether the area is large or small Next based on thisanalysis Lux determines whether the lighting category is task accent orambientThen the system selects a set of appropriate lighting fixtures basedon the lighting category and the architectural geometry For example it willsuggest track-lighting only if it finds an appropriate surface nearby formounting the lighting track Finally for each fixture that it deems appropriatefor the lighting category Lux identifies an appropriate locationThese stepsare described in the following sections

m Figure 1 Light Penrsquos two

components a 3-D sketch browser

(Space Pen) and a knowledge-based

advisor (Lux)

218 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 218

Analyze the model geometry

The architectural model is imported from a CAD program in VRML formatso it consists merely of a set of surfaces with no information about whatbuilding components or furniture the surfaces may represent Rather thantrying to classify all surfaces in the model a posteriori or requiring thedesigner to tag them a priori we chose to perform a local analysis on theparts of the model where the designer has sketched lighting marksThe Luxrules use simple predicates to reason about the local architectural geometryof the illumination problemThese predicates include tests for coplanarityvertical and horizontal surfaces as well as more specific tests to determinewhether a surface is a floor a work-height surface or near the ceiling

The designerrsquos marks on the 3-D model are passed as a query to theLux lighting design advisor which takes three actions

bull First it identifies the type of illumination needed based on the size ofthe light mark and the surface it is drawn on

bull Second it selects a lighting fixture suitable for the illumination typeeg track lights for task lighting a pendant lamp for ambientillumination

bull Third it proposes appropriate positions for mounting the lightingfixture

This information is conveyed back to the Space Pen which adds theproposed design elements in their positions to the model and displays thenew scene indicating with a cone of light the illumination effects theyprovide

Identify illumination task

Based on the analysis of architectural geometry and the size and location ofthe lighting sketch mark Lux then determines the desired category ofillumination It selects ldquotask lightingrdquo if the surfaces to be illuminated arehorizontal and the surface is close to the ceiling Otherwise if the area tobe lit is small it selects ldquoaccent lightingrdquo If the area to be lit is large or thesurfaces to be lit are both horizontal and vertical then Lux selectsldquoambientrdquo as the illumination category

Select Appropriate Fixtur

Next using a decision tree Lux selects a fixture or a set of fixturesappropriate to the illumination category that it determined in the previousstep If the illumination category is ldquotask lightingrdquo then the set of possiblefixtures includes several possibilities [spotlight desk lamp table lampfluorescent light long pendant light] If the illumination category is ldquoaccentrdquothen the only fixture it suggests is ldquospotlightrdquo If the illumination category is

219Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 219

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 6: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

the designerrsquos working environment to seamlessly integrate design andanalysis Designers often begin work with informal sketches and diagramsThat is why we want to employ pen-based interfaces to interact with theintelligent systems that serve as computational assistants in three-dimensional design domainsWe built Spot and Light Pen to demonstratehow such an interface can be used in the domain of architectural lightingdesign

22 Space Pen

Our platform for 3-D sketching is Space Pen [14] software we built tosupport Web based design collaboration with annotation capabilities in 3-DThe Space Pen server converts any VRML model posted by the architectsinto a Java 3D model in a standard Web browser Collaborating teammembers can then browse and annotate by drawing on model surfaces Forexample a team member reviewing a proposed architectural design drawson a wall-graffiti style-to indicate a proposed location for a new windowSpace Pen also supports text annotation with threaded discussions linkedto Post-It(r) style tags left in the model Designers mark on existing modelsurfaces or on a temporary drawing plane to add geometry to the modelSpace Pen identifies figures such as arrows rectangles and circles which itcan then rectify as model geometry or interpret as commands In short theSpace Pen provides a platform for drawing onto and into 3-D models

23 Sketching with light

Our work on Light Pen began after we demonstrated Space Pen [15] to aprofessional lighting designer [16]We wondered how to improve SpacePenrsquos rendering capabilities to support lighting design tasks She commentedthat commercial lighting design software applications render excellently thevisual appearance of a proposed lighting design configuration Givenpositions of a set of lighting fixtures windows and architectural geometrythe software renders the resulting lighting effects at a given date and timeWhat lighting designers really do she explained is ldquopaint with lightrdquo-theyidentify desired lighting effects at specific locations then reason backward todetermine the selection and location of lighting fixtures that produce theseeffectsWe realized that the Space Pen could be an effective interface forlighting designers to specify design intentions

Inverse calculation of lighting effects working backward from lightingeffect to position and characteristics of lighting sources is not newTheappositely named Painting with Light system [17] computes color andintensity values for fixed theater lighting based on a lighting effect that thescene designer paints on a model of the stage surface Inverse calculation isalso used in other design domains For example mechanical engineers useinverse kinematics to determine the geometry of a mechanism needed toproduce a given set of motions

216 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 216

This idea-sketching on a 3-D model to identify desired lighting effects-sparked the development of the Light Pen More generally we saw thatsketching in 3-D could be a direct and natural means to interact withsystems that reason about and calculate on three-dimensional modelsWeproposed to use light-painting as input to an automated design assistant thatwould help an architect select and position lamps to produce desiredlighting effects

24 Sketching for daylight visualization

Our work on Spot began with the interest of providing sunlight visualizationto 3D space and to complement Light Penrsquos lighting fixture design featuresSpot follows the same framework using Space Penrsquos Java 3D sketchingnavigation and annotation platform as an interface to knowledge basedsystems Spot provides direct sunlight visualization in a navigable 3D space Itcomputes 3D geometry (spatial variables XY and Z) and also the sun anglevariations with the diurnal and annual cycles (temporal variables date andtime)

Multi-dimensional data is usually displayed on a single 2D visualizationpaneThe Space Series project [18] uses a focus-plus-context technique tosupport display of spatial and temporal data variations For lighting expertswith specialized knowledge this 2D display is sufficient However easyvisualization and interaction techniques would better support architectsrsquoqualitative assessment when designing in 3D Software like Ecotect [19]displays the pattern of multiple shadows projected during a period of timeon a single diagram but it does not quantify the amount of received light

To initiate lighting visualization in Spot designers first sketch a boundaryshape on the 3D model indicating the area for simulation Spot thengenerates a representation of the spatial distribution of the illuminationlevel on the selected surfaces over time Spot also enables designers tovisualize the light distribution over time for a given point For each pointtapped by pen (or clicked by mouse) on the 3D model Spot generates acalendar chart where the X and Y axis represent the months of the yearand the time of the dayThe color of each calendar cell corresponds to thecalculated amounts of light reaching the point the user tapped

3 Light Pen

The Light Pen system consists of a 3D sketching front end to a rule-basedelectrical lighting fixture advisor Below we briefly describe the systemarchitecture and the components of Light Pen and a use scenario todemonstrate how it supports lighting design More technical detail onimplementation can be found in (Jung Gross and Do 2003) Here weprovide a conceptual explanation of the work and the current extensions ofthe rules

217Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 217

31 System architecture

Figure 1 illustrates Light Penrsquos system architecture It consists of twocommunicating components the Space Pen 3-D drawing program (left) andthe Lux lighting advisor (right)

The designer interacts directly with Space Pen which provides tools for3-D browsing and sketchingAfter importing a three-dimensional (VRML)model the designer marks it up to indicate desired lighting effectsThemodel geometry and the designerrsquos lighting sketch marks are passed to theLux lighting design advisor

Lux is Light Penrsquos lsquoback endrsquo intelligent system coded in Java as a simpleset of lighting design decision rules Lux accepts the lighting sketch marksand building geometry as inputs Based on the desired lighting and themodel geometry Lux recommends solutions selecting fixtures based ontheir desired characteristics Finally it passes these recommendations backto Space Pen which adds the fixtures to the 3-D model to indicate Luxrsquosproposed design solution

32The Lux lighting design advisor

Lux is the lsquoback endrsquo intelligent system component of Light Pen Lux firstanalyzes the model geometry on and near the designerrsquos lighting sketch-mark It determines for example whether the area to be lit is a floor wallor work-surface and whether the area is large or small Next based on thisanalysis Lux determines whether the lighting category is task accent orambientThen the system selects a set of appropriate lighting fixtures basedon the lighting category and the architectural geometry For example it willsuggest track-lighting only if it finds an appropriate surface nearby formounting the lighting track Finally for each fixture that it deems appropriatefor the lighting category Lux identifies an appropriate locationThese stepsare described in the following sections

m Figure 1 Light Penrsquos two

components a 3-D sketch browser

(Space Pen) and a knowledge-based

advisor (Lux)

218 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 218

Analyze the model geometry

The architectural model is imported from a CAD program in VRML formatso it consists merely of a set of surfaces with no information about whatbuilding components or furniture the surfaces may represent Rather thantrying to classify all surfaces in the model a posteriori or requiring thedesigner to tag them a priori we chose to perform a local analysis on theparts of the model where the designer has sketched lighting marksThe Luxrules use simple predicates to reason about the local architectural geometryof the illumination problemThese predicates include tests for coplanarityvertical and horizontal surfaces as well as more specific tests to determinewhether a surface is a floor a work-height surface or near the ceiling

The designerrsquos marks on the 3-D model are passed as a query to theLux lighting design advisor which takes three actions

bull First it identifies the type of illumination needed based on the size ofthe light mark and the surface it is drawn on

bull Second it selects a lighting fixture suitable for the illumination typeeg track lights for task lighting a pendant lamp for ambientillumination

bull Third it proposes appropriate positions for mounting the lightingfixture

This information is conveyed back to the Space Pen which adds theproposed design elements in their positions to the model and displays thenew scene indicating with a cone of light the illumination effects theyprovide

Identify illumination task

Based on the analysis of architectural geometry and the size and location ofthe lighting sketch mark Lux then determines the desired category ofillumination It selects ldquotask lightingrdquo if the surfaces to be illuminated arehorizontal and the surface is close to the ceiling Otherwise if the area tobe lit is small it selects ldquoaccent lightingrdquo If the area to be lit is large or thesurfaces to be lit are both horizontal and vertical then Lux selectsldquoambientrdquo as the illumination category

Select Appropriate Fixtur

Next using a decision tree Lux selects a fixture or a set of fixturesappropriate to the illumination category that it determined in the previousstep If the illumination category is ldquotask lightingrdquo then the set of possiblefixtures includes several possibilities [spotlight desk lamp table lampfluorescent light long pendant light] If the illumination category is ldquoaccentrdquothen the only fixture it suggests is ldquospotlightrdquo If the illumination category is

219Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 219

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 7: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

This idea-sketching on a 3-D model to identify desired lighting effects-sparked the development of the Light Pen More generally we saw thatsketching in 3-D could be a direct and natural means to interact withsystems that reason about and calculate on three-dimensional modelsWeproposed to use light-painting as input to an automated design assistant thatwould help an architect select and position lamps to produce desiredlighting effects

24 Sketching for daylight visualization

Our work on Spot began with the interest of providing sunlight visualizationto 3D space and to complement Light Penrsquos lighting fixture design featuresSpot follows the same framework using Space Penrsquos Java 3D sketchingnavigation and annotation platform as an interface to knowledge basedsystems Spot provides direct sunlight visualization in a navigable 3D space Itcomputes 3D geometry (spatial variables XY and Z) and also the sun anglevariations with the diurnal and annual cycles (temporal variables date andtime)

Multi-dimensional data is usually displayed on a single 2D visualizationpaneThe Space Series project [18] uses a focus-plus-context technique tosupport display of spatial and temporal data variations For lighting expertswith specialized knowledge this 2D display is sufficient However easyvisualization and interaction techniques would better support architectsrsquoqualitative assessment when designing in 3D Software like Ecotect [19]displays the pattern of multiple shadows projected during a period of timeon a single diagram but it does not quantify the amount of received light

To initiate lighting visualization in Spot designers first sketch a boundaryshape on the 3D model indicating the area for simulation Spot thengenerates a representation of the spatial distribution of the illuminationlevel on the selected surfaces over time Spot also enables designers tovisualize the light distribution over time for a given point For each pointtapped by pen (or clicked by mouse) on the 3D model Spot generates acalendar chart where the X and Y axis represent the months of the yearand the time of the dayThe color of each calendar cell corresponds to thecalculated amounts of light reaching the point the user tapped

3 Light Pen

The Light Pen system consists of a 3D sketching front end to a rule-basedelectrical lighting fixture advisor Below we briefly describe the systemarchitecture and the components of Light Pen and a use scenario todemonstrate how it supports lighting design More technical detail onimplementation can be found in (Jung Gross and Do 2003) Here weprovide a conceptual explanation of the work and the current extensions ofthe rules

217Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 217

31 System architecture

Figure 1 illustrates Light Penrsquos system architecture It consists of twocommunicating components the Space Pen 3-D drawing program (left) andthe Lux lighting advisor (right)

The designer interacts directly with Space Pen which provides tools for3-D browsing and sketchingAfter importing a three-dimensional (VRML)model the designer marks it up to indicate desired lighting effectsThemodel geometry and the designerrsquos lighting sketch marks are passed to theLux lighting design advisor

Lux is Light Penrsquos lsquoback endrsquo intelligent system coded in Java as a simpleset of lighting design decision rules Lux accepts the lighting sketch marksand building geometry as inputs Based on the desired lighting and themodel geometry Lux recommends solutions selecting fixtures based ontheir desired characteristics Finally it passes these recommendations backto Space Pen which adds the fixtures to the 3-D model to indicate Luxrsquosproposed design solution

32The Lux lighting design advisor

Lux is the lsquoback endrsquo intelligent system component of Light Pen Lux firstanalyzes the model geometry on and near the designerrsquos lighting sketch-mark It determines for example whether the area to be lit is a floor wallor work-surface and whether the area is large or small Next based on thisanalysis Lux determines whether the lighting category is task accent orambientThen the system selects a set of appropriate lighting fixtures basedon the lighting category and the architectural geometry For example it willsuggest track-lighting only if it finds an appropriate surface nearby formounting the lighting track Finally for each fixture that it deems appropriatefor the lighting category Lux identifies an appropriate locationThese stepsare described in the following sections

m Figure 1 Light Penrsquos two

components a 3-D sketch browser

(Space Pen) and a knowledge-based

advisor (Lux)

218 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 218

Analyze the model geometry

The architectural model is imported from a CAD program in VRML formatso it consists merely of a set of surfaces with no information about whatbuilding components or furniture the surfaces may represent Rather thantrying to classify all surfaces in the model a posteriori or requiring thedesigner to tag them a priori we chose to perform a local analysis on theparts of the model where the designer has sketched lighting marksThe Luxrules use simple predicates to reason about the local architectural geometryof the illumination problemThese predicates include tests for coplanarityvertical and horizontal surfaces as well as more specific tests to determinewhether a surface is a floor a work-height surface or near the ceiling

The designerrsquos marks on the 3-D model are passed as a query to theLux lighting design advisor which takes three actions

bull First it identifies the type of illumination needed based on the size ofthe light mark and the surface it is drawn on

bull Second it selects a lighting fixture suitable for the illumination typeeg track lights for task lighting a pendant lamp for ambientillumination

bull Third it proposes appropriate positions for mounting the lightingfixture

This information is conveyed back to the Space Pen which adds theproposed design elements in their positions to the model and displays thenew scene indicating with a cone of light the illumination effects theyprovide

Identify illumination task

Based on the analysis of architectural geometry and the size and location ofthe lighting sketch mark Lux then determines the desired category ofillumination It selects ldquotask lightingrdquo if the surfaces to be illuminated arehorizontal and the surface is close to the ceiling Otherwise if the area tobe lit is small it selects ldquoaccent lightingrdquo If the area to be lit is large or thesurfaces to be lit are both horizontal and vertical then Lux selectsldquoambientrdquo as the illumination category

Select Appropriate Fixtur

Next using a decision tree Lux selects a fixture or a set of fixturesappropriate to the illumination category that it determined in the previousstep If the illumination category is ldquotask lightingrdquo then the set of possiblefixtures includes several possibilities [spotlight desk lamp table lampfluorescent light long pendant light] If the illumination category is ldquoaccentrdquothen the only fixture it suggests is ldquospotlightrdquo If the illumination category is

219Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 219

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 8: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

31 System architecture

Figure 1 illustrates Light Penrsquos system architecture It consists of twocommunicating components the Space Pen 3-D drawing program (left) andthe Lux lighting advisor (right)

The designer interacts directly with Space Pen which provides tools for3-D browsing and sketchingAfter importing a three-dimensional (VRML)model the designer marks it up to indicate desired lighting effectsThemodel geometry and the designerrsquos lighting sketch marks are passed to theLux lighting design advisor

Lux is Light Penrsquos lsquoback endrsquo intelligent system coded in Java as a simpleset of lighting design decision rules Lux accepts the lighting sketch marksand building geometry as inputs Based on the desired lighting and themodel geometry Lux recommends solutions selecting fixtures based ontheir desired characteristics Finally it passes these recommendations backto Space Pen which adds the fixtures to the 3-D model to indicate Luxrsquosproposed design solution

32The Lux lighting design advisor

Lux is the lsquoback endrsquo intelligent system component of Light Pen Lux firstanalyzes the model geometry on and near the designerrsquos lighting sketch-mark It determines for example whether the area to be lit is a floor wallor work-surface and whether the area is large or small Next based on thisanalysis Lux determines whether the lighting category is task accent orambientThen the system selects a set of appropriate lighting fixtures basedon the lighting category and the architectural geometry For example it willsuggest track-lighting only if it finds an appropriate surface nearby formounting the lighting track Finally for each fixture that it deems appropriatefor the lighting category Lux identifies an appropriate locationThese stepsare described in the following sections

m Figure 1 Light Penrsquos two

components a 3-D sketch browser

(Space Pen) and a knowledge-based

advisor (Lux)

218 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 218

Analyze the model geometry

The architectural model is imported from a CAD program in VRML formatso it consists merely of a set of surfaces with no information about whatbuilding components or furniture the surfaces may represent Rather thantrying to classify all surfaces in the model a posteriori or requiring thedesigner to tag them a priori we chose to perform a local analysis on theparts of the model where the designer has sketched lighting marksThe Luxrules use simple predicates to reason about the local architectural geometryof the illumination problemThese predicates include tests for coplanarityvertical and horizontal surfaces as well as more specific tests to determinewhether a surface is a floor a work-height surface or near the ceiling

The designerrsquos marks on the 3-D model are passed as a query to theLux lighting design advisor which takes three actions

bull First it identifies the type of illumination needed based on the size ofthe light mark and the surface it is drawn on

bull Second it selects a lighting fixture suitable for the illumination typeeg track lights for task lighting a pendant lamp for ambientillumination

bull Third it proposes appropriate positions for mounting the lightingfixture

This information is conveyed back to the Space Pen which adds theproposed design elements in their positions to the model and displays thenew scene indicating with a cone of light the illumination effects theyprovide

Identify illumination task

Based on the analysis of architectural geometry and the size and location ofthe lighting sketch mark Lux then determines the desired category ofillumination It selects ldquotask lightingrdquo if the surfaces to be illuminated arehorizontal and the surface is close to the ceiling Otherwise if the area tobe lit is small it selects ldquoaccent lightingrdquo If the area to be lit is large or thesurfaces to be lit are both horizontal and vertical then Lux selectsldquoambientrdquo as the illumination category

Select Appropriate Fixtur

Next using a decision tree Lux selects a fixture or a set of fixturesappropriate to the illumination category that it determined in the previousstep If the illumination category is ldquotask lightingrdquo then the set of possiblefixtures includes several possibilities [spotlight desk lamp table lampfluorescent light long pendant light] If the illumination category is ldquoaccentrdquothen the only fixture it suggests is ldquospotlightrdquo If the illumination category is

219Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 219

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 9: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

Analyze the model geometry

The architectural model is imported from a CAD program in VRML formatso it consists merely of a set of surfaces with no information about whatbuilding components or furniture the surfaces may represent Rather thantrying to classify all surfaces in the model a posteriori or requiring thedesigner to tag them a priori we chose to perform a local analysis on theparts of the model where the designer has sketched lighting marksThe Luxrules use simple predicates to reason about the local architectural geometryof the illumination problemThese predicates include tests for coplanarityvertical and horizontal surfaces as well as more specific tests to determinewhether a surface is a floor a work-height surface or near the ceiling

The designerrsquos marks on the 3-D model are passed as a query to theLux lighting design advisor which takes three actions

bull First it identifies the type of illumination needed based on the size ofthe light mark and the surface it is drawn on

bull Second it selects a lighting fixture suitable for the illumination typeeg track lights for task lighting a pendant lamp for ambientillumination

bull Third it proposes appropriate positions for mounting the lightingfixture

This information is conveyed back to the Space Pen which adds theproposed design elements in their positions to the model and displays thenew scene indicating with a cone of light the illumination effects theyprovide

Identify illumination task

Based on the analysis of architectural geometry and the size and location ofthe lighting sketch mark Lux then determines the desired category ofillumination It selects ldquotask lightingrdquo if the surfaces to be illuminated arehorizontal and the surface is close to the ceiling Otherwise if the area tobe lit is small it selects ldquoaccent lightingrdquo If the area to be lit is large or thesurfaces to be lit are both horizontal and vertical then Lux selectsldquoambientrdquo as the illumination category

Select Appropriate Fixtur

Next using a decision tree Lux selects a fixture or a set of fixturesappropriate to the illumination category that it determined in the previousstep If the illumination category is ldquotask lightingrdquo then the set of possiblefixtures includes several possibilities [spotlight desk lamp table lampfluorescent light long pendant light] If the illumination category is ldquoaccentrdquothen the only fixture it suggests is ldquospotlightrdquo If the illumination category is

219Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 219

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 10: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

ldquoambientrdquo then the set of fixtures includes [pendant light floor lampspotlight fluorescent light] Lux considers the local architectural geometryin selecting appropriate fixtures from these sets

Place fixture in model

Finally the Lux lighting design advisor positions the chosen fixture into themodelThe previous fixture selection step guarantees that an appropriatesurface exists for mounting the fixture Still Lux must propose an exactposition so that the fixture can be added to the 3-D modelThe choose-fixture-position method takes as arguments the surface to be illuminatedthe lighting sketch mark and the fixture typeA vector is drawn from thesurface to be illuminated starting at the center of the lighting sketch markThe first surface that the vector intersects will be the mounting surface forthe fixture and the intersection point will mark the location of the fixtureFor a track light a line along the long direction of the lighting sketch markgenerates a corresponding position on the mounting surface

33 Light Pen at work

Figure 2 shows the Light Pen system in use Using Space Pen the designerhas posed a design problem by lsquosketching lightrsquo on surfaces in the modelwhere lighting is desired on the floor in the middle of the room and on thekitchen counter

In response the system proposes lighting fixtures and locationsThe Luxlighting advisor recognizes that ambient lighting is needed and proposes apendant lamp hanging in the middle of the room It proposes track lights toprovide task lighting over kitchen work surfaces

In Figure 3 the designer has moved to a different position in the modeland sketched light to illuminate the picture on the wall Lux suggests arecessed ceiling light fixture

m Figure 2 Designer sketches light Light Pen

recommends solutions

m Figure 3 Requesting illumination for a

picture

220 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 220

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 11: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

4 Spot Fetch sunlight

Spot is a system that consists of the 3D sketching front end to a rule-baseddaylight simulation Below we describe the system architecture and thesimulation componentsA more complete description of the Spot projectcan be found in [20]

41 Spot system architecture

The implementation of Spot contains two distinct and complementarycomponents 1) Time Projection and 2) Navigable Animation

The spatial variables (x y z) of 3D geometry are implemented usingSpace Pen in Java 3D for easy navigation with a standard interface (mousearrow keys or joystick) and text annotation and sketching (pen and tablet)The temporal variables (date and time) are displayed in additional viewswith a look and feel of a 2D graphic calendarThe resulting daylightsimulation is displayed on the 3D environment

42 Sunlight distribution in 3D space

Spot supports focused and selective simulationTo specify a surface forsimulation the user draws a boundary area on the 3D model Spot thenpaints the selected surface with colors of varied gradientsThe colors of thesurfacersquos pixels indicate the accumulated cumulative of illumination over timeThis interaction is shown in Figure 5The process of boundary informationinference and the cell surface calculation is briefly described below

Sketch recognition

When a line is sketched on the 3D model the stroke coordinates areparsed through a shape recognition analysis to determine the closest match

221Let There be Light Knowledge-Based 3-D Sketching Design Tools

m Figure 4 Spot includes a 3-D sketch

browser (Space Pen) and a sunlight

simulation system (Fetch Sunlight) that

includes spatial variables (xyz) and

temporal variables (annual and diural)

IJAC 2-2_3 proof 16-7-04 346 pm Page 221

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 12: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

(rectangle circle or triangle) Once a shape is recognized it can be rectifiedas an area for simulation User can also create a temporary translucentdrawing surface by sketching a straight line on any model surface

Defining the surface characteristics for the cells

Once the corners of the strokersquos bounding box are known a loop functionimplemented in Spot divides the simulated area into cellsThe displayresolution can be specified in an input windowThe simulation processingspeed depends on the number of rows and columns Spot computesillumination and determines the color of each vertex of the cellsThesystem then interpolates the color of each pixel of the surfaceAs a resultthe surface drawn by Spot is a smooth color gradientThis color displayshows the average light intensity for the chosen period of time

43Time projection

Besides displaying the average illumination values in space Spot also supportslsquobehind the scenersquo data visualization and comparison for any points on thelighting simulation result as shown in Figure 6 Clicking on a point will displaya calendar view showing the detail illumination distribution over time Eachcell of the calendar is colored according to the percentage of illumination itreceives Calendars can be generated for any point in the space

The pen acts as a magic information wandWhen user taps a point onthe 3D model Spot marks it with a sphere with a reference color thatserves as index to the corresponding calendar windowThe user can clickon several points to make a comparative analysis For example as shown in

b Figure 5 Left selecting an area for

simulation by drawing a boundary

shape on the 3D model Right shading

in rectangle shows illumination result

b Figure 6The time projection

functionality in SPOT Clicking a point

on the simulation result (left) retrieves

a calendar view (bottom right)

showing the light distribution over

annual and diurnal cycles and average

light intensity valueThis can be

compared with a calendar from

another point (top right)

222 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 222

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 13: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

Figure 7 two points of the simulation may result from different lightdistribution among different seasons

44 Navigable animation

Spotrsquos Navigable Animation enables the user to interactively visualizeshadow casting and animate it through time Designer can sketch on anysurface to indicate the intended area for simulation For example Figure 8(right) shows a rectangle sketch on the floor to indicate the area ofinterestA simple ray-tracer implemented in Spot rapidly renders shadowson a selected area

45 Selective simulation

Instead of waiting for the simulation to render the entire building like manyother programs Spot computes lighting only of the selected areaAs a resultSpot renders the shadow casting in real timeTime animation of shadowcasting appears just a few seconds after the area for simulation is sketched

c Figure 7 Comparing the temporal

distribution of light of two given points

c Figure 8 Selecting an area for

simulation by drawing on the 3D

model (right)

223Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 223

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 14: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

The Java 3D platform provides easy navigation through the 3Denvironment while viewing the animation in 3D space Once an area isrendered the date and time control panel appears on the screen (Figure 9panel on the bottom) By clicking the forward and backward buttons designersnavigate through the simulation results across date and timeThey can examinethe shadow effect over time while walking through the 3D space

5 Discussion 3-D interaction with intelligent systems

Our prototypes Light Pen and Spot demonstrate sketching in 3-D as ameans to interact with knowledge-based applications (eg expert systemssimulations and databases) Both systems use 3D sketching to specify anintended area for analysis or visualization of lighting designWe emphasizethat architectural lighting design exemplifies one appropriate domain for thistechnique but 3-D sketching has broader application

We are proposing that sketching to identify illuminated surfaces or areaof interest is a natural way to design and that an interface of 3D sketchingthat facilitates this approach will be usefulAs we move towards pen-basedcomputing that supports interacting with design documents by sketchingand 3D visualization such methods will become increasingly valuableWebelieve that 3D sketching could be useful as an interface for knowledgebased design systems For example we could extend the back endvisualization and simulation capabilities of Spot and Light Pen to includethermal and energy analysis or air ventilation and circulation

In both Light Pen and Spot sketching is limited to indicating the area wherethe designer wants illumination or simulation Sketching serves as an interfaceto specify the intended focus of attentionWe envision however that otherapplications would require an interface that could recognize and interpret amore sophisticated visual language Currently these systems only employsimple shape recognition (eg rectangles triangles circles arrows and lines)

b Figure 9 Lighting effect animation of

shadow casting provides an interface

(bottom bar) to move forward or

backward through the date (left) and

time (right) by clicking on arrow

buttons (and moving through 3D

space at the same time)

224 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 224

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 15: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

Future work could add recognition and training capabilities of more complexsymbols based on configurations of shapes and rule sets For example thesketch vocabulary could include symbols for sensors wall sconces skylightsfluorescent lights and window shades etc Recognition of these symbols couldtrigger operations to add and modify geometry or a command to activatesimulation or other action For example a circle immediately drawn before anarrow may indicate an intention to move (an object) to a new location asspecified by a symbol (another circle) drawn immediately after the arrow

Light Pen as explained earlier employs a forward chaining reasoningprocess to identify the correct surface for mounting the lighting fixture byfirst recognizing the intended surface for illumination and infer accordinglythe shortest path to a wall surface and the angle of incidenceThe systemadvises the design about fixture placement according to guidelines Forexample a task light will be placed directly above the intended illuminatedwork surface accent lighting will be projected from ceiling to wall andambient light placed to illuminate a larger area of space Future work couldconsider typical user locations and reduce glare or reflections for thesepreferred views Currently the system can deal with sketching on any singlevertical or horizontal plane as well as any two adjacent surfaces of differentanglesThe system currently places a single light fixture on the ceiling toilluminate both surfaces however it would be easy to produce a collectionof design alternatives for user selection of preferences as well as linking to alighting fixture product catalog and specification information

The system could take into account the designerrsquos sketches of windowand skylight openings and lighting fixture placements to generate quicksimulation and visualization Future work could employ simulation enginesfrom commercial software such as RadianceWe chose to implement ourown knowledge-based systems instead of using existing simulation systemsbecause they either have a complicated system architecture that requiresmodification of internal representations to add new functionality or theimplementation dictate a certain input format

Several designers have tested our systemsThey found the idea ofsketching light into a space is intriguing and argued that itrsquos an improvementover a multi-view 2D representationThey found the usual 2D lightingdesign reference chart and table useful but distracting for design tasksThenavigable 3D model provides spatial coherence and accessible perspectiveviews that are advantageous over orthogonal projects In ourimplementation of the Java 3D navigation we support a game like interfacethat uses arrow keys to move and pan because some people found thestandard VRML navigation difficultWith the Space Pen engine the viewpointwhere one sketches or annotates is automatically saved and displayed onthe real-time generated floor plan as an arrow marker Users of the systemor their web collaborators can easily navigate through the space by clickingon any previously saved viewpoints

225Let There be Light Knowledge-Based 3-D Sketching Design Tools

IJAC 2-2_3 proof 16-7-04 346 pm Page 225

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 16: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

While our project does not attempt to create a thorough expertsystem it could look at ways to make the information more useful orlsquotransparentrsquo to the users For example it will be an easy extension toprovide explanations about how the Light Pen arrived at the specific lightingdisplay suggestions such as a pendant fixture rather a floor lampTheprogram could show alternative variations to make the designer aware ofnew possibilities rather than defaulting to a certain type of fixture

Our interest is in coupling a 3-D sketch interface with intelligentsystemsTherefore we built our own calculation and ray-tracing engine forSpot so that we could design and control the interaction as needed insteadof being limited by the behavior model of any existing simulation softwareThe experience in building Light Sketch [21] demonstrated the feasibilitythough trivial to connect with existing software such as Radiance In LightPen we built into Lux only a primitive model of lighting design expertiseHowever a more comprehensive version would interact with the Space Penin much the same way Rather than extend Luxrsquos lighting expertise we weremore concerned with demonstrating this system architecturersquos generalutilityWe therefore built a second instance of a 3-D sketch interface thistime to a simulation programThe ldquoSpotrdquo system also in the architecturallighting domain enables a designer to pose queries to a daylight simulatorby sketching on the surfaces of a building model Spot responds to thesequeries by displaying the time-varying lighting effects on the surfaces thatthe designer has indicated

These experiments have encouraged us to work toward a generalarchitecture to support 3-D interaction with intelligent systems of varioustypes Such an architecture might go beyond sketching and embrace a multi-modal approach including speech and gesture

Acknowledgements

Funding from the French Region Lorraine (to Seacutebastien Bund) providedsupport for our initial ventures into the development of Spot Light Penproject was supported in part by the National Science Foundation underGrant IIS 00-96138The views and findings contained in this material arethose of the authors and do not necessarily reflect the views of theNational Science FoundationWe thank the anonymous reviewers and co-editors of this issue (Celine Pinet and Wassim Jabi) from whose helpfulcomments the paper has benefited

References1 Anders G Daylighting Performance and Design 1995 New YorkVan Nostrand

Reinhold

2 Egan MD and VW Olgyay Architectural Lighting 2 ed 2001 McGraw Hill

3 Zeleznik RC KP Herndon and JF Hughes Sketch An Interface for Sketching 3DScenes SIGGRAPH lsquo96 1996 pp 163-170

226 Ellen Yi-Luen Do and Mark D Gross

IJAC 2-2_3 proof 16-7-04 346 pm Page 226

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227

Page 17: international journal of ar chitectural computing … · international journal of ar chitectural computing issue 02, volume 02 211 IJAC 2-2_3 proof 16-7-04 3:46 pm Page 211. Let Ther

4 IgarashiT S Matsuoka and HTanakaTeddy a sketching interface for 3Dfreeform design Proceedings of the SIGGRAPH 1999 annual conference on Computergraphics 1999 pp 409-416

5 Do EY-L Drawing MarksActs and Reacts toward a computational sketchingfor architectural design AIEDAM ndash Artificial Intelligence in Engineering DesignAnalysis and Manufacturing I Parmee and I Smith eds 2002 CambridgeUniversity Press Cambridge UK pp 149-171

6 IgarashiT and JF Hughes A Suggestive Interface for 3D Drawing UIST User InterfaceSoftware and Technology 2001ACM pp 173-181

7 Schweikardt E and MD Gross Digital Clay Deriving Digital Models fromFreehand Sketches Digital Design Studios Do Computers Make A Difference ACADIA98T Seebohm and SVWyk eds 1998 pp 202-211

8 TurnerA D Chapman and A Penn Sketching Space in Computers and Graphics2000 No 24 pp 869-876

9 StahovichTH R Davis and H Shrobe Generating Multiple New Designs from aSketch Proceedings of AAAI 1996 pp 1022-1029

10 Egenhofer M Spatial-Query-by-Sketch IEEE Symposium on Visual Languages 1996pp 60-67

11 Landay JA and BA Myers Sketching InterfacesToward More Human InterfaceDesign IEEE Computer 2001Vol 34 No 3 pp 56-64

12 Forbus K J Usher and V Chapman Sketching for military courses of actiondiagrams ACM Intelligent User Interfaces 2002 pp 61-68

13 Gross MD and EY-L Do Drawing on the Back of an Envelope in Computersand Graphics Calligraphy Interface JA Jorge and E Glinert eds 2000 PergamonPress New York pp 835-849

14 JungT EY-L Do and MD Gross From Redliner to Space Pen ACM IntelligentUser Interfaces 2002 pp 95-102

15 JungT MD Gross and EY-L Do Space Pen annotation and sketching on 3Dmodels on the Internet CAAD Futures 2001 BdVries JPv Leeuwen and HHAchten eds 2001 Kluwer Academic Publishers Eindhoven pp 257-270

16 Erwine B personal communication (April 30) 2002

17 Schoeneman C J Dorsey B Smits JArvo et al Painting with Light SIGGraph1993 pp 143-146

18 Glaser D and M Hearst Space SeriesA focus+context technique for displayingspatial and temporal data IEEE Symposium on Information Visualization lsquo99 LateBreaking Hot Topics 1999 San Francisco

19 RobertsA and A Marsh Ecotect Environmental Prediction in ArchitecturalEducation in eCAADe 2001 2001 342-347

20 Bund SA 3D environment for direct sunlight visualization DEA ndash ldquoModelisationet Simulation ees Espaces Batisrdquo Master Thesis in Centre de Recherche enArchitecture et Ingeacutenierie (CRAI Ecole drsquoArchitecture de Nancy France)conducted at the Design Machine Group (University of Washington USA) 2003Ecole drsquoArchitecture de Nancy Nancy

21 Glaser D JVoung L Xiao BTai et al LightSketchA sketchmodelling programfor lighting analysis CAAD Futures 2003 Kluwer 2003371-382

227Let There be Light Knowledge-Based 3-D Sketching Design Tools

Ellen Yi-Luen Do and Mark D Gross Design Machine Group University ofWashington 208 Gould Department of Architecture Box 355720 SeattleWA 98195-5720 USA

ellendo mdgacmorg

IJAC 2-2_3 proof 16-7-04 346 pm Page 227