activision publishing v. novalogic

7/27/2019 Activision Publishing v. Novalogic

1/57

123456789

101112131415161718192021222324252627

ACTIVISION PUBLISHING, INC.,Plaintiff,

v.NOVALOGIC, INC.,

Defendant.

KARIN G. PAGNANE LLI (SBN 174763)[email protected]. MAYER (SBN 190969)memgmsk.cornGILBERT S. LEE (SBN 267247)[email protected] SILBERBERG & KNUPP LLP11377 W est Olympic BoulevardLos Angeles CA 90064-1683Telephone: 310) 312-2000Facsimile: ( 10) 312-3100OMER SALIK (SBN 223056)omer.salikgactivision.comACTIVISION BLIZZARD, INC.3100 Ocean Park Blvd.Santa Monica, California 90405Telephone: (310) 255-2642Facsimile: (310) 255-22152Attorneys for Plaintiff Activision Publishing, Inc.

UNITED STATES DISTRICT COURTCENTRAL DISTRICT OF CALIFORNIA

COMPLAINT FOR PATENTINFRINGEMENTDemand For Jury Trial

Mitchell 28Knupp LLP

COMPLAINT FOR PATENT INFRINGEMENT

Case 2:13-cv-05091-GAF-SH Document 1 Filed 07/16/13 Page 1 of 57 Page ID #:7


2/57

123456789

101112131415161718192021222324252627

Mitchell 28Silberberg &Knupp LLP

Plaintiff Activision Publishing, Inc. ("Activision") alleges as follows:1. This is an action for patent infringement arising under the patent laws

of the United States, Title 35 of the United States Code.2. Plaintiff, Activision Publishing, Inc., is a corporation organ ized andexisting under the laws of the State of Delaware, having a principal place of

business at 3100 Ocean Park B lvd., Santa Monica, California.3. On information and belief, Defendant NovaL ogic, Inc.

("Nova' .ogic"), is a corporation organized and existing under the laws of the Stateof California, with a principal place of business in the city of Malibu, Los AngelesCounty, California.

4. On information and belief, NovaLogic has offered and sold, andcontinues to offer and sell, in this judicial district and elsewhere in the U nitedStates, video games entitled Delta Force; Delta Force 2; Delta Force: LandWarrior; Delta Force: Black Hawk Dow n; Delta Force: Black Hawk D own: TeamSaber; Delta Force: Extreme; Delta Force: Extreme 2; Joint Operations:Combined Arms Gold and Comanche 4.

JURISDICTION AND VEN UE5. This Court has subject matter jurisdiction pursuant to 28 U.S.C.

1331 and 1338(a).6. This Court has personal jurisdiction over NovaLogic because

NovaL ogic has comm itted acts of infringem ent in violation of 35 U.S.C. 271 andhas offered and provided to customers in California NovaLogic's Accused Games.Additionally, NovaL ogic regularly does business, solicits business, engages inother persistent courses of conduct, and/or derives substantial revenue fromproducts and/or services prov ided in individuals in this district and haspurposefully established substantial, systematic, and con tinuous contacts with thisdistrict and expects, or should reasonably expect, to be hauled into court here.

1COMPLAINT FOR PATENT INFRINGEMENT



3/57

7. Venue is proper in this district pursuant to 28 U.S.C. 1391 and1400(b).

FIRST CLAIM FOR RELIEF(Infringement Of U .S. Patent No. 6,014,145)

8. On January 11, 2000, United States Patent No. 6,014,145 ("the '145Patent") was duly and legally issued for an invention entitled "Navigation withoptimum v iewpoints in three-dimensional workspace interactive displays havingthree-dimensional objects with collision barriers." The '145 Patent has beenassigned to Activision, and A ctivision is the sole and exclusive owner of the '145Patent and holds all rights and interests in the '145 Patent. Among other things,Activision holds the sole and exclusive right to enforce the '145 Patent aga installeged infringers. A copy of the '145 P atent is attached hereto as Exhibit A.

9. NovaL ogic has infringed and continues to infringe one or more claimsof the '145 Patent by its m anufacture, use, sale, importation, licensing and/or offerfor sale of its video games en titled Delta Force; Delta Force 2; Delta Force: LandWarrior; Delta Force: Black Hawk Dow n; Delta Force: Black Hawk D own: TeamSaber; Delta Force: Extreme; Delta Force: Extreme 2; and Joint Operations:Combined Arms Gold, which em body one or m ore claims of the '145 Patent.NovaL ogic also has infringed and continues to infringe one or more claims of the'145 Patent by contributing to and actively inducing others to use, sell, importand/or offer for sale these games. NovaLogic is liable for its infringem ent of the'145 Patent pursuant to 35 U.S.C. 271.

10. NovaL ogic's acts of infringement have caused dam age to Activision,and Activision is entitled to recover from N ovaLogic the damages it has sustainedas a result of NovaL ogic's wrongful acts in an am ount subject to proof at trial.

123456789

101112131415161718192021222324252627

Mitchell 28Silberberg &Knupp LLP 2COMPLAINT FOR PATENT INFRINGEMENT



4/57

11. NovaL ogic's infringem ent of Activision's '145 patent will continue todamage Activision, causing irreparable harm for which there is no adequateremedy at law, unless enjoined by this Cou rt.

SECOND CLAIM FOR RELIEF(Infringement of U .S. Patent No. 5,883,628)

12. On M arch 16, 1999, United States Patent No. 5,883,628 ("the '628Patent") was duly and legally issued for an invention en titled "Climab ility:property for objects in 3-D virtual environments." The '628 Patent has beenassigned to A ctivision, and A ctivision is the sole and exclusive owner of the '628Patent and holds all rights and interests in the '628 Patent. Among other things,Activision holds the sole and ex clusive right to enforce the '628 Patent againstalleged infringers. A copy of the '628 Patent is attached hereto as Exhibit B.

13. NovaL ogic has infringed and continues to infringe one or more claimsof the '628 Patent by its manufacture, use, sale, importation, licensing and/or offerfor sale of its videogam es entitled Delta Force; Delta Force 2; Delta Force: LandWarrior; Delta Force: Black Hawk D own; Delta Force: Black Hawk D own: TeamSaber; Delta Force: Extreme; Delta Force: Extreme 2; and Joint Operations:Combined Arms Gold, which embody one or m ore claims of the '628 Patent.NovaL ogic also has infringed and continues to infringe one or more claims of the'628 Patent by contributing to and actively inducing others to use, sell, importand/or offer for sale these videogames. NovaLogic is liable for its infringement ofthe '628 Patent pursuant to 35 U.S.C. 271.14. NovaL ogic's acts of infringem ent have caused dam age to Activision,and Activision is entitled to recover from NovaLogic the damages it has sustainedas a result of NovaL ogic's wrongful acts in an am ount subject to proof at trial.

123456789

101112131415161718192021222324252627

Mitchell 28Silberberg &Knupp LLP 3




5/57

15. NovaL ogic's infringem ent of Activision's '628 patent will continue todamage Activision, causing irreparable harm for which there is no adequateremedy at law, unless enjoined by this Cou rt.

THIRD CLAIM FOR RELIEF(Infringement of U.S. Patent No. RE 38865)

16. On November 1, 2005, United States Patent No. RE38865 ("the '865Patent") was duly and legally issued for an invention entitled "System and m ethodfor optimizing computer software and hardware." The '865 Patent has beenassigned to Activision, and Ac tivision is the sole and exclusive owner o f the '865Patent and holds all rights and interests in the '865 Patent. Among other things,Activision holds the sole and exclusive right to enforce the '865 Patent againstalleged infringers. A copy of the '865 Patent is attached hereto as Exhibit C.

17. NovaT ,ogic has infringed and con tinues to infringe one or more claimsof the '865 Patent by its manufacture, use, sale, importation, licensing and/or offerfor sale of its video games entitled Comanche 4 and Delta Force: Black HawkDown, which embody one or more claims of the '865 Patent. NovaLogic also hasinfringed and continues to infringe one or more claims of the '865 Patent bycontributing to and actively inducing others to use, sell, import and/or offer for salethese games. NovaLogic is liable for its infringement of the '865 Patent pursuantto 35 U.S.C. 271.

18. NovaLogic's acts of infringement have caused damage to Activision,and Activision is entitled to recover from N ovaLogic the damages it has sustainedas a result of NovaL ogic's wrongful acts in an am ount subject to proof at trial.

19. NovaLogic's infringement of A ctivision's '865 patent will continue todamage Activision, causing irreparable harm for which there is no adequateremedy at law, unless enjoined by this Court.

123456789

101112131415161718192021222324252627

Mitchell 28Knupp LL P 4




6/57

DATED: July 16, 2013 OMER SALIKACTIVISION BLIZZARD, INC.KARIN G. PAGNANELLIMARC E. MAYERMITCHELL SILBERBERG & KNUPP LLP

_...sri7SBN 190969)Attorney. p Plaintiff

Activis t Publishing, Inc.

123456789

101112131415161718192021222324252627

PRAYER FOR RELIEFWH EREF ORE, A ctivision requests entry of judgment in its favor and

against the NovaLogic defendants as follows;a. Declaring that NovaL ogic has infringed U.S. Patent Nos. 6,014,145;

5,883,628; and RE38865;b. Awarding damages arising out of NovaLogic's infringement of U.S.

Patent Nos. 6,014,145; 5,883,628; and RE38865, including prejudgment and post-judgment interest, in an amount according to proof;

c. Permanently enjoining NovaL ogic and its respective officers, agents,employees, and those acting in privity with them, from further infringement,including contributory infringement and/or inducing infringement, of U.S. PatentNos. 6,014,145; 5,883,628; and R E38865;

d. Aw arding such other costs and further relief as the Court m ay deemjust and proper.

Mitchell 28Silberberg &Knupp LL P 5COMPLAINT FOR PATENT INFRINGEMENT



7/57

1

DATED: July 16, 2013

JURY DEMANDActivision requests a trial by jury.

OMER SALIKACTIVISION BLIZZARD, INC.KARIN G. PAGNANELLIMARC E. MAYERMITCHELL SILBERBERG & KNUPP LLP

23456789

101112131415161718192021222324252627

By:M c MSBN 190969)AttorneylaintiffActivision Publishing, Inc.

Mitchell 28Knupp LLP 6




8/57

EXHIBIT A



9/57

11111111111110111 11111111111 1t Ipt111111111011 1111111111111United States Patent [19] [11] Patent Number:,014,145Bardon et al. [45] Date of Patent:an. 11, 2000[54] NAVAGA TION WITH OPTIMUMVIEWPOINTS IN THREE-DIMENSIONALWORKSPACE INTERACTIVE DISPLAYSHAVING THREE-DIMENSIONAL OBJECTS

WITH COLLISION BARRIERS[75] Inventors: Didier Daniel Claude Bardon, A ustin;Richard Edmond B erry; Scott HarlanIsensee, both of Georgetown; ShirleyLynn Martin, Austin, all of Tex.[73] Assignee: International Business MachinesCorporation, Armonk, N.Y.[21] Appl. No.: 08/888,804[22] Filed:ul. 7, 1997[51] hit. Cl. 7 GO6F 3/00[52] U.S. Cl. 345/427; 345/474[58] Field of Search 345/427, 428,

345/429, 418, 419, 420, 474[56] References CitedU.S. PATENT DOCUMENTS4,578,757 3/1986 Stark 364/4615,047,916 9/1991 Kondo 364/167.015,056,031 10/1991 Nakano et al. 364/461

5,347,459 9/1994 Greenspan et al. 364/4615,515,489 5/1996 Yaeger 395/1525,528,735 6/1996 Strasnick et al. 345/4275,548,694 8/1996 Frisken Gibson 395/1245,561,742 10/1996 Terada et al. 395/90Primary ExaminerPhu K. NguyenAssistant ExaminerCliff N. VoAttorney, Agent, or FirmJerry Kraft; Volel Emile[57] ABSTRACTIn a data processor controlled display system having three-dimensional objects laid out in a three-dimensional virtualworkspace, the objects have associated there with respectivecollision barriers which are stored with their correspondingobjects. Then during navigation via changing the usersviewpoint in to the workspace, when this changing view-point encounters any collision barrier which is offset from itsobject by a selected distance D, the viewpoint stops. Thisdistance of the barrier from the object is selected so as tostop the viewpoint at a distance sufficiently removed from itsparticular object so as to leave the user or viewer with a viewof the workscape of sufficient scope to permit the viewer toselect a different navigation path or path branch. In accor-dance with a further aspect of the present invention, theviewpoint is optionally provided with a similar collisionbarrier.

23 Claims, 5 Drawing Sheets

EX. A PG.



10/57

paiedS9

NETWORKAPPLICATION

40

OPERATINGSYSTEM 41

12

RO M16

CP U10

RA M14

I/OADAPTER18

COMMUNICATIONSADAPTER 34

20

38 soTaS

FIG. 132

USER INTERFACEADAPTER 22DISPLAYADAPTER 36

FRAMEBUFFER 39

28



11/57

Jan. 11, 2000heet 2 of 56,014,145.S. Patent

FIG. 20 42EX. A PG



12/57

U.S. Patentan. 11, 2000heet 3 of 5 6,014,14550 42FIG. 3

EX. A PG.



13/57

VFOR EACE OBJECT CREATEA COLLISION BARRIER AT A

PRESELECTED DISTANCEFROM OBJECT PERIPERY

STORE COLLISION BARRIER INASSOCIATION WITH IT'S CORRESPONDING OBJECT

VPROVIDE AND STORE VIEWPOINTNAVIGATION MEANS

60

U.S. Patentan. 11, 2000heet 4 of 5 6,014 145I START61

62

59

FIG. 4AUSING SUPERSCAPE VRT

LAYOUT AND STOREVIRTUAL REALITY 3D WORKSPACE

CREATE AND STORE APLURALITY OF 3D

VIRTUAL OBJECTS

PROVIDE AND STORE VIEWPOINT COLLISIONBARRIER AT A DISTANCE FROM VIEWPOINT:

TO BE OPTIONALLY ACTIVATED

PROVIDE AND STORE MEANS FORINTERACTIVELY CHANGING VIEWPOINT

AND OBECT COLLISION BARRIER DISTANCES

EX. A PG. 1



14/57

63FIG. 4B RUN PROGRAM70

67NO NOES

NAVIGATE TO AVIEWPOINT

NAVIGATE TOA VIEWPOINT68

69YES

HASVIEWPOINT

REACHED THEOBJECT

BARRIER

HASOBJECT BARRIER

OUCH ED VIEWPOINBARRIER?

LAYOUT WORKSPACE OBJECTS ANDOBJECT COLLISION BARRIERS

72

STOP THE VIEWPOINT

CHANGE VIEWPOINTNAVIGATION PATH

U.S. Patentan. 11, 2000heet 5 of 5 6,014,145

EX. A PG. 1



15/57

6,014,1451

NAVAGATION WITH OPTIMUMVIEWPOINTS IN THREE-DIMENSIONALWORKSPACE INTERACTIVE DISPLAYSHAVING THREE-DIMENSIONAL OBJECTSWITH COLLISION BARRIERSCROSS-REFERENCE TO RELATEDAPPLICATION

Copending application "Climbability: A Property ForObjects in 3-D Virtual Environment", J. Mullaly et al.(Attorney Docket No. AT9-97-212) relates to objects withbarriers which viewpoints may climb over.TECHNICAL FIELD

The present invention relates to user interactive computersupported display technology and particularly to such userinteractive systems and methods which are user friendly, i.e.provide even non-computer-literate users with an interfaceenvironment which is easy to use and intuitive.BACKGROUND OF THE INVENTION

The 1990's decade has been marked by a societal tech-nological revolution driven by the convergence of the dataprocessing industry with the consumer electronics industry.This advance has been even further accelerated by theextensive consumer and business involvement in the internetover the past two years. As a result of these changes, it seemsas if virtually all aspects of human endeavor in the indus-trialized world requires human-computer interfaces. As aresult of these profound changes, there is a need to makecomputer directed activities accessible to a substantial por-tion of the world's population which, up to a year or two ago,was computer-illiterate, or at best computer indifferent. Inorder for the vast computer supported market places tocontinue and be commercially productive, it will be neces-sary for a large segment of computer indifferent consumersto be involved in computer interfaces. Thus, the challenge ofour technology is to create interfaces to computers w hich areas close to the real world as possible.

Industry has been working towards this challenge andthere is presently a relatively high degree of realism possiblein interfaces. This presents a need and an opportunity foreven more realistic interaction techniques to better match thevisual metaphors used and to achieve a higher level of easeof use for computer systems. We are striving towards therepresentation of object as photo realistic, three-dimensional(3D) models rather than as the icons and two-dimensionaldesktops of conventional computer technology.

Some examples of current technology for the creation ofvirtual three-dimensional workspace display interfaces arecopending applications Ser. No. 08/813,891 filed Mar. 7,1997, entitled "VIEWER INTERACIIVE OBJECT INVIRTUAL THREE-DIMENSIONAL WORKSPACE" andSer. No. 08/813,848 filed Mar. 7, 1997, entitled "VIEWERINTERACTIVE OBJECT WITH MULTIPLE SELECT-ABLE FACE VIEWS IN VIRTUAL THREE-DIMENSIONAL WORKSPACE", both assigned to theassignee of the present application.

A 3D virtual workspace display environment is alsodescribed in an article entitled, "RAPID CONTROLLEDMOVEMENT THROUGH A VIRTUAL 3DWORKSPACE", Jock Mackinlay et al., Computer GraphicsPublication, Vol. 24, No. 4, August 1990, pp. 171-175, aswell as in its related U.S. Pat. No. 5,276,785.

It is clear that current technology in virtual three-dimensional workspaces has provided environments which

2are user friendly, i.e. make the casual computer user feelmore comfortable and at home w ith the interface. However,researchers in human factors have found downsides tothree-dimensional virtual reality displays. Because of the

5 many choices that the user has in wandering down various"streets and roads" or visiting a wide variety of "buildingsor stores" or going through many possible "doors", the usermay wander through this reality and perhaps get lost fromthe track or goal he is pursuing.

10he present invention addresses this problem, i.e. that ofhelping the interactive user in three-dimensional graphicenvironments to stay focused and relate to the paths he isseeking to travel to in the man ner he is seeking to travel toreach objects even when these objects are arranged in 3D

15 space in what appears to be infinite configurations. Theinvention facilitates the user's navigation in the 3D space sothat the user may easily and quickly continue on his navi-gational path and stick to his navigational objectives. Theinvention aids the navigating viewer in relating to objects

20 which could be impediments to his continued navigation.SUMMARY OF THE INVENTION

It is understood that in order to navigate through three-25 dimensional space, view the space or relate to objects withinthe space, a viewpoint is determined w ithin that space. Thatviewpoint is the virtual position of the viewer or person w hois navigating within the three-dimensional space. The view-point is commonly defined by its position and its orientation30 or direction. For purposes of describing this invention, wewill use the metaphor of a camera to understand the view-point. The camera's position an d orientation are where it isand which way it is pointing. Let us refer to another propertyof a viewpoint which is "field of view"; this is effectively the

resulting view from a given viewpoint. A key need of a35 viewer navigating through virtual three-dimensional space isto stay focused and navigate as easily as possible and thuspermit this user to concentrate and focus on his plannedtasks rather than the naviga tion itself.

40he present invention provides a user aid in navigation byan arrangement which prevents the user viewpoint fromcolliding with objects or other three-dimensional impedi-ments in the workspace, but more particularly stops theviewpoint short of the object or impediment at a position45 such that the viewpoint does n ot "bring the user's nose rightup against the object or impediment." Rather, the viewpointis stopped at a distance such that the user still has a view ofreasonable scope from which he may select other paths orbranches in his navigation. In other words, the user's field of

50 view from the stopped viewpoint will be such that his furthernavigation will be facilitated. To this end, the presentinvention stores in association with each of a plurality ofvirtual three-dimensional objects, a collision barrier aroundthe periphery of the object at a distance from the object such

55 that any viewpo int stopped at the collision barrier will stillhave a reasonable field of view from which to select furthernavigational paths. Data defining such a collision barrier isstored in association with the data defining the respectiveobject. Then, during the course of navigation when any

o viewpoint intersects such a collision barrier of a particularobject the viewpoint, it is stopped. From the stopped view-point the user has the option or ability to reorient himself andselect alternative paths or navigation branches.In according to a more particular aspect of the present

ss invention, the user of the display system is provided withmeans for interactively changing these collision barriers ofparticular objects in order to more specifically relate the

EX. A PG. 1



16/57

6,014,145system to his selected interactive function. In systems wherethe interactive user has the ability to alter objects or thecontent of objects, it is most desirable to give such users thefurther ability of changing the collision barriers associatedwith such changed objects to better relate these two desiredinteractive functions. This ability to change the collisionbarrier interactively is particularly desirable where theobject is changing dynamically. In such a case, it is likely tobe desirable that collision barrier be changed dynamically inresponse to the dynamically changed object. For example, ifthe functionally interactive object is one conveying readableinformation, the collision barrier may be at one distancewhen the readable material is graphics and at anotherdistance perhaps closer when the readable material is text.

Collision boundaries or portions of them may be interac-tively turned off by the user to accommodate particularchanges, especially in a dynamic environment as describedabove. Such turned-off collision boundaries may of coursebe interactively turned on again to accommodate otherchanges.In accordance with another aspect of the presentinvention, the collision barrier may also be associated with

a viewpoint, i.e., the viewpoint may have a collision barrierso that when the collision barrier of the viewpoint intersectsthe collision barrier of an object or the periphery of an objectwhen the object has no collision barrier, then the viewpointstops at such an intersection. Like that of objects, thecollision barrier of a viewpoint may atso be changed inter-actively so that the viewpoint is stopped at a relativelyoptimum distance from the object from which navigationmay be continued.

In accordance with one aspect of the present invention,the objects may have preselected collision barriers depen-dant upon the properties of the objects as described above,and the viewpoint m ay have a collision barrier, the distanceof which from the viewpoint may be changed in order to finetune the stopping position of the viewpoint at a position mostdesirable for further navigation even in a changing work-space.

BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is a block diagram of a data processing system

including a central processing unit which is capable ofimplementing the present invention;FIG. 2 is a selected three-dimensional workspace filledwith a variety of objects;FIG. 3 is a diagrammatic representation of a top orplanned view of the workspace of FIG. 2 with the collisionbarriers surrounding objects and the viewpoint being shownas dashed lines; andFIGS. 4A and 4B are flowcharts of the process imple-mented by the present invention (FIG. 4A) and for navigat-ing through the three-dimensional workspace layout (FIG.4B).

DETAILED DESCRIPTION OF THEPREFERRED EMBODIMENTBefore going into the details of specific embodiments, itwill be helpful to understand from a more general perspec-tive the various elements and method which may b e used to

implement the present invention. The present invention isimplemented in three-dimensional virtual workspace. Athree-dimensional workspace is a workspace that is per-ceived as extending in three orthogonal directions. Typicallya display has a two-dimensional display surface and the

4perception of a third dimension is effected by visual cluessuch as perspective lines extending toward a vanishingpoint. Distant objects are obscured by nearer objects. Thethree-dimensional effect is also provided by showing

5 changes in objects as they move toward or away from theviewer. Perspective shading of objects and a variety ofshadowing of ob jects at different distances from the vieweralso contribute to the three-dimensional effect.

A three-dimensional workspace is typically perceived as10 being viewed from a position within the workspace. Thisposition is a viewpoint. This viewpoint provides the virtualinterface between the display user and the display. Theviewpoint's direction of orientation is the direction from theviewpoint into the field of view alo ng the axis at the center15 of the field of view.

In order to present a three-dimensional workspace, asystem may store data indicating "coordinates" of the posi-tion of an object, a viewpoint or other display feature in theworkspace. Data indicating coordinates of a d isplay feature20 can then be used in presenting the display feature so that itis perceptible as positioned at the indicated coordinates. The"distance" between two display features is the perceptibledistance between them and can be determined from theircoordinates if they are presented so that they appear to be

25 positioned at their coordinates. It is in this manner that thedistances of the collision barriers of this invention from theirobjects are established.Techniques for providing and hand ling three-dimensional

30 objects in a three-dimensional virtual workspace have beendeveloped in the art and are available to display userinterface designers. U.S. Pat. No. 5,276,785 (Mackinlay etal., Jan. 4, 1994) is an example of the design techniquesavailable to such three-dimensional workspace interfacedesigners.35 The three-dimensional workspace or landscape is navi-gable using conventional three-dimensional navigation tech-niques. A user may move around or navigate within thethree-dimensional data representation to alter his perspec-

40 tive and view of the displayed representation of the data.Thus, a user may be referred to as a navigator. The navigatoris actually stationary, and his view of the display spacechanges to give him the sensation of moving within thethree-dimensional graphical space. Thus, we speak in terms

45 of the navigator's perceived motion when we refer tochanges in his view of the display space. As the user moves,his view of the data changes accordingly within the three-dimensional data representation. Some navigation modesinclude browsing, searching and data movement. U.S. Pat.

50 No. 5,555,354 (Strasnick et al., Sep. 10, 1996) describessome known navigation techniques.The programs are written in human readable script andthis script is provided to another program called a co mpilerto generate a machine readable numeric code wh ich can be

55 loaded into, and directly executed by the computer. The C++language possesses certain characteristics which allow asoftware developer to easily use programs w ritten by otherswhile still providing a great deal of control over the reuse ofprograms to prevent their destruction or improper use. The

60 C++ language is well known and m any articles and text areavailable which describe the language in detail.While the embodiment of the present invention, whichwill be subsequently described, can be implemented usingobject oriented techniques involving the C++ programming

65 language, we found it preferable to use SCL as used in VRT:the Virtual Reality Toolkit developed and marketed bySuperscape Ltd. having U.S. offices in Palo Alto, California.

EX. A PG. 1



17/57

6,014,1455

Extensive details of these programming techniques may befound in the Superscape VRT, Reference Manual, Version4-00, 2d Edition, Jan. 29, 1996.

It should be understood by those skilled in the art thatobject oriented programming techniques involve thedefinition, creation, use and instruction of "ob jects". Theseobjects are software entities comprising data elements androutines, or methods, which manipulate the data elements.The data and related methods are treated by the software asan entity and can be created, used and deleted as such. Thedata and functions enable objects to model their real worldequivalent entity in terms of its attributes, which can bepresented by the data elements, and its behavior which canbe represented by its methods. Thus, in implementing thepresent invention, the collision barrier associated withobjects may be regarded and stored as part of the softwareentity.

Objects are defined by creating "classes" which are notobjects themselves, but which act as templates whichinstruct a compiler how to construct the actual object. Forexample, a class may specify the number and type of datavariables and the steps involved in the functions whichmanipulate the data. An object is actually created in theprogram by mean s of a special function called a constructorwhich uses the corresponding class definition and additionalinformation, such as arguments provided during objectcreation, to construct the object. Objects are destroyed by aspecial function called a destructor.

Many benefits arise out of three basic properties of objectoriented programming techniques, encapsulation, polymor-phism and inheritance. Objects can be designed to hide, orencapsulate, all or a portion of, the internal data structureand the internal functions. More particularly, during pro-gram design, a program developer can define objects inwhich all or some of the data variables and all or some of therelated method are considered "private" or for use only bythe object itself. Other data or methods can be declared"public" or available for use by other software programs.Access to the private variables and methods by other pro-grams can be controlled by defining public methods whichaccess the object's private data. The public methods form aninterface between the private data and external programs. Anattempt to write program code which directly accesses theprivate variables causes a compiler to generate an errorduring program compilation. This error stops the compila-tion process and presents the program from being run.

Polymorphism allows objects and functions which havethe same overall format, but which work w ith different data,to function differently to produce consistent results. Forexample, an addition method may be defined as variableA+variable B, (A+B). The same format can be used whetherthe A and B are numbers, characters or dollars and cents.However, the actual program code which performs theaddition may differ widely depending on the type of vari-ables which comprise A and B. Thus, each type of variable(numbers, characters and dollars). After the methods havebeen defined, a program can later refer to the additionmethod by its common format (A+B) and, duringcompilation, the compiler will determine which o f the threemethods to be used by examining the variable types. Thecompiler will then substitute the proper function code.

A third property of object oriented programm ing is inher-itance which allows program developers to reuse pre-existing programs. Inheritance allows a software developerto define classes and the o bjects which are later created fromthem as related through a class hierarchy. Specifically,

6classes may be designated as subclasses of other baseclasses. A subclass "inherits" and has access to all of thepublic functions of its base classes as though these functionsappeared in the subclass. Alternatively, a subclass can over-

5 ride some or all of its inherited functions or may modifysome or all of its inherited functions by defining a newfunction with the same form.The creation of a new subclass borrowing the function-ality of another class allows software developers to easily

10 customize existing code to meet their particular needs.Although object oriented programming o ffers significantimprovements over other programming concepts, program

development still requires significant outlays of time andeffort, especially if no pre-existing software programs are

15 available for modification. Consequently, a set ofpredefined, interconnected classes are sometimes providedto create a set of objects and additional miscellaneousroutines which are all directed to performing commonlyencountered tasks in a particular environment. Such pre-

20 defined classes and libraries are typically called "frame-works" and essentially provide a prefabricated structure as abasis for creating a working application program.In object oriented programming such as the previouslydescribed VRT software platform, there is provided for the25 user interface a framework containing a set of predefinedinterface objects. The framework contains predefined

classes which can be used as base classes and a developermay accept and incorporate some of the objects into thesebase classes, or he may modify or override objects or30 combinations of objects in these base classes to extend theframework and create customized solutions in particularareas of expertise.

This object oriented approach provides a m ajor advantageover traditional programming since the programmer is notchanging the original program, but rather extending thecapabilities of the original program.The above-described Superscape Virtual Reality Toolkit

(VR1) provides the architectural guidance and modeling,40 but at the same time frees developers to supply specificactions unique to the particular problem domain which thedeveloper is addressing. Those skilled in the art will under-stand how the present invention is implemented using objectoriented programming techniques as described above.45 With this background of the various expedients which

may be used to implement the present invention, the pre-ferred embodiments will now be described.Referring to FIG. 1, a typical data processing system isshown which may be used in conjunction with objectso oriented software in implementing the present invention. Acentral processing unit (CPU), such as one of the PowerPCmicroprocessors available from International BusinessMachines Corporation (Powe rPC is a trademark of Interna-tional Business Machines Corporation) is provided and

55 interconnected to various other components by system bus12. An operating system 41 runs on CPU 10 and providescontrol and is used to coordinate the fun ction of the variouscomponents of FIG. 1. Operating system 41 may be one ofthe commercially available operating systems such as DOS,

60 or the OS/2 operating system available from InternationalBusiness Machines Corporation (0S/2 is a trademark ofInternational Business Machines Corporation). A programapplication such as the program in the above-mentionedVRT platform 40 runs in conjunction with operating system

65 41 and provides output calls to the operating system 41which implements the various functions to be performed bythe application 40.

EX. A PG. 1



18/57

6,014,1457

A read only memory (ROM) 16 is connected to CPU 10,via bus 12 and includes the basic input/output system(BIOS) that controls the basic computer functions. Randomaccess memory (RAM) 14, I/O adapter 18 and communi-cations adapter 34 are also interconnected to system bus 12 .It should be noted that software components including theoperating system 41 and application 40 are loaded into RAM14 which is the computer system's main memory. I/Oadapter 18 may be a small computer system interface (SCSI)adapter that communicates with the disk storage device 20,i.e. a hard drive. Communications adapter 34 interconnectsbus 12 with an outside network enabling the data processingsystem to commun icate with other such systems over a localarea network (LAN), wide area network (WAN), or the like.I/O devices are also connected to system bus 12 via userinterface adapter 22 and display adapter 36. Keyboard 24,trackball 32, mouse 26 and speaker 28 are all interconnectedto bus 12 through user interface adapter 22. Display adapter36 includes a frame buffer 39 which is a storage device thatholds a representation of each pixel on the display screen 38.Images may be stored in frame buffer 39 for display onmonitor 38 through various components such as a digital toanalog converter (not shown) and the like. By using theaforementioned I/O devices, a user is capable of inputtinginformation to the system through the keyboard 24, trackball32 or mouse 26 and receiving output information from thesystem via speaker 28 and display 38.

There will now be described a simple illustration of thepresent invention. When the im ages are described, it will beunderstood that these may be rendered by storing a virtualreality three-dimensional image creation program such asthe previously described 'VRT of Superscape in the RAM 14of the system of FIG. 1. Also stored on the RAM will be asuitable operating system such as DOS or Windows. Theoperating system of the VRT application is diagrammati-cally shown in FIG. 1 as operating system 41 in whichapplication 40 operates.

An embodiment of the present invention will now bedescribed starting with respect to the task specific portion ofa virtual reality three-dimensional workspace show n in FIG.2. Workspace 42 has many objects, among them: book-shelves 43, desk 44, podium 45, window wall 46, painting47, shelves 48 and 49. Let us consider the image in FIG. 2a view point of a three-dimensional space presented to theviewer at a display interface such as that which could beshown on display monitor 38 of FIG. 1. In accordance withconventional techniques, the user may control the viewpoint50 through conventional I/O devices such as mouse 26 inFIG. 1 which operates through user interface 22 to call uponthe VRT programs in RAM 14 cooperating with the oper-ating system 41 to create the images in frame buffer 39 ofdisplay adapter 36 to control the display on monitor 38.Using conventional virtual three-dimensional workspacenavigation techniques, the viewpoint interface of the usershown in FIG. 2 is changeable as the user moves closer orbacks away from objects in the workspace or moves to theright or left in the workspace. All this may b e controlled bya suitable I/O device such as mouse 26 of FIG. 1.

While for simplicity of illustration, the view of theworkspace shown in FIG . 3 which is a top view of the spaceshown in three dim ensions in FIG. 2 is presented in a singleplane in FIG. 3, it should be understood that the movementmay be along any path in the three orthogonal: x y and zdirections.

With reference to FIG. 3, the viewpoint 50 which is thepoint from which the field of view shown in FIG. 2 is viewedby the user, is shown as crosspoint 50. The workplane 42 of

8FIG. 3 has all of the objects in FIG. 2 presented in a plan o roverhead view: bookcase 43, desk 44, podium 45, windowwall 46, painting 47, shelves 48 and bookcase 49. Thecollision barrier associated with each of these objects is

5 shown as a continuous dashed line 51 spaced from theseobjects by a distance d which varies with the nature of theobject. Thus the section 52 of collision barrier 51 which isclose to desk 44 is at a relatively short distance from theperiphery of the desk 44 . The reason for this is that desk 44

10 as shown in FIG. 2 is a low lying impediment. Thus, whenthe viewpoint is stopped short of the ob ject by the collisionbarrier, the viewpoint will be such that the user can easilyselect the navigational path or branch since he will be ableto see over the low lying object. On the other hand, the

5 section 53 of the collision barrier 51 which is near bookcase43 is at a relatively substantial distance from the bookcase.As seen from FIG. 2 bookcase 43 is quite tall and the viewerat the stopped viewpoint adjacent to bookcase 43 must havea viewpoint that is distanced so that he can see around the20 bookcase in order to decide the most appropriate naviga-tional path. Similarly, section 54 of collision barrier 51 nearpainting 47 is also at a substantial distance from it's paintingobject 47. This is because the painting in combination with

shelves 48 is a formidable and tall barrier from which the25 viewer has to be distanced in order to best select an alternatenavigational path. In addition, the distance at section 54provides the viewer or user with an optimum distance from

which he may if he desires study painting 47.As set forth hereinabove, in accordance w ith an additional

30 aspect of the present invention the viewpoint 50 may havean optional viewpoint collision barrier which is depicted intop view of FIG. 3 as dashed line 55. Thus if collision barrier55 is optionally turned on, then the viewpoint will stop whenviewpoint collision barrier 55 touches object collision bar-35 rier 53 at any point. On the other hand if the viewpointcollision barrier 55 is not turned on, then the viewpoint 50will stop when it touches object collision barrier 51.

It should be understood that both the object collisionbarrier 51 and viewpoint collision barrier 55 if the later is

40 optionally rendered operational, may both be setup so as tobe interactively changeable by the user i.e., user may inter-actively change distance d or change the distance fromcollision barrier 55 to the viewpoint. This is usually desir-able in systems where the user may interactively alter his45 objects or their positions. In such a ca se, the user may needthe ability to change object collision barriers to accommo-date changes in potential navigation views brought about bysuch changes in the objects. Even further, there may besystems in which the system itself under certain circum-

50 stances may automatically change objects. In such cases, thesystem may be setup so as to automatically also alter thecollision barriers to accommodate the change in potentialnavigational views resulting in such object changes.Now w ith reference to FIGS. 4A and 4B , we will describe

55 a process implemented by the present invention in conjunc-tion with the flowcharts of FIGS. 4A and 4B. First, the stepsin FIG. 4A relate to the development of the virtual realityworkscape, the functional object and their positioning in theworkscape in accordance with the present invention using

60 the previously described Superscape VRT programmingtoolkit. It also includes the development for each object ofa collision barrier at a preselected distance from the objectperiphery. First, step 56, the desired virtual reality three-dimensional workspace is created and stored. This would be,

65 for example, the workspace shown in FIG. 2 and diagram-matically shown as a plan view of FIG. 3. N ext, step 57, thevirtual reality three-dimensional objects which will be

EX. A PG. 1



19/57

6,014,1459

placed in a landscape are created a nd stored. Then, step 58,a collision barrier is selected for each object at a preselecteddistance from the object periphery. Then, step 59, suchcollision barriers are stored in association w ith their respec-tive corresponding objects.

Next, step 60, there is provided and stored navigationmeans for changing the viewpoint of the viewer at thedisplay interface responsive to the viewer interactivelyselecting appropriate navigation paths. These navigationmeans can be any con ventional navigation means either for 10proceeding along a particular path. Next, step 61, a view-point collision barrier is created and stored. This barrier willbe a preselected distance from the viewpoint and willnormally be inactive but may be optionally activated. Then,step 62, means are provided and stored for interactively 15changing both the object and viewpoint collision barrierdistances.

The process now proceed s to point A in FIG. 4B whereatthe created virtual reality landscape program is run, step 63.As previously mentioned, the program is run on a system 20such as that shown in FIG. 1 with particular applicationprograms being loaded on RAM 14, connected to displaybuffer 36 which forms the stored images via frame buffer 39controlling the display monitor 38. The program initiallysets up the workspace layout on the display, step 64; this 25includes laying out the objects as w ell as their correspondingobject collision barriers. Now, before there is any navigationby means of moving a viewpoint a determination is made,decision step 65 as to whether the viewpoint collision barrieris active. Let us first assume that the collision barrier around 30the viewpoint is not active, resulting in a no from step 65;the system then begins to navigate through a sequence ofviewpoints, step 66. At each sequential viewpoint, a decisionor determination is made, step 67 as to whether the view-point has reached an object collision barrier. If it has not, 35then the system returns to navigation step 66 and thenavigation through a sequence of viewpoints along a par-ticular path continues. On the other hand, if the decisionfrom step 67 is that an object collision barrier has been

1 0We claim:1. A data processor controlled display system for display-ing a virtual three-dimensional workspace comprising:means for displaying a plurality of virtual three-dimensional objects within said workspace;user interactive navigation means for moving users to a

plurality of viewpoints within said workspace;means for storing in association with each of a plurality ofsaid virtual objects, a collision barrier around theperiphery of the object at a selected distance from theobject, andmeans responsive to moving a user to a viewpoint inter-secting the collision barrier of a selected object forstopping the viewpoint at said barrier.2. The display system of claim 1 further including user-interactive means for changing the distance of a collisionbarrier from its associated object.3. The display system of claim 1 further including meansfor changing at least one of said plurality of virtual threedimensional objects, and means responsive to said changefor changing the distance of the collision barrier of saidchanged object from said changed object.4. The display system of claim 3 wherein said means forchanging said object, dynamically change said object.5. The display system of claim 1 further including

viewpoint collision barrier means tracking each of saidviewpoints during navigation at a selected distancefrom said viewpoint, andmeans, responsive to moving a user to a viewpointwhereat the viewpoint collision barrier intersects the

collision barrier of an object, for stopping said view-point.6. The display system of claim 5, further including means,responsive to moving a user to a viewpoint whereat theviewpoint collision barrier intersects the periphery of anobject without a collision barrier, for stopping said view-point.7. A computer implemented method for displaying avirtual three-dimensional workspace comprising:

displaying a plurality of virtual three-dimensional objectswithin said workspace;user-interactively moving users to a plurality of view-points within said workspace;storing in association with each of a plurality of saidvirtual objects, a collision barrier around the peripheryof the object at a selected distance from the object, andresponsive to moving a user to a viewpoint intersectingthe collision barrier of a selected object, stopping theviewpoint at said barrier.8. The method of claim 7 further including the step ofchanging the distance of a collision barrier from its associ-ated object.9. The method of claim 7 further includingthe step of changing at least one of said plurality of virtualthree dimensional objects, andresponsive to said change, the step of changing thedistance of the collision barrier of said changed objectfrom said changed object.10 . The method of claim 9 w herein said object is dynami-cally changed.11 . The method of claim 7 further including the steps of:providing a viewpoint collision barrier tracking each ofsaid viewpoints during navigation at a selected distancefrom said viewpoint, andresponsive to moving a user to a viewpoint whereat theviewpoint collision barrier intersects the collision bar-rier of an object, stopping said viewpoint.

reached, the viewpoint stops, step 70. 40Considering now the flow of the process from decisionstep 65 if it is determined that the viewpoint has its optionalcollision barrier active, then the system proce eds to step 68where the viewpoint is navigated and a determination is 45made, step 69, as to whether the ob ject collision barrier hastouched or intersected the viewpoint collision barrier. If ithas not, then the system returns to step 68 and the navigationcontinues. On the other hand if a determination is made indecision step 69 that the object collision barrier has inter- sosected the viewpoint collision barrier, then the viewpoint isstopped again, step 70.At this point, decision step 71, a determination is made asto whether the user wishes to end the session. Such decisionsmay be m ade by the user all along in the p rocess. If it is the 55

end of the session, yes from step 71, then the session isended step 73. On the other hand, if the session is tocontinue, then step 72, the user is now stopped at a pointwhere he m ay conveniently consider his options in selectinga changed navigation path from the stopped viewpoint. After 60he changes the na vigation path, the flow returns to decisionstep 65 from this new navigation path, and the processcontinues as described above.

Although certain preferred embodiments have beenshown and described, it will be understood that many 65changes and modifications may be made therein withoutdeparting from the scope and intent of the appended claims.

EX. A PG. 1



20/57

6,014,14511

12 . A computer program having data structures includedon a computer readable medium which causes the display ona data processor supported display of a virtual three-dimensional workspace comprising:means for displaying a plurality of virtual three-dimensional objects within said workspace;user interactive navigation means for moving users to aplurality of viewpoints within said workspace;means for storing in association with each of a plurality ofsaid virtual objects, a collision barrier around theperiphery of the object at a selected distance from theobject, andmeans responsive to moving a user to a viewpoint inter-

secting the collision barrier of a selected object forstopping the viewpoint at said barrier.13 . The computer program according to claim 12 furtherincluding user-interactive means for changing the distanceof a collision barrier from its associated object.14 . The computer program according to claim 12 furtherincluding means for changing at least one of said plurality ofvirtual three dimensional objects, and means responsive tosaid change for changing the distance of the co llision barrierof said changed object from said changed object.15 . The computer according to claim 14 wherein said 25means for changing said object, dynamically change saidobject.16 . The computer program according to claim 12 furtherincluding

viewpoint collision barrier means tracking each of said 30viewpoints during navigation at a selected distancefrom said viewpoint, and

means, responsive to moving a user to a viewpointwhereat the viewpoint collision barrier intersects thecollision barrier of an object, for stopping said view- 35point.17 . The computer program according to claim 16, furtherincluding means, responsive to moving a user to a viewpointwhereat the viewpo int collision barrier intersects the periph-

ery of an object without a collision barrier, for stopping said 40viewpoint.18 . A data processor controlled display system for dis-playing a virtual three-dimensional workspace comprising:means for displaying a plurality of virtual three-dimensional objects within said workspace;

12user interactive navigation means for moving users to aplurality of viewpoints within said workspace;viewpoint collision barrier means tracking each of saidviewpoints during navigation at a selected distancefrom said viewpoint, andmeans, responsive to moving a user to a viewpoint

whereat the viewpoint collision barrier intersects theperiphery of an object, for stopping said viewpoint.19 . The display system of claim 18, further includingmeans changing the distance of said collision barrier fromsaid viewpoint.20 . A computer implemented method for displaying avirtual three-dimensional workspace comprising:displaying a plurality of virtual three-dimensional objectswithin said workspace;user-interactively moving users to a plurality of view-points within said workspace;providing a viewpoint collision barrier tracking each ofsaid viewpoints during navigation at a selected distancefrom said viewpoint, andresponsive to moving a user to a viewpoint whereat theviewpoint collision barrier intersects the periphery ofan object, stopping said viewpoint.21 . The method of claim 20 further including the step ofchanging the distance of said collision barrier from saidviewpoint.22 . A computer program having data structures includedon a computer readable medium which causes the display ona data processor supported display of a virtual three-dimensional workspace comprising:means for displaying a plurality of virtual three-dimensional objects within said workspace;user interactive navigation means for moving users to aplurality of viewpoints within said workspace;viewpoint collision barrier means tracking each of said

viewpoints during navigation at a selected distancefrom said viewpoint, andmeans, responsive to moving a user to a viewpoint

whereat the viewpoint collision barrier intersects theperiphery of an object, for stopping said viewpoint.23 . The computer program acco rding to claim 22, furtherincluding means changing the distance of said collisionbarrier from said viewpoint.

5

10

15

20

EX. A PG. 1



21/57

EXHIBIT B



22/57

1111 11111111111111111111US005883628AUnited States Patent [19] [11] Patent Number:,883,628MuItaly et al. [45] Date of Patent:ar. 16, 1999[54] CLIMABILITY: PROPERTY FOR OBJECTSIN 3-D VIRTUAL ENVIRONMENTS[75] Inventors: John Martin M ullaly; Didier DanielClaude Bardon, both of Austin;Richard Edmond Berry; Scott HarlanIsensee, both of Georgetown; ShirleyLynn Martin, Austin, all of Tex.

[73] Assignee: International Business MachinesCorporation, Armonk, N.Y.[21] Appl. No.: 887,806[22] Filed:ul. 3, 1997[51] hit. CI.6 GO6F 3/00[52] U.S. Cl. 345/355[58] Field of Search 345/355,357[56] References Cited

U.S. PATENT DOCUMENTS5,502,638 3/1996 Takenaka 364/424.025,577,981 11/1996 Jarvik 482/45,584,700 12/1996 Feldman et al. 482/2475,594,644 1/1997 Hasegawa et al.364/424.0275,737,533 4/1998 de Henri 395/357 X

OTHER PUBLICATIONSSpinney, L., "Seeing is unfreezing," New Scientist V. 153,Feb. 1997, p. 38-40.Jacobson, J., "Collision avoidance in virtual environments,"1997 IEEE Int. Conf. on SMC pp. 1704-1709.Primary ExaminerA. KatbabAttorney, Agent, or FirmPaul Kraft[57] ABSTRACTThe problem addressed by this invention is that of resolvingwhat occurs when a user's viewpoint comes into contactwith an object residing in a three-dimensional workspace asthe viewer navigates through that workspace. There areseveral possibilities in such a situation. For example, theviewer's viewpoint may be stopped and prevented fromadvancing further or it may penetrate the object. The instantinvention provides a climbability property for objects in 3-Dvirtual environments. This property, when stored in thesoftware along with the properties which define that object,can alter the viewer's viewpoint such that it will automati-cally be adjusted to simulate the behavior of climbing overthe object.

17 Claims, 9 Drawing Sheets

EX. B PG. 1



23/57

DISPLAYADAPTER 6FRAMEBUFFER 39

FIG. 1

6OIWEIS

32

28

USER INTERFACEADAPTER 2224

APPLICATION 40

lualed'S'l

20 NETWORK

12

CPU10

RA M14

I/0ADAPTER18

COMMUNICATIONSADAPTER 34ROM

16

OPERATINGSYSTEM 41



24/57

U.S. Patentar. 16, 1999heet 2 of 95,883,628_

EX. B PG. 2



25/57

U.S. Patentar. 16, 1999heet 3 of 9 5,883,628

EX. B PG. 2



26/57

U.S. Patentar. 16, 1999heet 4 of 9 5,883,6280CN

0CN

EX. B PG. 2



27/57

Th0(N

U.S. Patentar. 16, 1999heet 5 of 9 5,883,628TI

0

EX. B PG.



28/57

U.S. Patentat 16, 1999heet 6 of 9 5 9883 ,628

Ex. B PG.



29/57

U.S. Patentar. 16, 1999heet 7 of 9 5,883,628

EX. B PG.



30/57

U.S. Patentar. 16, 1999heet 8 of 9 5,883,628lo0CO

EX. B PG. 2



31/57

[ START051

53

VIEWPOINTCHANGE?

PROCESSOTHERINPUT

68NO

GO TO

CHECK COLLISION PROPERTYOF OBJECT

ADJUST VIEWPOINTTO CLIMB OVER

OBJECTADJUST VIEWPOINTAS PER OTHER

SETTINGOTHER SETTING

GO TOA

WAIT FOR INPUT

VIEWPOINTCOLLIDES WITH

OBJECT?

READ INITIAL SORTED DATA FORENVIRONMENT, OBJECTS, AND VIEWPOINT

DISPLAY VIEW OF ENVIRONMENT WITHOBJECTS FROM PERSPECTIVE OF THE VIEWPOINT

-- 52

U.S. Patentar. 16, 1999heet 9 of 95,883,628

EX. B PG. 2



32/57

5,883,6281 2CLIMABILITY: PROPERTY FOR OBJECTSIN 3-D VIRTUAL ENVIRONMENTS

TECHNICAL FIELDThe present invention relates to user interactive computersupported display technology and particularly to such user

interactive systems and methods which are user friendly, i.e.provide even noncomputer literate users with an interfaceenvironment which is easy to use and intuitive.BACKGROUND OF THE INVENTION ANDPRIOR ART

The 1990's decade has been marked by a societal tech-nological revolution driven by the convergence of the dataprocessing industry with the consumer electronics industry.This advance has been even further accelerated by theextensive consumer and business involvement in the internetover the past two years. As a result of these changes, it seemsas if virtually all aspects of human endeavor in the indus-trialized world requires human-computer interfaces. As aresult of these profound changes, there is a need to makecomputer directed activities accessible to a substantial por-tion of the world's population which, up to a year or two ago,was computer-illiterate, or at best computer indifferent. Inorder for the vast computer supported market places tocontinue and be commercially productive, it will be neces-sary for a large segment of computer indifferent consumersto be involved in computer interfaces. Thus, the challenge ofour technology is to create interfaces to computers w hich areas close to the real world as possible.

Industry has been working towards this challenge andthere is presently a relatively high degree of realism possiblein interfaces. This presents a need and an opportunity foreven more realistic interaction techniques to better match thevisual metaphors used and to achieve a higher level of easeof use for computer systems. We are striving towards therepresentation of object as photo realistic, three-dimensional(3D) models rather than as the icons and two-dimensionaldesktops of conventional computer technology.Some examples of current technology for the creation ofvirtual three-dimensional workspace display interfaces arecopending application Ser. No. 08/753,081, entitled "CRE-ATING REAL WORLD OBJECTS" and Ser. No. 08/753,076, entitled "SYSTEM AND METHOD FOR MAINTAIN-ING SIZE AND POSITION RELATIONSHIPS FORNONRECTANGULAR REAL WORLD OBJECTS",assigned to the Assignee of the present application.

A 3D virtual workspace display environment is alsodescribed in an article entitled, "RAPID CONTROLLEDMOVEMENT THROUGH A VIRTUAL 3DWORKSPACE", Jock Mackinlay et al., Computer GraphicsPublication, Vol. 24, No. 4, August 1990, pp. 171-175, aswell as in its related U.S. Pat. No. 5,276,785.

A more recent copending application assigned to theAssignee of the present invention is Ser. No. 08/813,891,entitled, "VIEWER INTERACTIVE OBJECT IN VIR-TUAL THREE-DIMENSIONAL WO RKSPACE", D. Bar-don et al., which covers face views of virtual three-dimensional objects which may be triggered to appear on adisplay by interactive viewer input.

It is clear that current technology in virtual three-dimensional workspaces has provided environments whichare user friendly, i.e. make the casual computer user feelmore comfortable and at hom e with the interface. However,researchers in human factors have found downsides to

three-dimensional virtual reality displays. Because of themany choices that the user has in wandering down various"streets and roads" or visiting a wide variety of "buildingsor stores" or going through many possible "doors", the user5 may wander through this reality and perhaps get lost fromthe track or goal he is pursuing.

The present invention addresses this problem, i.e. that ofhelping the interactive user in three-dimensional graphicenvironments to stay focused and relate to the objects he is

10 seeking to relate to in the mann er he is seeking to relate tosuch objects even when these objects are arranged in 3Dspace in what appears to be infinite configurations. Thisinvention also facilitates the design of 3-D environmentswhich then facilitate the tasks of the end users.

15n these virtual reality 3D environments as in the realworld, the viewer or user is relating to the virtual objects inorder to carry out a wide variety of tasks, some of w hich arequite simple and some very complex. In order for the userto stay focused and carry out his tasks as expeditiously aszo possible, it would be optimum for the virtual 3D system toprovide simpler user interfaces for simple tasks and morecomprehensive user interfaces for m ore complex tasks.

Thus, when the viewer's task is a simple one such asgetting more information about a current movie film or25 about a newly released music CD, the user may be presentedwith his information in an interface as simple as a face viewof a virtual 3D object which contains the information. Forexample, in the virtual 3D world or workspace, the viewermay navigate to a virtual three-dimensional object of a

30 theater and get his desired movie film information from aface view of the object which presents a marquee of thetheater. Similarly, the viewer seeking CD information mightnavigate to and be presented with a face view of a virtual CDvending kiosk which presents him with his desired infor-

35 mation. The above-mentioned patent application, "VIEWERINTERACTIVE OBJECT IN VIRTUAL THREE-DIMENSIONAL WORKSPACE", D. B. Bardon et al.,describes such face views of 3D virtual objects. With suchsimple tasks, the viewer notes his desired information,

40 perhaps makes some simple choices and moves on with hisnavigation through the virtual 3D workspace.On the other hand, the navigating viewer's task may be amore complex one like tracking and upd ating product salesinformation of a business or group of b usinesses or within a

45 report or filing a tax statement.SUMMARY OF THE INVENTION

Before setting forth the present invention, we shouldestablish some basic characteristics of the virtual three-50 dimensional environment as described in the above-referenced patents and applications. It is understood that inorder to navigate through three-dimensional space, view the

space or relate to objects within the space, a viewpoint isdetermined within that space. That viewpoint is the virtual55 position of the viewer or person who is navigating within thethree-dimensional space. The viewpoint is commonly

defined by its position and its orientation or direction. Forpurposes of describing this invention, we will use themetaphor of a camera to understand the viewpoint. The

60 camera's position and orientation are where it is and whichway it is pointing. Let us refer to another property of aviewpoint which is "field of view"; this is effectively theresulting view from a given viewpoint. A key need of aviewer navigating through virtual three-dimensional space isGs to stay focused.

As set forth above, it is easier to stay focused when thetask for which he is accessing the object is a simple one. The

EX. B PG.



33/57

5,883,628

5

25

3present invention deals with helping viewers to stay focusedin more complex tasks.

The present invention operates within the previouslydescribed data processor controlled display system for dis-playing a virtual three-dimensional object having three-dimensional objects which are interactively functional, i.e.may be picked by the viewer or user for various computerinteractive functions.

The present invention specifically provides the designerand/or user with a mean s of dealing with a situation whereinthe viewer's viewpoint is confronted with an object in a 3-Dvirtual reality environment as they navigate through 3-Dspace. How do they deal with such a situation. This inven-tion provides a mechanism for resolving this dilemma.

In three dimensional (3D) graphical environments,objects are arranged in three dimensional space and the usersmove through the space by moving their viewpoint withinthat space. Movement within 3D space which containsobjects is an aspect human-computer interaction whichpresents many problems or challenges to the usability ofsoftware with 3D interfaces.

The specific problem addressed by this invention is that ofresolving what occurs when a user's viewpoint and anyembodiment attached to the viewpoint in the virtual space,such as a came ra, vehicle or avatar, comes into contact withany of the objects residing in a 3-D environment. There areseveral possibilities in such a situation, such as,the view-point may be stopped and prevented from advancing further.Other possibilities are that it may be deflected away from theobject, it may be allowed to penetrate the object or asdescribed in the specifics of the instant invention it mayclimb up and over the object.

The instant invention provides a "climbability" propertyfor objects in 3-D virtual environments. This property couldthen be set either by designers of 3-D objects or by usersthemselves. With such a property setting, when the user'sviewpoint comes into contact with the object, the viewpointwill automatically be adjusted to simulate the behavior ofclimbing over the object.

In today's 3D virtual environments, the resolution of whathappens when the user's viewpoint comes in contact withobjects is handled in a variety of ways. This includesstopping the movement of the viewpoint, deflecting theviewpoint, allowing the viewpoint to penetrate the object, oras the instant "climbability" invention demonstrates, havingthe viewpoint climb over the object. However currentlythere is no means provided to the user or to designers ofobjects for virtual environments, which would allow them tocontrol the climbability behavior of specific objects. There isno mechanism that wo uld support climbability as a propertyof an object which may be controlled by a designer and oruser and is independent of the user or the user's chosennavigation mechanism.

The inventors believe that their solution is novel and thebest solution to the problem. By this is meant the ability ofbeing able to have the designers of objects for a 3-Denvironment, and or users of these environments, to controlwhat happens when the user's viewpoint com es into contactwith objects. Additionally, it allows them to specify that theobject is "climbable" so that the viewpoint is automaticallyadjusted to simulate the effect of "climbing" over the object.BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a data processing systemincluding a central processing unit which is capable ofimplementing the present invention;

4FIG. 2 shows exam ples of two unrelated objects in virtual3 dimensional workspace,they are a desk and a stair case toanother landing.FIG. 3A through 3C demonstrate that the property of

5 "penetrability" of the virtual 3 dimensional object, such asa desk. Thus the user's path of navigation to the doorinvolves penetration through the desk . The trajectory of theviewpoint toward the door, is unaffected by the presence ofthe desk.o FIGS. 4A through 4C, represent the embodiment of theinstant application. They demonstrate the property of"climbability" of the virtual 3 dimensional object, such as astaircase, which can be can be stored in the software alongwith the properties that define the object itself. Thus theuser's viewpoint comes into contact with the staircaseobject, on the path of navigation to reach a door at the topof a staircase, the viewpoint automatically adjusts to simu-late the behavior of climbing the stairs to reach the door.FIG. 5 is a flowchart for the process implemen ted by thepresent invention for creating the effect of a climbing m eans

20 to adjust the viewpoint, when the user's viewpoint comesinto contact with an object in virtual 3-D space. As a result,the viewpoint will automatically be adjusted to simulate thebehavior of climbing over the object to reach the desired 3-Dobject.

DETAILED DESCRIPTION OF THEPREFERRED EMBODIMENTBefore going into the details of specific embodiments, itwill be helpful to understand from a more general perspec-

30 tive the various elements and method which m ay be used toimplement the present invention. The present invention isimplemented in three-dimensional virtual workspace. Athree-dimensional workspace is a workspace that is per-ceived as extending in three orthogonal directions. Typically

35 a display has a two-dimensional display surface and theperception of a third dimension is effected by visual cluessuch as perspective lines extending toward a vanishingpoint. Distant objects are obscured by nearer objects. Thethree-dimensional effect is also provided by showing

40 changes in objects as they move toward or away from theviewer. Perspective shading of objects and a variety ofshadowing of ob jects at different distances from the vieweralso contribute to the three-dimensional effect.A three-dimensional workspace is typically perceived as

45 being viewed from a position within the workspace. Thisposition is a viewpoint. This viewpoint provides the virtualinterface between the display user and the display. Theviewpoint's direction of orientation is the direction from theviewpoint into the field of view a long the axis at the center50 of he field of view.

In order to present a three-dimensional workspace, asystem may store data indicating "coordinates" of the posi-tion of an object, a viewpoint or other display feature in theworkspace. Data indicating coordinates of a display feature55 can then be used in presenting the display feature so that itis perceptible as positioned at the indicated coordinates. The

"distance" between two display features is the perceptibledistance between them, and can be determined from theircoordinates if they are presented so that they appear to beso positioned at their coordinates.

Techniques for providing and hand ling three-dimensionalobjects in a three-dimensional virtual workspace have beendeveloped in the art and are available to display userinterface designers. U.S. Pat. No. 5,276,785 (Mackinlay et65 al., Jan. 4, 1994) is an example of the design techniques

available to such three-dimensional workspace interfacedesigners.

EX. B PG.



34/57

5,883,628

45

5The description of the present invention often refers tonavigation within the three-dimensional virtual workspace.The workspace or landscape is navigable using conventionalthree-dimensional navigation techniques. A user may movearound or navigate within the three-dimensional data repre-sentation to alter his perspective and view of the displayedrepresentation of the data. Thus, a user may be referred to asa navigator. The navigator is actually stationary, and hisview of the display space changes to give him the sensationof moving within the three-dimensional graphical space.Thus, we speak in terms of the navigator's perceived motionwhen we refer to changes in his view of the display space.As the user moves, his view of the data changes accordingly

within the three-dimensional data representation. Somenavigation modes include browsing, searching and datamovement. U.S. Pat. No. 5,555,354 (Strasnick et al., Sep.10, 1996) describes some known navigation techniques.The three-dimensional objects which will be subsequentlydescribed in embodiments of the present invention may bebest implemented using object oriented programmingtechniques, such as the o bject oriented techniques describedin the above-mentioned copending application Ser. No.08/753,076 assigned to the Assignee of the present inven-tion. The objects of that copending application are imple-mented using the C++ programming language. C++ is acompiled language.

The programs are written in human readable script andthis script is provided to another program called a compilerto generate a machine readable numeric code which can beloaded into, and directly executed by the computer. The C++language possesses certain characteristics which allow asoftware developer to easily use programs written by otherswhile still providing a great deal of control over the reuse ofprograms to prevent their destruction or improper use. TheC++ language is well known and m any articles and text areavailable which describe the language in detail.

While the embodiment of the present invention, whichwill be subsequently described, can be implemented usingobject oriented techniques involving the C ++ programminglanguage, we found it desirable to use SCL as used in VRT:the Virtual Reality Toolkit developed and marketed bySuperscape Ltd. having U.S. offices in Palo Alto, Calif.Extensive details of these programming techniques may befound in the Superscape VRT, Reference Manual, Version4-00, 2d Edition, Jan. 29, 1996.

It should be understood by those skilled in the art thatobject oriented programming techniques involve thedefinition, creation, use and instruction of "objects". The seobjects are software entities comprising data elements androutines, or methods, which manipulate the data elements.The data and related methods are treated by the software asan entity and can be created, used and deleted as such. Thedata and functions enable objects to model their real worldequivalent entity in terms of its attributes, which can bepresented by the data elements, and its behavior which canbe represented by its methods.

Objects are defined by creating "classes" which are notobjects themselves, but which act as templates whichinstruct a compiler how to construct the actual object. Forexample, a class may specify the number and type of datavariables and the steps involved in the functions whichmanipulate the data. An object is actually created in theprogram by mean s of a special function called a constructorwhich uses the corresponding class definition and additionalinformation, such as arguments provided during objectcreation, to construct the object. Objects are destroyed by aspecial function called a destructor.

6Many benefits arise out of three basic properties of objectoriented programming techniques, encapsulation, polymor-phism and inheritance. Objects can be designed to hide, or

encapsulate, all or a portion of, the internal data structure5 and the internal functions. More particularly, during pro-gram design, a program developer can define objects inwhich all or some of the data variables and all or some of therelated method are considered "private" or for use only bythe object itself. Other data or methods can be declared

to "public" or available for use by other software programs.Access to the private variables and methods by other pro-grams can be controlled by defining public methods whichaccess the object's private data. The public methods form aninterface between the private data and external programs. An15 attempt to write program code which directly accesses theprivate variables causes a compiler to generate an errorduring program compilation. This error stops the compila-tion process and presents the program from being run.

Polymorphism allows objects and functions which have20 the same overall format, but which w ork with different data,

to function differently to produce consistent results. Forexample, an addition method may be defined as variableA+variable B, (A+B). The same format can be used w hetherthe A and B are numbers, characters or dollars and cents.

25 However, the actual program code which performs theaddition may differ widely depending on the type of vari-ables which comprise A and B. After the methods have beendefined, a program can later refer to the addition method byits common format (A+B) and, during compilation, the30 compiler will determine which of the three methods to beused by examining the variable types. The compiler willthen substitute the proper function code.

A third property of object oriented programm ing is inher-itance which allows program developers to reuse pre-35 existing programs. Inheritance allows a software developerto define classes and the objects which are later created fromthem as related through a class hierarchy. Specifically,classes may be designated as subclasses of other base

classes. A subclass "inherits" and has access to all of the40 public functions of its base classes as though these functionsappeared in the subclass. A lternatively, a subclass can over-

ride some or all of its inherited functions or may modifysome or all of its inherited functions by defining a newfunction with the same form.The creation of a new subclass borrowing the function-

ality of another class allows software developers to easilycustomize existing code to meet their particular needs.Although object oriented programm ing offers significant

50 improvements over other programming concepts, programdevelopment still requires significant outlays of time andeffort, especially if no pre-existing software programs areavailable for modification. Consequently, a set ofpredefined, interconnected classes are sometimes provided

55 to create a set of objects and additional miscellaneousroutines which are all directed to performing commonlyencountered tasks in a particular environment. Such pre-defined classes and libraries are typically called "frame-works" and essentially provide a prefabricated structure as a

60 basis for creating a working application program.In object oriented programming such as the previouslydescribed VRT software platform, there is provided for the

user interface a framework containing a set of predefinedinterface objects. The framework contains predefined65 classes which can be used as base classes and a developermay accept and incorporate some of the objects into these

base classes, or he may modify or override objects or

EX. B PG. 3



35/57

5,883,6287

combinations of objects in these base classes to extend theframework and create customized solutions in particularareas of expertise.

This object oriented approach provides a m ajor advantageover traditional programming since the programmer is notchanging the original program, but rather extending thecapabilities of the original program.The above-described Superscap e Virtual Reality TooIlcit(VRT) provides the architectural guidance and modeling,but at the same time frees developers to supply specificactions unique to the particular problem domain which thedeveloper is addressing.

Therefore, those skilled in the art will understand how thepresent invention is implemented using object orientedprogramming techniques as described above.Referring to FIG. 1, a typical data processing system isshown which may be used in conjunction with objectoriented software in implementing the present invention. Acentral processing unit (CPU), such as one of the PowerPCmicroprocessors available from International BusinessMachines Corpo ration (PowerPC is a trademark of Interna-

tional Business Machines Corporation) is provided andinterconnected to various other components by system bus12. An operating system 41 runs on CPU 10 and providescontrol and is used to coord inate the function of the variouscomponents of FIG. 1. Operating system 41 may be one ofthe commercially available operating systems such as DOS,or the OS/2 operating system available from InternationalBusiness Machines Corporation (0S/2 is a trademark ofInternational Business Machines Corporation). A programapplication such as the program in the above-mentionedVRT platform 40 runs in conjunction w ith operating system41 and provides output calls to the operating system 41which implements the various functions to be performed bythe application 40.

A read only memory (ROM) 16 is connected to CPU 10,via bus 12 and includes the basic input/output system(BIOS) that controls the basic computer functions. Randomaccess memory (RAM) 14, I/O adapter 18 and communi-cations adapter 34 are also interconnected to system bus 12 .It should be noted that software components including theoperating system 41 and application 40 are loaded into RAM14 which is the computer system's main memory. I/Oadapter 18 may be a small computer system interface (SCSI)adapter that communicates with the disk storage device 20,i.e. a hard drive. Communications adapter 34 interconnectsbus 12 w ith an outside network enabling the data processingsystem to comm unicate with other such systems over a localarea network (LAN), wide area network (WAN), or the like.I/O devices are also connected to system bus 12 via userinterface adapter 22 and display adapter 36. Keyboard 24,trackball 32, mouse 26 and speaker 28 are all interconnectedto bus 12 through user interface adapter 22. Display adapter36 includes a frame buffer 39 which is a storage device thatholds a representation of each pixel on the display screen 38.Images may be stored in frame buffer 39 for display onmonitor 38 through various com ponents such as a digital toanalog converter (not shown) and the like. By using theaforementioned I/O devices, a user is capable of inputtinginformation to the system through the keyboard 24, trackball32 or mouse 26 and receiving output information from thesystem via speaker 28 and display 38.

There will now be described a simple illustration of thepresent invention. When the images are described, it will beunderstood that these may be rendered by storing a virtualreality three-dimensional image creation application pro-

8gram 40 such as the previously described VRT of Super-scape in the RAM 14 of the system of FIG. 1. Also storedon the RAM will be a suitable operating system 41 such asDOS or Windows.

5n embodiment of the present invention will now bedescribed with respect to the virtual reality workspaceshown in FIG. 2; which shows examples of two unrelatedobjects in virtual 3 dimensional workspace which are a desk102 and a staircase 100 to another landing. In the case of the

10 desk 102, we will subsequently see in FIGS. 3AC, theproperties of "penetrability" and contrast this with theproperty of "climbability" FIGS. 4A C, the embodiment ofthe instant invention.With regard to FIG . 3A, this demonstrates several objects

15 that exist in three-dimensional workspace 2 00; specifically adesk 201 and a staircase 202 and a bookcase 203. Theworkspace 200 FIG. 3 as noted is shown as an officeenvironment. The workspace 200 is centered within a view-point interface which is presented to the viewer on display20 monitor 38 of FIG. 1. In accordance with conventionaltechniques, the user may control the viewpoint 200Athrough a conventional I/O device such as mouse 26 or F IG.1 which operates through the user interface 22 of FIG. 1 tocall upon VRT programs in RAM 14 operating with the

25 operating system 41 to

activision publishing v. novalogic

Documents