vr juggler technical overview a virtual platform for virtual reality application development allen...
TRANSCRIPT
VR Juggler Technical OverviewVR Juggler Technical Overview
A Virtual Platform for Virtual Reality A Virtual Platform for Virtual Reality Application DevelopmentApplication Development
Allen BierbaumAllen Bierbaum
Iowa State UniversityIowa State University
IEEE VR TutorialIEEE VR Tutorial
www.vrjuggler.org 2
Who am I?Who am I?
• Allen BierbaumAllen Bierbaum• WorkWork
– VR Juggler TeamVR Juggler Team– PhD StudentPhD Student
• Iowa State University – Virtual Reality Iowa State University – Virtual Reality Applications Center (VRAC)Applications Center (VRAC)
www.vrjuggler.org 3
ObjectivesObjectives
• Understand VR Juggler architectureUnderstand VR Juggler architecture• Knowledge of some technical detailsKnowledge of some technical details• Introduction to future VR Juggler Introduction to future VR Juggler
developmentsdevelopments
• Please: Ask questions!!! Please: Ask questions!!!
www.vrjuggler.org 4
What is VR Juggler?What is VR Juggler?
• Open Source VR software development Open Source VR software development environmentenvironment
• VR Virtual platform & Application VR Virtual platform & Application frameworkframework
• Developed at Iowa State UniversityDeveloped at Iowa State University– Project director: Dr. Carolina Cruz-NeiraProject director: Dr. Carolina Cruz-Neira– 5+ years of development5+ years of development– VR Juggler team (3 people VR Juggler team (3 people 10+ people at 10+ people at
VRAC)VRAC)
www.vrjuggler.org 5
VR Juggler backgroundVR Juggler background
MotivationMotivation• Standard VR development environment that is:Standard VR development environment that is:
– Flexible & extensibleFlexible & extensible– MaintainableMaintainable– ModularModular
– Cross-platform / Cross-VR SystemCross-platform / Cross-VR System– Freely available – OpenSourceFreely available – OpenSource– Allow community involvementAllow community involvement
– Extension and refinementExtension and refinement– Peer reviewPeer review
www.vrjuggler.org 6
VR Juggler featuresVR Juggler features• Virtual platformVirtual platform• Application objects and frameworkApplication objects and framework• Performance drivenPerformance driven
– Performance monitoringPerformance monitoring
• Flexible and extensibleFlexible and extensible• ModularModular• Open sourceOpen source• Cross-platform – Irix, Linux, Win32, others…Cross-platform – Irix, Linux, Win32, others…• Configuration systemConfiguration system
– Parameterization of VR system components Parameterization of VR system components – Run-time reconfigurableRun-time reconfigurable– GUI – vjControlGUI – vjControl
www.vrjuggler.org 7
Virtual platformVirtual platform
• Separates system dependent and system Separates system dependent and system independent componentsindependent components
www.vrjuggler.org 8
Virtual platform: BenefitsVirtual platform: Benefits
BenefitsBenefits• Architecture and OS independenceArchitecture and OS independence
– Handles cross platform issuesHandles cross platform issues– Virtual platform scalesVirtual platform scales
• Device abstractionDevice abstraction– Separates application from hardwareSeparates application from hardware– Proxies – handles to devicesProxies – handles to devices
• Defines an operating environment for VR appsDefines an operating environment for VR apps– The VP allocates resources and processing timeThe VP allocates resources and processing time
www.vrjuggler.org 9
System ParameterizationSystem Parameterization
• Juggler “config chunks” specify single unit of Juggler “config chunks” specify single unit of configurationconfiguration
• Parameterize all configurable parts of systemParameterize all configurable parts of system• Edited with Java base GUI (vjControl)Edited with Java base GUI (vjControl)
www.vrjuggler.org 10
www.vrjuggler.org 11
Kernel subsystem: MotivationKernel subsystem: Motivation
Motivating needsMotivating needs• System must be extensibleSystem must be extensible
– Add new devices and graphics APIs Add new devices and graphics APIs
• System needs to be flexibleSystem needs to be flexible– Adapt to VR systems, multiple configurationsAdapt to VR systems, multiple configurations
• System needs to be dynamicSystem needs to be dynamic– Changes at run-time, added modularityChanges at run-time, added modularity
• System must evolve over timeSystem must evolve over time– App survivabilityApp survivability
www.vrjuggler.org 12
Kernel subsystemKernel subsystem
Microkernel architectureMicrokernel architecture• Controls the entire systemControls the entire system• Manages communicationManages communication• Internal ManagersInternal Managers
– Core functionality beyond the scope of the Core functionality beyond the scope of the kernelkernel
• External ManagersExternal Managers– Provide an interface to the system that is Provide an interface to the system that is
specific to application typespecific to application type
www.vrjuggler.org 13
Kernel subsystem: BenefitsKernel subsystem: Benefits
BenefitsBenefits• Modular systemModular system
– Managers can be added, removed, and Managers can be added, removed, and reconfiguredreconfigured
– Discourages monolithic architecturesDiscourages monolithic architectures
• MediatorMediator– Captures interactionCaptures interaction– SynchronizationSynchronization
• PortabilityPortability
www.vrjuggler.org 14
Internal managersInternal managers
• Input managerInput manager• Display managerDisplay manager• Config managerConfig manager• Environment managerEnvironment manager
www.vrjuggler.org 15
Input ManagerInput Manager
• Controls all devices in the systemControls all devices in the system• Updates the devicesUpdates the devices• Allows users to request devicesAllows users to request devices
www.vrjuggler.org 16
DevicesDevices
• Base classes for all device typesBase classes for all device types• Extend system hereExtend system here
www.vrjuggler.org 17
Device ProxiesDevice Proxies
• All Devices are accessed through proxiesAll Devices are accessed through proxies• Proxies hide the true device typeProxies hide the true device type• Allows code and configuration to be device independentAllows code and configuration to be device independent
0..1
0..1
0..*
0..*
vjPosProxy
vjPosProxy~vjPosProxyupdateDatagetUpdateTimesetTransformsetgetDatagetUnitgetPositionPtrgetTransformtransformDatagetChunkTypeconfiggetProxiedInputDevice
vjPosInterface
vjPosInterfaceoperator->operator*refresh
vjInput
vjPosition
vjPosition~vjPositionconfigstartSamplingstopSamplingsampleupdateDatagetDeviceNamegetPosDatagetPosUpdateTime
vjInputManagervjProxy
vjProxyconfiggetProxiedInputDevicegetChunkType
www.vrjuggler.org 18
Device FactoryDevice Factory
• All device types are registeredAll device types are registered• Allows for dynamic discovery and loadingAllows for dynamic discovery and loading• Separates devices from the rest of the Separates devices from the rest of the
systemsystem
www.vrjuggler.org 19
Display managerDisplay manager
• Contains all information about display Contains all information about display windowswindows
• Performs all viewing calculationsPerforms all viewing calculations
www.vrjuggler.org 20
Other internal managersOther internal managers
Config managerConfig manager• Holds (re)configuration requestsHolds (re)configuration requests
Environment managerEnvironment manager• Communicates the state of the systemCommunicates the state of the system
www.vrjuggler.org 21
External managersExternal managers
• Draw managerDraw manager• Sound managerSound manager
www.vrjuggler.org 22
Draw managerDraw manager
• Provides external interface for applicationsProvides external interface for applications• Allows access to API specific functionalityAllows access to API specific functionality• Handles all details of the APIHandles all details of the API
www.vrjuggler.org 23
Other External managersOther External managers
Sonix: Sound managerSonix: Sound manager• Generic interface for several sound libsGeneric interface for several sound libs• More in later talk…More in later talk…
Future…Future…• Area for extensionArea for extension
www.vrjuggler.org 24
Application objectsApplication objects
– All VR Juggler applications are objectsAll VR Juggler applications are objects
www.vrjuggler.org 25
Application objectsApplication objects
– Override methods of a pre-defined interfaceOverride methods of a pre-defined interface– Derived class for each graphics APIDerived class for each graphics API
www.vrjuggler.org 26
Application objectsApplication objects
BenefitsBenefits• Single interface between app and systemSingle interface between app and system
– Low-coupling Low-coupling Robustness Robustness– Application survivabilityApplication survivability
• Allows for flexibility in kernelAllows for flexibility in kernel– Run-time changesRun-time changes
• Application managementApplication management– Capture application in single unitCapture application in single unit– Multiple applicationsMultiple applications
www.vrjuggler.org 27
Run-time (re)configurationRun-time (re)configuration
Most current systemsMost current systems• Static configurationStatic configuration• No way to modify settings at run-timeNo way to modify settings at run-time• Configuring systemConfiguring system
– Test settings Test settings Change settings Change settings Restart application Restart application
Run-time ConfigurationRun-time Configuration• Configuration is done dynamically at runtimeConfiguration is done dynamically at runtime
Run-time ReconfigurationRun-time Reconfiguration• Maximum flexibilityMaximum flexibility• Allows settings to change during executionAllows settings to change during execution
www.vrjuggler.org 28
Reconfiguration interfaceReconfiguration interface
• vjConfigChunkHandlervjConfigChunkHandler– API for handling configuration requestsAPI for handling configuration requests– Implemented by all Managers and some applicationsImplemented by all Managers and some applications
www.vrjuggler.org 29
Config ManagerConfig Manager
• Pending configuration queuePending configuration queue• List of configuration requestsList of configuration requests• Stored in queue until kernel attempts reconfigurationStored in queue until kernel attempts reconfiguration
www.vrjuggler.org 30
Reconfiguration processReconfiguration process
www.vrjuggler.org 31
Reconfiguration benefitsReconfiguration benefitsSystem setup and configurationSystem setup and configuration
• Easier initial setup using interactive testingEasier initial setup using interactive testing– Reconfigure displaysReconfigure displays– Change physical devicesChange physical devices
Application adaptationApplication adaptation• Application switching at run-timeApplication switching at run-time• Configure applications parametersConfigure applications parameters
– Model loaded, navigation methodModel loaded, navigation method
RobustnessRobustness• Graceful recovery from device or app failuresGraceful recovery from device or app failures
Performance tuningPerformance tuning• Adjust configuration based upon performanceAdjust configuration based upon performance
www.vrjuggler.org 32
Dependency managementDependency management
• Dependency checking between componentsDependency checking between components• Components needing checking register a checkerComponents needing checking register a checker
www.vrjuggler.org 33
System interactionSystem interactionvjKernel
while(!quit)
vjInputManager
vjDisplayManager
vjGlApp
vjGlDrawManager
vjGlPipe
while(!quit)
draw():void
draw():void
wait()
triggerRender():void
<constructor>()
<constructor>()
sync():void
updateProjections():void
updateAllData():void
checkForReconfig():void
postFrame():void
intraFrame():void
draw():void
preFrame():void
start():void
initialize()
getDrawManager():vjDrawManager *
setup()
<constructor>()
setup()
<constructor>()
setApplication()
<constructor>()
start():int
www.vrjuggler.org 34
System interaction (cont)System interaction (cont)
while(!quit)
vjGlDrawManager
vjGlPipe
while(!quit)
draw():void
draw():void
wait()
triggerRender():void
<constructor>()
<constructor>()
sync():void
updateProjections():void
updateAllData():void
checkForReconfig():void
postFrame():void
intraFrame():void
draw():void
preFrame():void
start():void
initialize()
getDrawManager():vjDrawManager *
setup()
<constructor>()
setup()
<constructor>()
setApplication()
<constructor>()
start():int
VR Juggler 1.1VR Juggler 1.1
VR Juggler development VR Juggler development directiondirection
www.vrjuggler.org 36
MotivationMotivation
New abilities and research areasNew abilities and research areas• High-level toolsHigh-level tools• Technological advancesTechnological advances
Fix design problemsFix design problems• Yes, we actually made some mistakes…Yes, we actually made some mistakes…
www.vrjuggler.org 37
www.vrjuggler.org 38
VR Juggler 1.1 ModulesVR Juggler 1.1 Modules
Current ModulesCurrent Modules• Vapor – System abstractionVapor – System abstraction• Gadgeteer – Device managementGadgeteer – Device management• GMTL – High-performance mathGMTL – High-performance math• JCCL – ConfigurationJCCL – Configuration• VR Juggler – VR virtual platformVR Juggler – VR virtual platform• Sonix – Sound managementSonix – Sound management• Tweek – Common GUI frameworkTweek – Common GUI framework
This all used to be just VR Juggler…This all used to be just VR Juggler…• Now the Juggler suite of toolsNow the Juggler suite of tools• More on the way…More on the way…
www.vrjuggler.org 39
New featuresNew features
VaporVapor• Serial port abstractionSerial port abstraction
GadgeteerGadgeteer• ClusteringClustering• Run-time device type extensionRun-time device type extension• Input from gfx windowsInput from gfx windows• Cross-platform device driversCross-platform device drivers
www.vrjuggler.org 40
New features (cont)New features (cont)
VR JugglerVR Juggler• Revamped configuration GUIRevamped configuration GUI• Window viewportsWindow viewports• OpenSG & OpenSceneGraphOpenSG & OpenSceneGraph• Performer reconfigurabilityPerformer reconfigurability• Kernel shutdownKernel shutdown
TweekTweek• Palmtop interactionPalmtop interaction
SonixSonix• Sound managementSound management
www.vrjuggler.org 41
High-level toolsHigh-level tools
• Many new high-level tools in the pipelineMany new high-level tools in the pipeline• All part of the Juggler suiteAll part of the Juggler suite
In developmentIn development• Collaborative networkingCollaborative networking• SonificationSonification• User interfacesUser interfaces• and more…and more…
www.vrjuggler.org 42
AvailabilityAvailability
• Latest version always in CVSLatest version always in CVS• Developer releasesDeveloper releases
– VRJ 1.1 DR1 “on the way”VRJ 1.1 DR1 “on the way”
• New website with documentationNew website with documentation