access grid update robert olson argonne national laboratory sura/vide 5 th annual digital video...
TRANSCRIPT
Access Grid Update
Robert Olson
Argonne National Laboratory
SURA/ViDe 5th Annual Digital Video Workshop
Abstract
• In the years since we released the first Access Grid specifications and software, we have learned a great deal about how one might use this technology to enhance collaboration. With the 2.0 release of the AG software, we apply this knowledge to produce a system that is much more capable of enhancing collaboration between groups of people and the tools they use. In this talk I discuss what is new with the AG 2.0 software release and how its capabilities may be applied.
The Access Grid
• The Access Grid project’s focus is to enable groups of people to interact with Grid resources and to use the Grid technology to support group to group collaboration at a distance
• Supporting distributed research collaborations• Distributed Lectures and seminars• Remote participation in design and development• Virtual site visits and team meetings• Complex distributed grid based applications• Long term collaborative workflows
Access Grid – The Novel Ideas
• Peer-to-peer Virtual Venues servers to enable worldwide, secure virtual communities through the use of high-end collaboration environments
• Collaborative work sharing beyond simple application sharing
• Integration of high-end visualization environments into collaborative spaces
• Methods of asynchronous collaboration: capture, synchronization, record, playback and annotation of collaborative experiences.
HW Components of an AG Node
DisplayComputer
Video CaptureComputer
Audio CaptureComputer
Mixer
EchoCanceller
Network
RGB Video
NTSC Video
Analog Audio
Digital Video
Digital Video
Digital Audio
Shared App,Control
ControlComputer
RS232 Serial
Access Grid Project Goals
• Enable Group-to-Group Interaction and Collaboration• Connecting People and Teams via the Grid
• Improve the User Experience: Go Beyond Teleconferencing• Provide a Sense of Presence• Support Natural Interaction Modalities
• Use Quality but Affordable Digital IP Based Audio/video• Leverage IP Open Source Tools
• Enable Complex Multisite Visual and Collaborative Experiences• Integrate With High-end Visualization Environments• ActiveMural, Powerwall, CAVE Family, Workbenches
• Build on Integrated Grid Services Architecture• Develop New Tools Specifically Support Group Collaboration
Our Approach• Attack Research Questions in the context of real world
experience• Build up a critical mass of groups using the AG Platform • Involve multiple groups in trying new ideas and evaluation
• Build Working Infrastructure as well as Prototype Software • Argonne has five working AG nodes under development• New Software is used weekly/Daily as part of standard
nanocruises
• Involve multiple groups in deployment, use and research• Active collaborations with over a dozen groups working on AG
technology
• Release software early and often (use open source model)
• Contribute to the Community Code base
Group-to-Group Interaction is Different• Large-scale scientific and technical
collaborations often involve multiple teams working together
• Group-to-group interactions are more complex than than individual-to-individual interactions
• The Access Grid project is aimed at exploring and supporting this more complex set of requirements and functions
• The Access Grid will integrate and leverage desktop tools as needed
Some Access Grid Active Research Issues• Scalable wide area communication
• Evolution of multicast related techniques, and time shifting issues
• Scoping of resources and persistence• Value of spatial metaphors, security models• Virtual Venues, synchronous and asynchronous models
• Improving sense of presence and point of view• Wide Field Video, Tiled Video, High-resolution video codecs
• Network monitoring and bandwidth management • Beacons and network flow engine
• Role of Back-channel communications • Text channels and private audio
• Recording and playback of multistream media
What is the Access Grid?• Virtual Venues
• Places where users collaborate
• Network Services• Advanced Middleware
• Virtual Venues Client• User Software
• Nodes• Shared Nodes
• Administratively scoped set of resources
• Resources• Provide capabilities
• Personal Nodes• User scoped set of
Resources
Users collaborate by sharing:•Data•Applications•Resources
Access Grid Architecture
Venue Server
VenueClient
VenueServer
Management
Access Grid Node
NodeService
NetworkServices
Venue7
Venue5Venue3
Venue1 Venue8
Lobby
ServiceManager
Service
ServiceManager
Service
ServiceManager
ServiceNetworkServices
NetworkServices
Node ManagementClient
Virtual Venues• What is a Virtual Venue?
• A Virtual Venue is a virtual space for people to collaborate• What do Virtual Venues provide?
• Entry/Exit Authorization Information• Connections to other Venues• Coherence among Users
• Venue Environment, Users, Data• Client Capabilities Negotiation
• List of Available Network Services • Keep track of resulting Stream Configurations
• Applications• Virtual Venues have two interfaces
• Administrative – Venue Management Software• Client – Virtual Venue Client Software
Virtual Venues Client• Enable face-to-face meeting
activities• What can be done:
• Sharing Data• Shared Applications
• Applications:• Distributed PowerPoint• Shared Web browser• Whiteboard• Voting Tool• Question & Answer Tool• Shared Desktop Tool
• Integrate legacy single-user apps
Network Services• Network Services
• Provide a middleware layer for enabling the richest collaborations• Are invisible to Venues Clients, used by Virtual Venues• Primarily Transform streaming data• Can be anywhere on the network• Can be composed to build complex solutions:
• Venue Audio Stream Audio Transcoder Audio to Text Two-Way Pager
• Two-Way Pager Text to Audio Audio Transcoder Venue Audio Stream
• Network Services provide opportunities for third party developers
• ANL is working on Network Services for• Audio Transcoding (16KHz ↔ 8KHz)• Video Stream Selection
Access Grid Nodes• Access Grid Nodes
• Comprise a set of collaboration resources• Expose those resources through Node Services
• Basic Node Services include:• Audio & Video Services• Network Performance Monitoring Service• Network Reliability/Fallback Service• Leashing Service – Registering presence with a shared
node
• Extended Node Services could be:• Display Service with enhanced layout control• Video Service supporting new CODECs• Automatic performance adaptation• Application Hosting Service
Access Grid 2.0 Design Requirements• Secure Communication Throughout• Reliable, Robust Data Transport
• Example: Network Failover Technology• More Diverse Reference Platforms
• Handhelds High End Solutions• Personal and Shared Nodes
• More Usable Software• Well Documented Interfaces• Federated Operation• Integrate Grid Computing Technology
• AG 2.0 is Web Services based• AG 2.0 uses GT2.X • AG 2.0 can enhance an OGSI by providing collaboration
services
Access Grid Nodes• Access Grid 2.0 reference platforms:
1. Advanced Node – Tiled Display, Multiple Video Streams, Localized Audio
2. Room Node – Shared Display, Multiple Video Streams, Single Audio Stream (AG 1.x Node)
3. Desktop Node – Desktop Monitor, Multiple Video Streams, Single Audio Stream (AG 1.X PIG)
4. Laptop Node – Laptop Display, Single Video Stream, Single Audio Stream
5. Minimal Node – Compact Display, Single Video Stream, Single Audio Stream
• What Hardware?• Cameras, Microphones, Speakers, Display, Input Devices• Get Audio Correct!
• Software Requirements?• Python 2.2, wxPython, GT2.0, pyGlobus
Summary of Changes from 1.0 to 2.01.0• Virtual Venues
• Static Media Configurations• Assumed Multicast Technology
• Single Server assumption
• Virtual Venues Client• Web Browser
• Nodes• Non-extensible single reference
platform• AG 1.1 1.2 PIGs introduced
• Applications layered outside of AG software
2.0• Virtual Venues
• Dynamic Media Configurations• Capability Brokering Functionality
• Integrated Data Storage• Support for highly scalable
deployments• Multicast Addressing• Topological Simplicity (connections as
URLs)
• Virtual Venues Client• Streamlined Client• Integrated Grid Security• Workspace Docking• Application Development Interfaces
Exposed• Nodes
• Nodes defined in terms of resources• Management UI• Interfaces exposed for building new
Services• Broader set of Reference Platforms
• Applications• Venue Hosted Collaborative Apps
• Network Services
Access Grid 2.0 Development• Technology Details:
• Windows (2000, XP)• Linux• Globus Toolkit 2.X• Web Services
• We prefer Python
• Partners Tools• Globus toolkit • Python CoG kit (LBNL) • LBNL Intergroup
Communications (LBNL)• Condor ClassAds
• Project Strategy• Open Source Project
Model• Standard Tools
• CVS, Bugzilla
• Access Grid Project Meetings• First Tuesday of each
month• Argonne Institutional
Venue• Next meeting April. 1st,
2003, 10-12am CST.
Access Grid 2.0 Timeline• Code Available now from:
• :pserver:[email protected]:/cvsroot co AccessGrid
• 2.0 Beta 1 Available now (March 15, 2003)• Virtual Venues Server
• Transitional Venue Server for AG1.X AG2.0 Migration• https://vv2.mcs.anl.gov:9000/Venues/default (The
Access Grid Lobby)
• Basic Node Services• Virtual Venues Client Software• Venues Management UI
• Final 2.0 Toolkit April 15th, 2003
Access Grid Technology Overview
Technological Goals• Enable comprehensive security• Leverage existing technology
• Globus Toolkit• SOAP + WSDL
• Provide a low barrier of entry for …• New developers• Rapid development of new functionality• Adding third-party extensions
Security: General goals• Identification of users and services• Authentication of the identity of these users
and services• Authorization for access to resources• Privacy of data (files, streams, control, etc.)
• Public Key Infrastructure provides standards and mechanisms to fulfill these needs
Security: Identification• Users and services identified with a
public key identity certificate issued by a trusted certificate authority
• An identity certificate contains:• Information about the subject of the
certificate• A public key representing the subject• The digital signature of the CA issuing the
cert
• For example, a Globus identity certificate:% openssl x509 -noout -text -in ~/.globus/usercert.pem Certificate: Data: Version: 3 (0x2) Serial Number: 6060 (0x17ac) Signature Algorithm: md5WithRSAEncryption Issuer: C=US, O=Globus, CN=Globus Certification Authority Validity Not Before: Jan 7 20:22:19 2002 GMT Not After : Jan 7 20:22:19 2003 GMT Subject: O=Grid, O=Globus, OU=mcs.anl.gov, CN=Bob Olson Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (1024 bit) Modulus (1024 bit): 00:cd:7d:bb:ae:30:bb:c1:74:2d:e4:6e:d4:30:6e: [etc]
Exponent: 65537 (0x10001) X509v3 extensions: Netscape Cert Type: SSL Client, SSL Server Signature Algorithm: md5WithRSAEncryption 23:14:96:05:0d:db:ce:aa:70:17:03:5a:07:31:a0:81:e3:10:
…
Identity Certificates
Subject information
Subject Public Key
CA Signature
Security: Authentication• Assumptions:
• Authentication takes place on a transaction between a client and a server
• Client and server each hold an identity cert• Authentication is mutual: After completion,
client and server have verified identity of the other party
• Secured communications in AG2 use Globus…
• …which uses SSL/TLS• SSL/TLS defines protocol for a secure
handshake with mutual authentication.
Security: Authorization• Authorization is the process of gating access
to a resource based on some criteria.• Many different approaches, few standards.
• Access control lists• Role-based authorization• Attribute certificates
• AG2 approach: provide building blocks for applications to define authorization.
• Reference implementation uses a basic role-based authorization scheme.
Security: Privacy• Usually what people think when they think
security• Straightforward, once authentication and
authorization issues overcome• Globus Security Infrastructure uses SSL/TLS
mechanisms for privacy• Typically, symmetric encryption with session
keys negotiated at session startup.• Media data uses AES encryption with
session keys distributed by secure channels.
Practical security issues• In AG2.0 Alpha, each user must have an identity
certificate• Identity certs issued by Certificate Authorities
• AG Development CA• Globus test CA• DOE Science Grid CA• Commercial CA (Verisign, Thawte, …)
• Certificate Safety• If the private key for a cert is compromised, the cert
cannot be trusted• Hence, users have responsibility for maintaining safety of
their keys
• The use of identity certificates is often cumbersome
Identity Maintenance Alternatives• NCSA MyProxy
• Online proxy storage for standard identity certificates• Medium-term expiration proxies kept at central server• Proxies created via username/password authentication
• Online CA with username/password support• Identity certificates held at an online CA• Proxies created via username/password authentication• No requirement for user storage of certs• Integration with Shibboleth or other single sign-on
infrastructure
Trust issues• If a CA is not trusted by a service, then no
certificates issued by that CA are trusted• CA trust is a minimum requirement for access
Application development
Building on Access Grid 2.0• The Access Grid Toolkit is extensible by the
addition of Node Services, Network Services and Applications.
• This tutorial covers the building of a Node Service and the building of two Applications:• A Shared Web Browser• A Distributed Presentation Viewer
Building a Node Service• Node Services for Streaming Media
• Responsibilities• Adhere to Node Service interface• Packetize and send media streams to network• Respond to stream description updates
Configuration
NodeService
VenueClient
vic
VideoProducerService
NodeManagement
Client
Data
Building Applications• Applications rely the Venue for discovery,
coherence, coordination and synchronization.
• In order to support Applications, we have added a new piece an Application Factory to the Venue.
• Application Factory Application Objects + Application Clients
Distributed Applications integrated with AG
Application Architecture: Venue Side• An Application
Factory creates Venue-resident applications.
• Each application is represented in the venue by an Application Object
• An application object can store local data and can have one event channel
• Event channels utilize a Venue-based Event Service
VenueEvent Service
Application Object name type webServiceUri channel
Event Channel
Local Data
ApplicationFactory
Venue
Application Architecture: User Side• On the user side the Venue
Client is the key.• The user can install
applications, which are then available to the Venue Client.
• When a user enters a Venue if there are application objects, the Venue Client looks for applications that are of the same type.
• The Venue Client also enables the user to start a local application, and create the Application Object in the Venue.
Venue ClientApplication Client
Type: X
Application ClientType: Y
Application ClientType: Z
Venue State (Including Application Objects)
Example Applications• Shared Web Browser
• Stateless – Current page not stored anywhere but in the clients
• Shared Presentation Viewer• Stateful –
• Master Presenter• Current Slide• Slides
A Shared Web Browser• Application task: Web browsing• All users see the same page• The Venue serves as a rendezvous
mechanism• Application state: webpage URL• State is distributed; that is, there is no
central server maintaining the state• With each state change, an event is
distributed to all interested clients
Distributed Presentation• Application task: coordinated display of
presentation material• PowerPoint is the prevalent client, but the
application can be built in a platform independent way.
• Application state• A presentation (set of slides / images / pages) • Current location within the presentation• A presenter who has control of progress through
the presentation
Access Grid 2.0 - Overview of Core• Venue Server Management• Nodes and Node Management• Network Services• Plans for the next year
Component Overview
Venue7
Venue6
Venue5
Venue4
Venue3
Venue2
Venue1
Venue8
VenueServer
VenueClient
NodeService
ServiceManager
ServiceManager
ServiceManager
VideoConsumer
Service
VideoProducerService
AudioService
TextService
VenueServer
Management
Venue Server Management
Venue Server Management• Administrators• Multicast address allocation
• Standard range• Custom range
• Storage Location
Nodes and Node Management• An AG Node often
consists of multiple machines
• The central NodeService communicates with a ServiceManager on each machine
NodeService
ServiceManager
ServiceManager
ServiceManager
Nodes and Node Management• Users install available
services to establish the capabilities of the Node
• Adding services extends the collaborative capabilities of the Node
• Services are simple to develop and integrate, facilitating third-party development
NodeService
ServiceManager
ServiceManager
ServiceManager
VideoConsumer
Service
VideoProducerService
AudioService
Nodes and Node Management• The structure of an
AG2 Node is more flexible than AG1 Nodes. On a personal Node, all the services would run on a single machine.
NodeService
ServiceManager
AudioService
VideoProducerService
VideoConsumer
Service
Nodes and Node Management• Node Services
• Expose resources on the machines in the node• Implement a specific network interface• Provide capabilities to the node
• Video, h261, 25fps• Audio, 16kHz
NodeService
ServiceManager
ServiceManager
ServiceManager
VideoConsumer
Service
VideoProducerService
AudioService
NodeService
ServiceManager
AudioService
VideoProducerService
VideoConsumer
Service
Nodes and Node Management• Node Management UI
• Add Service Managers• Add Services• Configure Services• Store/Load Configuration
Nodes and Node Management• Where AG1 Nodes consist of a collection of
hardware in a single, static configuration, AG2 enables multiple configurations:• One stream for a presentation (e.g. an instructor)• Multiple streams for a group attending a
presentation (e.g. a classroom)• High-bandwidth/Low-bandwidth configurations
Node/VenueClient/Venue interaction• A users enter the Venue with the collected
capabilities of his node• If the user has the capability of producing a
media stream not present in the Venue, the Venue allocates a new stream
• Streams are present in the Venue as long as participants are providing that stream to the Venue; the addresses assigned for media transmission are allocated and freed dynamically
Network Services• Users with incompatible capability sets
could be in a venue together, but would be unable to interact
• The Venue could resolve these incompatibilities with appropriate adapters, through Capabilities Negotiation• Video, h261 => Video, jpeg• Audio, 16kHz => Audio, 8kHz
• We call these adapters “Network Services”
Plans for the next Year• OGSI and GT3• Multiple certificates / Graduated security /
Authorization• ?? Graduated security == Roles?• ?? Authorization
• Building communities with the AGTk (~VO)• ?? Community authorization server
Plans for the next Year• Venue Client
• Client-side datastore/Docking• Subgroup communications• Automatic bridging
Plans for the next Year• Nodes
• Node advertisement
• Node Services• ? High-res Video• ? Stereo Audio• Display Service supporting advanced layout• Camera Control Service
• Operator Panel• Consolidate as much of the hands-on operational
functionality of existing media tools in a single, compact interface
Plans for the next year• Venues and Venue Server
• Capabilities Negotiation• Venue Server Registry• Stream Selection