a new jersey statewide video portal based on open source
TRANSCRIPT
A New Jersey Statewide Video Portal A New Jersey Statewide Video Portal Based on Open Source TechnologiesBased on Open Source Technologies
Isaiah BeardIsaiah BeardDigital Standards & Workflow Manager - SCCDigital Standards & Workflow Manager - SCC
Repository Architects: Ron C. Jantz and Grace AgnewRepository Architects: Ron C. Jantz and Grace Agnew
Rutgers, the State University of New JerseyRutgers, the State University of New Jersey
Internet2 Spring Member MeetingInternet2 Spring Member MeetingArlington, VA - April 23, 2008Arlington, VA - April 23, 2008
Concepts and Concepts and HistoryHistory
Architecture and Architecture and Usage ScenariosUsage Scenarios
Challenges unique Challenges unique to NJVidto NJVid
Demos/Demos/Proof of conceptProof of concept
Outline & Overview Outline & Overview
2
Concepts & HistoryConcepts & History
BeginningsBeginnings
Predecessor TechnologiesPredecessor TechnologiesNew Jersey Digital Highway New Jersey Digital Highway (njdigitalhighway.org)(njdigitalhighway.org)
Statewide repository - IMLS grantStatewide repository - IMLS grant
Established Foundation PlatformEstablished Foundation Platform
RUcore - Rutgers Community RepositoryRUcore - Rutgers Community Repository
• Expansion of NJDH technologyExpansion of NJDH technology
• Customization, portalsCustomization, portals
• Born Digital ArchivingBorn Digital Archiving
4
Predecessor TechnologiesPredecessor Technologies
Existing Video CollectionsExisting Video Collections
Partner Institutions with video contentPartner Institutions with video content
New Jersey Digital HighwayNew Jersey Digital Highway
Local Collections - WPUNJ, Montclair, etc.Local Collections - WPUNJ, Montclair, etc.
5
Collection/Video TypesCollection/Video Types
Three Major Video Collection TypesThree Major Video Collection Types
Video CommonsVideo Commons
Locally Owned/Locally producedLocally Owned/Locally produced
• Commercial Videos - ConsortiaCommercial Videos - Consortia
• Lectures on DemandLectures on Demand
6
Architecture & WorkflowArchitecture & Workflow
Nuts & BoltsNuts & Bolts
A Trusted RepositoryA Trusted Repository
Reliable content deliveryReliable content delivery
Standards-basedStandards-baseddigitizationdigitization
Preservation PlatformPreservation Platform
Rights EnforcementRights Enforcement
• Authentication/Authentication/AuthorizationAuthorization
Reference ArchitectureReference Architecture
8
Reference Architecture:Reference Architecture:Technologies in UseTechnologies in Use
9
AreaArea Software UsedSoftware Used
Applications/Web InterfaceApplications/Web Interface PHP / MySQLPHP / MySQL
JournalsJournals Open Journal System (PKP)Open Journal System (PKP)
Search EngineSearch Engine AmberfishAmberfish
Workflow MangementWorkflow Mangement PHP / MySQL PHP / MySQL (developed in-house)(developed in-house)
Collection ServicesCollection Services PHP / MySQL PHP / MySQL (developed in-house)(developed in-house)
Handles/Persistent IDsHandles/Persistent IDs CNRI Handle ServerCNRI Handle Server
Video Streaming ServerVideo Streaming Server Darwin (Apple)Darwin (Apple)
Digital Library FrameworkDigital Library Framework FEDORAFEDORA
OS/Web ServerOS/Web Server Linux/ApacheLinux/Apache
MatroxMatroxAxioAxio
EncoderEncoder
Reference ArchitectureReference Architecture
10
User InputUser Input(Metadata)(Metadata)
Workflow ManagementWorkflow ManagementSystemSystem
VideoVideo
<XML><XML> Digital ObjectDigital ObjectRepositoryRepository(FEDORA)(FEDORA)
DigitizedDigitizedVideo MasterVideo Master
FEDORA Repository ServiceFEDORA Repository Service
Collection/Preservation ServicesCollection/Preservation Services
NJVid CommonsNJVid Commons
Partner PortalsPartner Portals
Large Master Video FilesLarge Master Video Files
Master DV/AVI files @ 20GB+ per hour Master DV/AVI files @ 20GB+ per hour
• Ingest, presentation, preservationIngest, presentation, preservation
• Lectures on Demand - Annotate VideoLectures on Demand - Annotate Video
• Authentication/Authorization FacilityAuthentication/Authorization Facility
Architectural ChallengesArchitectural Challenges
11
Architectural ChallengesArchitectural Challenges
12
Object ArchitectureObject ArchitectureRepository IDRepository ID
MetadataMetadata
BehaviorsBehaviors(dissemintaors)(dissemintaors)
Data StreamsData StreamsSMAP1 - StrMap (TOC)SMAP1 - StrMap (TOC)
FLV1-DownloadFLV1-Download
MOV1-Quicktime StreamMOV1-Quicktime Stream
PDF1 - TranscriptPDF1 - Transcript
XACML PolicyXACML Policy
ARCH1 - Archival MasterARCH1 - Archival Master
DescriptiveDescriptive
TechnicalTechnical
SourceSource
RightsRights
Digital ProvenanceDigital Provenance
MD StructureMD Structure
Large Files: Large Files: The Digital Video ObjectThe Digital Video Object
14
Archival MastersArchival Masters
Film transfer/VHS/Betacam/Analog SourcesFilm transfer/VHS/Betacam/Analog Sources
Uncompressed AVI as MasterUncompressed AVI as Master
Digital (DVCAM, miniDV, HDV, etc.)Digital (DVCAM, miniDV, HDV, etc.)
• DV file with native format/encodingDV file with native format/encoding
• DVDsDVDs
• Disk Image (ISO) kept as archive MastersDisk Image (ISO) kept as archive Masters
Large Files: Large Files: The Digital Video ObjectThe Digital Video Object
15
Presentation StreamsPresentation StreamsStreaming Protocol: MPEG-4 H.264 Streaming Protocol: MPEG-4 H.264 (Quicktime)(Quicktime)
15-20 frames/sec, 640x48015-20 frames/sec, 640x480
Data rate: 384-512 kbpsData rate: 384-512 kbps
Progressive DownloadProgressive Download
• Flash Video (FLV with SWF wrapper)Flash Video (FLV with SWF wrapper)
• 15 fps, 640 x 480 - 256-384 kbps15 fps, 640 x 480 - 256-384 kbps
• Intended as an alternate presentation modeIntended as an alternate presentation mode
Large Files: Large Files: The Digital Video ObjectThe Digital Video Object
16
Presentation StreamsPresentation Streams
Codec Selection CriteriaCodec Selection Criteria
Easy to Use/InstallEasy to Use/Install
Supports Multiple PlatformsSupports Multiple Platforms
Built on established industry standardsBuilt on established industry standards
Excellent quality/bandwidth ratioExcellent quality/bandwidth ratio
Widely Used (Akamai, YouTube, etc)Widely Used (Akamai, YouTube, etc)
Large Files: Large Files: The Digital Video ObjectThe Digital Video Object
17
Content DemoContent Demo
Demo Video Clip from Existing Demo Video Clip from Existing CollectionsCollections
Large Files: Large Files: The Digital Video ObjectThe Digital Video Object
18
Presentation StreamsPresentation Streams
Open Source-able architectureOpen Source-able architecture - - Darwin Streaming Server (DSS)Darwin Streaming Server (DSS)
Can run on all major OS platformsCan run on all major OS platforms
Free, open source license (APSL)Free, open source license (APSL)
Large Files: Large Files: The Digital Video ObjectThe Digital Video Object
19
Presentation StreamsPresentation Streams
TranscriptsTranscripts
When availableWhen available
Searchable & IndexableSearchable & IndexableText/XMLText/XML
Will require labor Will require labor (transcribing & data entry)(transcribing & data entry)
Application ArchitectureApplication Architecture
20
Backend StorageBackend StorageSubsystemSubsystem
Digital ObjectDigital ObjectRepositoryRepository(FEDORA)(FEDORA)
Partner/Custom PortalsPartner/Custom PortalsMiddlewareMiddleware
(Host Server/Shibboleth)(Host Server/Shibboleth)
Archival MasterArchival MasterFilesFiles
Darwin Streaming ServerDarwin Streaming Server(Video Content Delivery)(Video Content Delivery)
Progressive DownloadProgressive Download(Flash Video)(Flash Video)
Retrieve Streams in Retrieve Streams in FEDORA via NFSFEDORA via NFS
- External Storage- External Storage- Checksum Integrity Validation- Checksum Integrity Validation
Quicktime/H.264 StreamQuicktime/H.264 Stream
The Front-End InterfaceThe Front-End Interface
Usage ScenariosUsage Scenarios
Faculty Annotation for Lectures on Faculty Annotation for Lectures on DemandDemand
Specific clips, segments for disseminationSpecific clips, segments for dissemination
• General Public Access - NJVid CommonsGeneral Public Access - NJVid Commons
• Student Access to Licensed VideosStudent Access to Licensed Videos
• A/A required for selective accessA/A required for selective access
End User ScenariosEnd User Scenarios
22
WorkflowWorkflow
Instructor previews video and selects segments Instructor previews video and selects segments for studentsfor students
Lectures on Demand:Lectures on Demand:Annotated Video ClipAnnotated Video Clip
23
WorkflowWorkflow
Instructor previews video and selects segments Instructor previews video and selects segments for studentsfor students
Lectures on Demand:Lectures on Demand:Annotated Video ClipAnnotated Video Clip
24
Instructor authenticates, and creates annotation Instructor authenticates, and creates annotation objects for the video, specifying in and out objects for the video, specifying in and out pointspoints
WorkflowWorkflow
A structure map with appropriate time segments A structure map with appropriate time segments and persistent ID of the source video is created. and persistent ID of the source video is created. All pieces fall into place.All pieces fall into place.
Lectures on Demand:Lectures on Demand:Annotated Video ClipAnnotated Video Clip
25
Object IDObject ID
MetadataMetadata
BehaviorsBehaviors
Data StreamData Stream
TOCTOC
Video for Class Video for Class 101:01101:01
Structure MapStructure MapIntro: 0:00.00 - 11:01.29Intro: 0:00.00 - 11:01.29History: 20:00.09 to 45:08.10History: 20:00.09 to 45:08.10Wrap-Up: 65:22.24 to Wrap-Up: 65:22.24 to 70:01.1870:01.18
DarwinDarwinStreamingStreaming
ServerServer
Presented Video Presented Video ObjectObject
Finalization of Video Architecture Finalization of Video Architecture (Summer 2008)(Summer 2008)
Implement Shibboleth for Authentication and Implement Shibboleth for Authentication and Authorization (Fall 2008)Authorization (Fall 2008)
Fine-Grained object access via XACML (Fall Fine-Grained object access via XACML (Fall 2008)2008)
Video Table of Contents (early to mid 2009)Video Table of Contents (early to mid 2009)
Multiple Structure Map Annotations for Complex Multiple Structure Map Annotations for Complex Segments (mid 2009)Segments (mid 2009)
NJVID Custom Installation ConfigurationNJVID Custom Installation Configuration
Development Needs Development Needs for NJVIDfor NJVID
Questions?Questions?
Isaiah BeardIsaiah BeardDigital Standards & Workflow Manager Digital Standards & Workflow Manager
Scholarly Communication CenterScholarly Communication Center
Rutgers University LibrariesRutgers University Libraries
Internet2 Spring Member MeetingInternet2 Spring Member MeetingArlington, VA - April 23, 2008Arlington, VA - April 23, 2008
Watch This Space!Watch This Space!
View our blog, sample View our blog, sample videos and progress videos and progress reports at:reports at:
www.njvid.org
Internet2 Spring Member MeetingInternet2 Spring Member MeetingArlington, VA - April 23, 2008Arlington, VA - April 23, 2008