computer assisted manufacturing

PISAProduction, Indexing and Search

of Audio-visual Material

Computer Assisted ManufacturingDieter Van Rijsselbergen, IBBT – MMLab – UGent

35

PISA - Overview

Abstract

Information

Footage

Concept

Virtual

Model

Model Driven Development:

• Setting (Stage properties, light)

• Character

• Synthetic Speech

• Sound effects

• Character animation

• Virtual camera

VirtualModelling

AutomatedProductionRealisation

• Ingest

• Editing

• Mastering

• Reproduction to alternative distribution channels

Computer Assisted Design

Computer Assisted Manufacturing

Script Editing

• Parse scenario

• Shooting script editor

• Storyboard

Script Editing

Reverse Engineering

• Shot segmentation

• Video footprint and reuse detection

• Biometric face detection

• Background analysis

• Speech-to-text

Interpretation

• Character identification

• Background categorisation

and identification

• Topic and eventdetection

Intelligent Analysis and

Quantization

Quantization

Analysis

Indexing

Retrieval

• Timecode based indexing

• Geo-temporal reference

• Taxonomy based indexing and search

• Facetted search

Search Engine

36

Computer Assisted Manufacturing

! Computer Assisted Manufacturing is about automation

of media production processes

! Based on extensive production metadata

! high level Product Engineering for narratives

! low level Manufacturing Engineering for actual

manufacturing specifications

! Using clever algorithms to fill in the gaps

! supported by Trouvaille

37

Drama Manufacturing Processes

Product Engineering

Manufacturing Engineering

Manufacturing

Synopsis

Writing

3-D

Previsualization

Dialogue

Writing

Shooting

ScriptingEditing

Quality

Assurance

Acquisition? ? ?

38

Objectives

! Extend manufacturing workflow with 3-D previsualization

! Build evolving products

! Re-use previsualization parameters

during manufacturing

! Increase efficiency and automation in production

processes

! 1) Rough Cut Editing based on 3-D previsualization

! 2) Repurposing for multiple output channels

Computer Assisted

Manufacturing

Rough Cut Editing

40

Rough Cut Editing

! Inclusion of 3-D previsualization into thedrama manufacturing process

! Use previsualization as shooting script

! Integration into the MAMS and Craft Editing systems

! Building evolving products

!mixture of previsualized and shot footage

! evolve from all-previsualized to all-shot composition

41

Rough Cut Editing Processes

Product Engineering

Manufacturing Engineering

Manufacturing

Synopsis

Writing

Previz.

Rendering

Dialogue

Writing

EditingQuality

Assurance

Acquisition

3-D

Previsualization

Computer Assisted

Manufacturing

Repurposing

43

Computer Aided Repurposing (CAR)

! Automated Repurposing of Drama

! Different output channels

!HDTV

!SDTV

! iPod/Cellular

! True Universal Media Experience

! requires semantically sensible scaling/cropping

44

Repurposing Concept

! Automated Repurposing of Drama

! Spatial adaptation using semantic Objects of Interest

! as in “focus on Capt. Kirk”

! decided by creative staff in function of the narrative

! is a form of Manufacturing Engineering

! can be defined in early 3-D Previsualization

! preparation not only of shooting script

but also of adaptations

! parameters refined in Quality Assurance

45

Repurposing: How?

! 1) Determine where the semantic elements are

! Semantic Objects of Interest

are linked with regions of interest

!manual annotation

! automated

algorithms

! aided by

Product

Engineering

metadata!Jenny Waldek

46

Repurposing: How?

! 2) Provide reframing instructions, e.g.:

! Object Orientation and Range

! “The Bad must be shown on the right…”

! Object Size

! “… and I want it to be a full shot of The Bad.”

0.75

Frame SizeObject of Interest

Size: (30%, 90%)

47

Repurposing: Channels

! HDTV

! Originally acquired

high definition material

! SDTV

! Legacy applications

! Common pan/scan

! Editing

(AVID)

! Mobile (iPod)

! Semantically

repurposed

48

Repurposing Processes

Product Engineering

Manufacturing Engineering

Manufacturing

Synopsis

Writing

3-D

Previsualization

Previz.

Rendering

Dialogue

Writing

EditingQuality

Assurance

AcquisitionEDL

RenderingRepurposing

Repurposing

Scripting

49

Workflow (1): Synopsis

50

Workflow (2): Management

51

Workflow (3): 3-D Previz. – Scenario

52

Workflow (4): 3-D Previz. – Shooting Script

53

Workflow (5): 3-D Previz. Editing

54

Workflow (6): Acquisition

55

Workflow (7): Editing

56

Workflow (8): Quality Assurance & Analysis

Plane

57

Workflow (9): Repurposing

58

Workflow (10): Conform & Publish

59

Computer Aided Manufacturing: Workflow