“challenges in achieving optimum fmv...

31
“Challenges in Achieving Optimum FMV Capabilities” What’s Broken, How Do We Fix It, and What Will Future Look Like? November 2011 Stephen W. Long Director, ISR and BMC2 Integration and Technology Northrop Grumman Aerospace, Melbourne, Florida [email protected] Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Upload: volien

Post on 22-Jun-2018

223 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

“Challenges in Achieving Optimum FMV Capabilities”

What’s Broken, How Do We Fix It, and What Will Future Look Like?

November 2011

Stephen W. Long Director, ISR and BMC2 Integration and Technology

Northrop Grumman Aerospace, Melbourne, Florida [email protected]

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 2: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Prologue: Wars Have Changed, ISR Needs to Change

•  “The 20th Century is dead…get over it!” –  MG John Custer, Commanding General of Ft Huachuca

•  Early in the current wars in Iraq and Afghanistan, it was recognized that warfare has evolved and is different than the “force-on-force” constructs that drive traditional DoD planning

–  Looking for individuals that blend into the populace –  Looking for patterns of life / networks that support terrorists and insurgents

•  Operations Drive Intelligence, Surveillance, Reconnaissance (ISR) - ISR Drives Operations

–  ISR has become the LIMFAC for missions…

•  Full Motion Video (FMV) has emerged as a critical enabler for modern ISR support to Warfighters (and Analysts & First Responders and …)

2 Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 3: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Overview and Introduction

•  Looking back over the past ~15 years of the ~invention, initial deployments and now wide deployment of Motion Imagery (Full Motion Video - FMV), one gains a bit of perspective

–  Today’s talk will look at where the FMV community has come from, discuss ongoing challenges in achieving optimum FMV capabilities, and start a dialog on where the community needs to go…What are the Big things we need to fix / invent?

•  In speeches last year I described and Defined FMV Eras / Epochs in terms of FMV 1.0, FMV 2.0, FMV 3.0…

–  Boundaries between Epochs have been defined by on-going revolutions in the FMV, Information Technology, and ISR worlds -

•  Analog to Digital, Digital to Data…Standard Definition to High Definition to ….No Metadata to Fully Enabled Metadata…Vertical Stovepipe to Horizontally Integrating Systems…

–  As I prepared this talk today, friends requested I restate Stephen's FMV Laws (Modeled on Murphy’s Laws) – so these “laws” are interspersed throughout this talk. They are meant to be humorous - You are allowed to laugh!

•  So, What’s Next? – What Will FMV 2.5, then FMV 3.0 and then beyond look like?

3 Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 4: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Epochs: FMV 1.0 (1995 -2007)

4

Past // Present

FMV1.0

Analog 480i Sensor

No Metadata /

Limited Analog Metadata / 1st Gen.

Digital Metadata

Analog Processing Analog to MPEG-2

Analog to H.264 1st Gen Digital PED

•  Video? We don’t need no stinking Video •  ("Badges? We don't need no stinking badges! “ Blazing Saddles/1974)

•  “We don’t do video here at NIMA” •  (NIMA Acquisition O-6, circa 1997)

•  Metadata? What’s Metadata? •  “The USAF has no need for any machine

readable data for the Predator drone” •  (USAF ACC O-6, circa 1998)

•  “Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has.”

•  Margaret Mead US anthropologist (1901 - 1978)

•  The Mission of the Motion Imagery Program Office, should you decide to accept it…

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 5: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Stephen's FMV Laws 1-6 [FMV 1.0]

•  Law #1. Delivered FMV Quality is Determined by the Least Informed Engineer in the Collection, Processing, Exploitation, Dissemination (CPED) system creation chain

•  Law #2. Interlace is Evil, Analog is Bad, Digital is Good –  Can’t negotiate w/ Evil – must progressively fight it - Analog still exists – See Law 1

•  Law #3. Miracle Compression Rates Require Miracles –  Quality FMV Requires BW - Interlace & Analog FMV waste BW – See Law #2 –  If your FMV Quality is poor, See Law #1

•  Law #4. Temporal Compression Improves Image Quality –  If you don’t know that Law #4 is actually true, you are a Still Imagery Engineer, Not

a Motion Imagery Engineer •  Law #5. Persistent Surveillance Systems are Designed by Still Imagery Engineers

–  If you doubt Law 5, See Law 4; If you doubt Law 4, See Law 1 •  Law #6. Metadata is Good

–  If you don’t have Metadata, See Law #1. If you don’t know what Metadata is, you are probably the Boss of the Engineer in Law #1

–  Metadata is fundamental – It enables everything in the Modern World –  Video without Metadata is just watching rabbit ears TV

5 Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 6: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Epochs: 1.0 – What’s Broken, How Do We Fix It?

6

Community Has Been “Stuck” in FMV 1.0 for FIFTEEN YEARS!!!

•  Summary Recommendations dating back to 1990’s –  We Must Stop The Analog Madness

–  NTSC / RS-170A Specification Written in 1956!! Vacuum Tube Era –  Must move to all digital processing, with digital metadata

–  Motion Imagery Standards Board set the standards for the Digital Future of FMV back in 1997!

–  Must tackle the Sensors - Many sensors already have analog AND digital outputs – feed digital signals to H.264 encoders

–  Install H.264 encoders on UAS – sooner rather than later - Flight qualified H.264 encoders are available COTS (catalog items)

–  STOP any further deployment of RS-170a Sensors - Digital 480p / 720p sensors available at same cost / SWAP – 1080p mainstream

–  STOP use of RS-170a (analog interlace video) for interconnection nodes - RS-170A should not be used as the lowest common denominator, especially for Information Assurance problems

–  STOP burning in Electronic Support Data (ESD) – Ruins Image Quality, Ruins ability advanced image processing – BURN IN IS NOT NECESSARY – overlay Metadata at point of consumption – not point of origin!

Past // Present

FMV1.0

Analog 480i Sensor

No Metadata /

Limited Analog Metadata / 1st Gen.

Digital Metadata

Analog Processing Analog to MPEG-2

Analog to H.264 1st Gen Digital PED

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 7: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Standard Definition (Interlace) vs. Enhanced Definition (Progressive Scan) Comparison

7

480i 480p

Analog / Interlace Digital / Progressive

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 8: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Metadata - What Problem Are We Trying to Solve (Did We Solve)?

8

291M ANC Packet AES3

Packet MPEG-2 TS Packet

RTP / UDP / IP Packet

Other Packet(s)

Segmentation / Extension Options

Essence Or

Metadata Application

Layer (Dictionaries)

Transport Layer

SMPTE Metadata Dictionary Contents

SMPTE Metadata Dictionary Structure

Essence Types

(1, 2, …)

Essence Types

(3, 4, …)

Metadata Types

(1, 2, …)

Metadata Types

(3, 4, …)

This is the UNIVERSAL Interoperability Problem! How Do You Get X number of Applications to Interact With Y number of Transports? Next harder problem – How Do You Get People to Agree? How Do You Share Data With People You have Never Met? Even harder problem – How Do You Manage FORWARD Interoperability – enable new changes without breaking every existing application – including those applications that you have Never Met?

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 9: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Key-Length-Value (KLV) Protocol As Solution

9

291M ANC Packet AES3

Packet MPEG-2 TS Packet

RTP / UDP / IP Packet

Other Packet(s)

Segmentation / Extension Options

Essence Or

Metadata Application

Layer (Dictionaries)

Transport Layer

SMPTE Metadata Dictionary Contents

SMPTE Metadata Dictionary Structure

Essence Types

(1, 2, …)

Essence Types

(3, 4, …)

Metadata Types

(1, 2, …)

Metadata Types

(3, 4, …)

Concise KLV Protocol for Very Low Value Lengths

Key - Length -Value (KLV) Protocol K-L-V Group/Pack Strategies

Value Types

KLV was invented to address this UNIVERSAL Interoperability Problem! KLV enables X number of Applications to Interact With Y number of Transports – and it enables FORWARD Interoperability It took ten years of effort to get KLV standardized and KLV first generation systems built - KLV is now a widely installed International Standard and it is embedded across the pro media universe and DoD / IC / NATO communities

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 10: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Matrix Analysis: Updating Key Architecture Elements Yields Significant System of System Improvements (Proof of Stephen’s Laws 1 - 6)

10

Sensor Model EO //

IR // LD

Sensor FMV WF

over Interface

Sensor Metadata Type over Interface

D LOS Metadata Type over Interface

D LOS FMV WF

over Interface

D LOS Compress Sys // Type

over Transport

D LOS Transport over Sys //

Band

D LOS Primary Receiver

(GCS)

D LOS Decompress

over Transport

D LOS Pri Rec Out: FMV WF

over Interface

D LOS Pri Rec Out: Metadata Type over Interface

Next Node Metadata Type over Interface

Next Node FMV WF

over Interface

Next Node Compress Sys // Type

over Transport

Next Node Transport over Sys //

Band

MX-15Di

RS-170A Over Comp

Vid

CoT Over

RS-232

RS-232 //

CoT

______ // MJPEG

over ______

____ Over SATCOM //

Ku

D LOS Primary Receiver

(GCS)

MJPEG to RS-170A

over Comp Vid

CoT Over

RS-232

CoT over RS-232 to

Line 21

H.264 SD // KLV PDS over UDP/

IP

Piranha Compress

// DVB-RCS

RS-170A Over Comp

Vid

D LOS to Pri Rec

D to A

RS-170A Over Comp

Vid

D LOS Rec Out to Next

A to D

RS-170A Over Comp

Vid

03 17 18 20 23 25 26

02 16 19 21 22 24 27

01

03 17 18 20 23 25 26

02 16 19 21 22 24 27

01

Matrix Row C

Sensor to D LOS

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 11: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Matrix Analysis: Updating Key Architecture Elements Yields Significant System of System Improvements (Proof of Stephen’s Laws 1 - 6)

11

Sensor Model EO //

IR // LD

Sensor FMV WF

over Interface

Sensor Metadata Type over Interface

D LOS Metadata Type over Interface

D LOS FMV WF

over Interface

D LOS Compress Sys // Type

over Transport

D LOS Transport over Sys //

Band

D LOS Primary Receiver

(GCS)

D LOS Decompress

over Transport

D LOS Pri Rec Out: FMV WF

over Interface

D LOS Pri Rec Out: Metadata Type over Interface

Next Node Metadata Type over Interface

Next Node FMV WF

over Interface

Next Node Compress Sys // Type

over Transport

Next Node Transport over Sys //

Band

MX-15Di

RS-170A Over Comp

Vid

CoT Over

RS-232

RS-232 //

CoT

______ // MJPEG

over ______

____ Over SATCOM //

Ku

D LOS Primary Receiver

(GCS)

MJPEG to RS-170A

over Comp Vid

CoT Over

RS-232

CoT over RS-232 to

Line 21

H.264 SD // KLV PDS over UDP/

IP

Piranha Compress

// DVB-RCS

RS-170A Over Comp

Vid

D LOS to Pri Rec

D to A

RS-170A Over Comp

Vid

D LOS Rec Out to Next

A to D

RS-170A Over Comp

Vid

03 17 18 20 23 25 26

02 16 19 21 22 24 27

01

03 17 18 20 23 25 26

02 16 19 21 22 24 27

01

Matrix Row C

Sensor to D LOS

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 12: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Matrix Analysis: Updating Key Architecture Elements Yields Significant System of System Improvements (Proof of Stephen’s Laws 1 - 6)

12

Sensor Model EO //

IR // LD

Sensor FMV WF

over Interface

Sensor Metadata Type over Interface

D LOS Metadata Type over Interface

D LOS FMV WF

over Interface

D LOS Compress Sys // Type

over Transport

D LOS Transport over Sys //

Band

D LOS Primary Receiver

(GCS)

D LOS Decompress

over Transport

D LOS Pri Rec Out: FMV WF

over Interface

D LOS Pri Rec Out: Metadata Type over Interface

Next Node Metadata Type over Interface

Next Node FMV WF

over Interface

Next Node Compress Sys // Type

over Transport

Next Node Transport over Sys //

Band

MX-15Di

RS-170A Over Comp

Vid

CoT Over

RS-232

RS-232 //

CoT

____ Over SATCOM //

Ku

D LOS Primary Receiver

(GCS)

Ethernet // DVB-

RCS

RS-170A Over Comp

Vid

D LOS to Pri Rec

D to D

D LOS Rec Out to Next

D to D

03 17 18 20 23 25 26

02 16 19 21 22 24 27

01

03 17 18 20 23 25 26

02 16 19 21 22 24 27

01

Matrix Row C

UDP/IP Pass

Through

H.264 SD // KLV PDS over UDP/

IP

UDP/IP Pass

Through

UDP/IP Pass

Through

UDP/IP Pass

Through

UDP/IP Pass

Through

UDP/IP Pass

Through

Sensor to D LOS

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 13: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Epochs: FMV 2.0 (2008 - ?)

13

•  The Analog Madness Continues! •  The Unquenchable Thirst for FMV

•  ISR-TF Response •  The “thirty” Orbit JUON

•  We don’t need no stinking HD •  ("Badges? We don't need no stinking badges! “ Blazing Saddles/1974)

•  Yes Sir, Of Course We Will Deliver HD

Past // Present

FMV1.0

Analog 480i Sensor

No Metadata /

Limited Analog Metadata / 1st Gen.

Digital Metadata

Analog Processing Analog to MPEG-2

Analog to H.264 1st Gen Digital PED

Present / In Progress

FMV 2.0

Digital 720p HD Sensor

2nd Gen.

Digital KLV Metadata

H.264 All Digital Processing,

2st Gen. Digital PED

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 14: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

2008 - Gates Forms ISR-Task Force for Warfighters

•  Defense Secretary Robert M. Gates established the ISR-Task Force to ensure the DoD is doing everything possible to provide ISR assets to support warfighters…The ISR-Task Force will move the intelligence, surveillance and reconnaissance issue to the front burner as it explores “more innovative and bold ways to help those whose lives are on the line.”

–  Donna Miles American Forces Press Service, April 2008

•  ISR TF Priorities Included: –  Need for better ISR for these wars has grown, and can best be described as the

need to provide improved “Identity Resolution” –  We need higher resolution & fidelity, in both space & time, than in past

•  In many cases, looking for individuals that blend into the populace - looking for the patterns of life and networks that support terrorists…

–  FMV is a critical ISR tool for these searches

•  ISR TF and Services have quickly added additional FMV collection platforms, and significantly increased quantity of necessary PED

–  To directly support needed increases in Identity Resolution, ISR TF has supported upgrades to FMV sensors & PED systems to move from antiquated, analog video to fully digital cutting edge HD FMV - providing significantly improved FMV in 2010+

14 Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 15: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

ISR-TF: Solving Unquenchable Thirst (circa 2008)

15

C-12: LPA, MARSS, JAVAMAN

Long Endurance UAV: Max Prod. MQ-X

Specialized G-3 Aircraft

Lighter-Than-Air PTDS, PGSS, LEM-V

LIDAR Imaging

GMTI Collection SIGINT Collection

PED, PED and More PED

High Definition FMV

“The ISR-Task Force will move the intelligence, surveillance and reconnaissance issue to the front burner as it explores “more innovative and bold ways to help those whose lives are on the line.” Defense Secretary Robert M. Gates

ISR-TF has greatly accelerated production and deployment of Collection, Communications, and PED capabilities for OIF and OEF

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 16: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Image Quality Scale and Mission Impacts

16

Analog FMV Digital FMV (SD) Digital FMV (ED) Digital FMV (HD)

Video National Imagery Interpretability Rating Scale (V-NIIRS)

5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0

HVI

or C

OIN

En

gage

men

t

Dis

mou

nt T

rack

ing

Or O

ver w

atch

Vehi

cle

Trac

king

O

r Ove

r wat

ch

Isolate and track an individual pedestrian Meandering through a crowded open market

Track the movement of a car or SUV On an urban roadway in medium traffic

Detect individuals, when not in group

Isolate body and arm movement of a pedestrian using A handheld communications device (cell phone)

Isolate hand & forearm movement of an individual raising an assault rifle or large frame pistol to an aimed firing position. (MP5, AK74, M1911, UZI)

Track large groups of pedestrians (25+) crossing an intersection or meandering through an open marketplace

▲ ▲ ▲ ▲

Other High Resolution EEIs: * Coat Vs. Chest Rack * Hostage / No Hostage * Person Under Duress/Not * Rug or Weapon on Shoulder * Sentry Duty Vs. Shepherd

Military Age Male Vs. Women / Children Key Zone for Find, Fix, Finish Mission Impacts

Notes: NIIRS is LOG Scale – +1 NIIRS equals doubling standoff range +1 NIIRS equals doubling resolution

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Essential Elements of Information (EEI) High Value Individuals (HVI) Counter Insurgence (COIN)

Page 17: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

2001 720p HDTV “Serial 0” Sensor Demonstration

17

36,000’ Slant Range at 10,000 ft MSL

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 18: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Epochs (Revised)

18

Past // Present

FMV1.0

Analog 480i Sensor

No Metadata /

Limited Analog Metadata / 1st Gen.

Digital Metadata

Analog Processing Analog to MPEG-2

Analog to H.264 1st Gen Digital PED

Present / In Progress

FMV 2.0

Digital 720p HD Sensor

2nd Gen.

Digital KLV Metadata

H.264 All Digital Processing,

2st Gen. Digital PED

FMV 2.5

In Progress / Future

Digital 720p / 1080p HD Sensors

2nd Gen.

Digital KLV Metadata

(Stable & Ubiquitous)

H.264 All Digital Processing,

Better 2nd Gen. PED & Repository

Tools

Originally Envisioned FMV 3.0 As

“FMV As It Should Have Always Been”

Instead, to Meet Urgent Warfighter Needs Community Focused on Fast,

Much More Quantity (Right Decision)

Improved Quality in

some FMV ISR systems,

Net Result: Better FMV

(but not FMV 3.0)

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 19: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Epoch: FMV 2.5 (2009 - ?)

19

Analog Interlace

Digital Progressive à Digital

Interlace

Digital Cinema

4K

Digital Cinema

2K

1080p60

HDTV 1080p30 HDTV

720p60

HDTV 720p30 EDTV

480p60 EDTV 480p30 SDTV

480i30 NTSC

300 MP/s

250 MP/s

200 MP/s

150 MP/s

100 MP/s

50 MP/s

0

MTS-A / MX-15 HD

MTS-B / MX-20 HD

MTS-C Future Large

Aperture Gimbal

•  Truck Sized Targets –  NIIRS 6

•  1-3 Mbps Data Link •  Smallest Size/Wgt/Pwr

•  People Sized Targets –  NIIRS 7-8+

•  3-10 Mbps Data Link •  Moderate Size/Wgt/Pwr

•  Less than People Sized Targets - NIIRS 8-9

•  10-45++ Mbps Data Link •  Large Size/Wgt/Pwr

Standard Definition (legacy systems) (Consumer Video Class

Optics)

High Definition (systems in

pipeline) (Professional Video /

Scientific Class Optics)

Ultra High Definition (Future)

(Scientific Class, Large Optics)

“Sweet Spot”

FMV 2.5

In Progress / Future

Digital 720p / 1080p HD Sensors

2nd Gen.

Digital KLV Metadata

(Stable & Ubiquitous)

H.264 All Digital Processing,

Better 2nd Gen. PED & Repository

Tools

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 20: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

FMV Epochs – What’s Next? FMV 3.0: “FMV As It Should Have Always Been”

20

Past // Present

FMV1.0

Analog 480i Sensor

No Metadata /

Limited Analog Metadata / 1st Gen.

Digital Metadata

Analog Processing Analog to MPEG-2

Analog to H.264 1st Gen Digital PED

Present / In Progress

FMV 2.0

Digital 720p HD Sensor

2nd Gen.

Digital KLV Metadata

H.264 All Digital Processing,

2st Gen. Digital PED

FMV 2.5

In Progress / Future

Digital 720p / 1080p HD Sensors

2nd Gen.

Digital KLV Metadata

(Stable & Ubiquitous)

H.264 All Digital Processing,

Better 2nd Gen. PED & Repository

Tools

FMV 3.0

Digital 720p / 1080p / 2K / 4K Metric Sensors

3rd Gen. Digital KLV Metadata,

Targeting Enabled

H.264 All Digital Processing,

Advanced PED & Repository Tools

ISR Integration (TCSF3 / .ISR) &

Motion Intelligence / Activities Based

Intelligence

In Progress / Future

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 21: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

ISR-NExT™ -“ISR Nexus for EXcellence and Technology” Mission - “Envision the Future and Show the Way”

21

ISR-NExT™ core premise - all players are “stewards not owners” of intelligence data and technologies - we need integrated solutions across TCPED doctrinal framework boundaries – “One Team, One Fight” is our Motto

•  Create, integrate and demonstrate advanced technologies for ISR and Motion Intelligence systems to support, protect & defend…

•  ISR-NEXT ™ - Geographically diverse, network enabled “technology innovation, advancement and meeting place,” that will support the invention, advancement, application and understanding of ISR Integration and Motion Intelligence systems

•  Our initial technology development efforts will focus on Unifying Disparate Data

The Future Is Not Something That Just Happens – It Has to Be Invented! Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 22: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

What is the PLAN to unify DoD/IC disparate data types into a seamless whole?

22

This is the UNIVERSAL Interoperability Problem! How Do You Get X number of Intelligence Modalities to Interact With Y number of Users / Customers? How do you Unify disparate data within a Single INT? How do you Unify disparate data across different INTs? Even harder problems – How Do You Get People to Agree? How Do You Share Data With People You have Never Met? How Do You Manage FORWARD Interoperability – enable new data changes without breaking every existing application – including those applications that you have Never Met?

SIGINT S/GMTI MASINT LIDAR / HSI / MSI

IMINT / GEOINT /

FMV / WAAS

Other INTs…

SIGINT S/GMTI MASINT LIDAR / HSI / MSI

IMINT / GEOINT /

FMV / WAAS

Other INTs…

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 23: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Solution: .ISR/TCSF3 - Unified Exchange of Disparate Data

23

The underlying technology (Key-Length-Value: KLV) behind .ISR and TCSF3 was invented to address this UNIVERSAL Interoperability Problem! KLV (and its new children .ISR and TCSF3) enables X number of Applications to Interact With Y number of Transports – and it enables FORWARD Interoperability It took a decade of effort to get KLV invented, standardized and KLV / HD generation systems built - KLV is now a widely installed International Standard and it is embedded across the pro media universe and DoD / IC / NATO communities

SIGINT S/GMTI MASINT LIDAR / HSI / MSI

IMINT / GEOINT /

FMV / WAAS

Other INTs…

SIGINT S/GMTI MASINT LIDAR / HSI / MSI

IMINT / GEOINT /

FMV / WAAS

Other INTs…

.ISR and Tip-Cue-Slew-Find-Fix-Finish (all based on / derived from KLV Protocol)

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

.ISR / TCSF3

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 24: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Concept: Inventing Different Approach to ISR/BMC2 Responsive Tasking: Tip-Cue-Slew-Find-Fix-Finish

24

As Opponent touches any part of web, full ISR Web becomes Aware /

Reacts

ISR Web would link the Sensor Front Ends and

pass multi-mode Messages / Data

into the center (we are spider waiting in

the center of web)

Nodes need to perform machine to machine

Cueing which shapes better responses from

more and more of other ISR web nodes, enabled

By NRT “Messages” Passed around the

network: “Messages = Metadata”

ISR Web

• Tips [enable recognition of] Cues, • Cues [enable and cause] Slews,

• Slews [enable] Find, • Find [enables] Fix,

• Fix [enables] Finish…

•  Create / enable a next generation approach to ISR responsive tasking & messaging: A new Real-time message data language – Tip-Cue-Slew-Find-Fix-Finish (TCSFFF) to pass tips and cues and … across disparate ISR/BMC2 systems: GMTI to FMV, SIGINT to FMV, FMV to MASINT…

–  KLV Protocol is already in use across DoD/IC today!

–  DoD/IC/NATO ISR systems already partially “understand” KLV protocol because of FMV – KLV parsing is built into GCCS-I3, DCGS-USAF, DCGS-Army, OGA, NGA,..

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 25: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Concept: Universal Data Interchange Using Super Wrappers: .ISR - Super Wrapper for Disparate Essence

25

LAS (LIDAR) Implicitly

Understood Essence

NITF Header

JPEG Wrapped Essence

Super Wrapper for Disparate Essence .ISR

Disparate Essence, Duration, Object Relationships, Synchronization, History, etc.

H.264 FMV

JPEG 2000

WAAS

Stills

LAS (LIDAR)

GMTI Other GEOINT

SIGINT MASINT …

.ISR is a framework for managing all of the inputs and processes (value additions) in a multimedia work-flow and moving that value to

the next analysis node (or moving end product to the customer). Enables the tracking of all of the steps used in content creation / analysis and the logical, spatial, temporal relationships among

multimedia objects

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 26: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

.ISR Nested Capabilities

26

Unified File

Exchange

Motion Intelligence (Persistent Surveillance)

Data Management

Enable “Valued Added Paradigm” (Reinventing How We

Manage Intelligence Analysis)

LIDAR

Time

Annotation and Graphics Overlay

FMV / WAS

Previous Reporting

Still Image

Summary Report

SIGINT MASINT

GMTI

.ISR Implementations will grown as nested capabilities: Unified File Exchange can be enabled NOW, with immediate system benefits (single or multi-INT); Persistent Surveillance Data Management is different from legacy non-temporal data management – we must have better tools that understand concept of Duration; Real payoff is the Value Added Paradigm -

New fundamental approach on how we improve Intelligence Analysis

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 27: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

.ISR in Action - Value Added Paradigm: Propagating “Value” - Increasing Value Over Time (Six Cs and counting)

27

Processing Disparate Data Types “Raw” Data/Metadata “Improved” Data/Metadata

Processing Processing Processing

Add Value Add Value Add Value Forward

Value Forward

Value

.ISR .ISR .ISR

Unified Data Types

Collect (Gather)

Pull Multiple

Sources Together For Subsequent

Processing

PROCESSING

Conflate (Combine)

Layer and Combine

Information

PROCESSING

Condense (Cut / Trim)

Spatially,

Temporally Reduce

(pointers or Actual)

Unnecessary Sub-Elements

PROCESSING

Correct (Normalize)

Enable Sub-

Elements To be spatially,

temporally aligned (such as

Ortho rectification)

PROCESSING

Composite (Cartograph

Onto Common Display) Visualize

Information Within COP

PROCESSING

Communicate

Forward Composite Value to Next Node

Or to Customer

PROCESSING

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 28: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Stephen's FMV Laws 7-10 [FMV 3.0…?]

•  Law #7: FMV 2.5 = FMV 2.0 + Integral [Tip, Cue, Slew, Find, Fix, Finish] –  Law # 7 Not Possible Without Law #6 (Metadata)

•  Law #8. .ISR Enables the “Value Added Paradigm” –  Value Added Paradigm - New fundamental approach on how we improve

Intelligence Analysis

•  Law #9. Motion Intelligence Is Superset of {FMV, WAS, GMTI, LIDAR,…} –  2+2=5 –  Integration of Motion Intelligence is (will be) a profound path-finding experiment

•  Law #10. Law 7 + Law 8 + Law 9 = Unifying Framework Across Intelligence Disciplines (Future of ISR)

–  Future of ISR = FMV 2.5 + Integral [TCSF3] + .ISR + VAP + Integral [MotionINT]

28

“It’s kind of fun to do the impossible”

Walt Disney

“We keep moving forward, opening new doors, and doing new things, because we're curious

and curiosity keeps leading us down new paths.” Walt Disney

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 29: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Closing: Challenges in Achieving Optimum FMV Capabilities”

•  What’s Broken, How Do We Fix It, and What Will Future Look Like? –  Stop the Analog Madness - Embrace Digital Processing, KLV Metadata, HD… –  Treat FMV With Respect – Build “FMV As It Should Have Always Been”

•  Favorite Quote: –  “The Best Way to Predict the Future is to Invent It.” - Alan Kay, 1971

•  In the mid to late 1990s, we envisioned where we wanted FMV to be in the future – our predictions came true – we invented it! (the digital part)

•  Now is the time to envision and invent the next generation of capabilities - the new domain of Motion Intelligence

–  Law #10 = Future of ISR = FMV 2.5 + Integral [TCSF3] + .ISR + VAP + Integral [Motion Intelligence]

•  Together, we can build the future of ISR that our Nation needs: –  “Envision the Future - Show the Way” - Invent, build, and deploy fully integrated

ISR and Motion Intelligence systems to support, protect and defend…

29 Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11

Page 30: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com
Page 31: “Challenges in Achieving Optimum FMV Capabilities”geoint2011.com/.../NGC_Challenges_in_Achieving_Optimum_FMV_Cap… · Northrop Grumman Aerospace, Melbourne, Florida Stephen.long2@ngc.com

Stephen’s FMV Laws

•  Law #1. Delivered Intelligence Value is Determined by the Least Informed Engineer in the TCPED chain

•  Law #2. Interlace is Evil, Analog is Bad, Digital is Good •  Law #3. Miracle Compression Rates Require Miracles •  Law #4. Temporal Compression Improves Image Quality (True) •  Law #5. Persistent Surveillance Systems are Designed by Still Imagery

Engineers (historical norm) •  Law #6. Metadata is Good •  Law #7. FMV 2.5 = FMV 2.0 + Integral [Tip, Cue, Slew, Find, Fix, Finish] •  Law #8. .ISR Enables the “Value Added Paradigm” •  Law #9. Motion Intelligence Is Superset {FMV, WAS, GMTI, LIDAR,..} •  Law #10. Future of ISR = FMV 2.5 + Integral [TCSF3] + .ISR + VAP +

Integral [MotionINT]

Approved for Public Release: Northrop Grumman Aerospace Systems Case 11-1334 Dated 10/31/11