a quint: cross information sharing and integration for the intelligence community

1

A Quint: Cross Information Sharing and Integration for the Intelligence Community

NCOIC-NGA Kickoff MeetingBrand Niemann

Director and Senior Enterprise Architect – Data ScientistSemantic CommunityDecember 15, 2011

It is not just where you put your data (cloud), but how you put it there!

2

Preface

• Prepared for:– Letitia Long, Director, National Geospatial-Intelligence Agency– Todd Myers, Chief Technology Advisor, NSG Expeditionary

Architecture, National Geospatial-Intelligence Agency– Gus Hunt, CTO, Central Intelligence Agency– George Strawn, Director, National Coordination Office, OSTP,

Executive Office of the President– Aneesh Chopra, Federal CTO, Executive Office of the President– Wyatt Kash, Editor in Chief, AOL Government– 13th SOA for eGov Conference, MITRE, McLean, VA, April 3,

2012

3

Two Recent Stories

• Intell agencies rethink old ideas about information sharing (Quint)– Federal Computer Week, November 28, 2011.

• GEOINT community hammers out integration strategies: Highlights from GEOINT 2011 Symposium– DefenseSystems.com, November 2011, page 14.

4

GEOINT 2011http://geoint2011.com/

http://semanticommunity.info/A_Quint:_Cross_Information_Sharing_and_Integration#GEOINT_2011_Symposium

5

The Quint

• Defense Intelligence Agency– http://www.dia.mil/

• National Geospatial – Intelligence Agency– https://www1.nga.mil/Pages/default.aspx

• National Reconnaissance Office– http://www.nro.gov/

• National Security Agency– http://www.nsa.gov/

• Central Intelligence Agency– https://www.cia.gov/

6

My Recent AOL Government Stories

• DoD Needs Data Science for Semantic Interoperability RFI and Audit Readiness Deadline• Army Cyber Warrior of the Future: Patriotic and Trustworthy Geeks Wanted• METI Adopts Desktop-As-A-Service Cloud Computing• Social Media - Six Degrees of Separation and Now Even Less• Army's Resource Forest Is Good Metaphor For System Of Systems Approach• Data.gov Geospatial Platform Needs 'Data Life Cycle' Management• DoD Seeks IT Interoperability With Enterprise Information Web• Big Data: Big Problem, Big Answer for the CIA• Build The NIEM Information Exchange Clearinghouse In The Cloud• In Search Of Practical Ways To Share IT Services In Government• Four Lessons Learned Using Dashboards in Government• Message At Amazon Cloud Summit: Provisioning Instead of Procurement• Using Social Media Comments To Speculate About Future Cyber Events• Cartwright, McGrath say semantic interoperability will drive DoD's information

environmenthttp://gov.aol.com/bloggers/brand-niemann/

7

Background

• March 1, 2010: Voice of the Customer – Working with NCOIC:– Todd Myers, National Geospatial-Intelligence

Agency, Vision (Pre-NSG Expeditionary Architecture)– Brand Niemann, Kevin L. Jackson's Seven Possible

Use Cases for the NIST Cloud Computing Business Use Cases Working Group in Support of the NCOIC-NGA Geospatial Community Cloud Demonstration (February 11, 2011) and GeoEye Pilot That Did the Seven Use Cases!

http://semanticommunity.info/Federal_Cloud_Computing/GeoEye

8

Background• Geospatial Summit Closing Keynote: Putting the Power of GEOINT in Your Hands,

September 13th, 2011, 3:15 pm - 4:15 pm– Letitia Long, Director, National Geospatial-Intelligence Agency

• As the nation's premier source of geospatial intelligence (GEOINT), the National Geospatial-Intelligence Agency (NGA), which is both a Department of Defense combat support agency and a member of the U.S. Intelligence Community, develops geospatial intelligence capabilities in support of U.S. national defense, disaster relief and intelligence objectives.

• To meet the 21st century challenges the nation faces today, NGA is working to provide the nation's warfighters, intelligence community and first responders with the tools they need to meet those challenges head on. To do this, NGA is focused on fundamentally changing the way it develops and provides geospatial intelligence to it users. This GEOINT Vision focuses on two principal, near-term goals to put the power of GEOINT in the hands of users: 1) Providing online, on-demand access to NGA's GEOINT knowledge and 2) creating new value by broadening and deepening NGA's analytic expertise.

• Hear from Director Letitia Long about NGA's mission and the GEOINT Vision in the conference's closing keynote address.– She asked to meet attendees afterwards with Advanced Analytical Capabilities and I did!

http://geospatialsummit.com/events/2011/sessions/tuesday/closing-keynote.aspx

9

Background• Self-Service Geospatial Cloud Computing for Government

– The challenge is to make government geospatial work self-service in the cloud so it is easier and less expensive. The new Data.gov lacks tools to enable users to work with the geospatial and non-geospatial data sets together. This presentation and tutorial addresses that need in a very practical and easily understood way and empowers government and non-government attendees to do their own self-service geospatial cloud computing.

• Geospatial Cloud Computing Applications for Data.gov and NCOIC/NGA Community Cloud Demonstration:– The challenge is to move geospatial work to the cloud. This presentation will feature several leading cloud

technology tools for doing that and some results for Data.gov and the NGA/NCOIC Demonstration Project. The possible use cases for the NIST Cloud Computing Business Use Cases Working Group will be demonstrated and discussed as an aid to promoting this kind of innovation in your organizations. This breakout presentation will include three other recognized experts. See background at GeoEye

• Geospatial applications for Data.gov and NGA/NCOIC that address the IAC/ACT Geospatial Committee Action Items:– NCOIC/NGA Community Cloud Demonstration– Possible Use Cases for the NIST Cloud Computing Business Use Cases Working Group in Support of the NCOIC-NGA

Geospatial Community Cloud Demonstration. See GeoEye– Presentors:

• Kevin Jackson (Mel Greer)• Jon Polay• Wendy Henry (John Garaghty)• Brand Niemann

http://semanticommunity.info/FOSE_Institute/GeospatialNote: Accepted but not given due to conference shortening.

10

Background

http://semanticommunity.info/A_Quint:_Cross_Information_Sharing_and_Integration#Slide_2_Outline

October 18, 2011, Todd Myers, AWS Gov Cloud Summit II

11

NGA Preferences for NCOIC Pilot

• Capacity versus Capability– Capacity is cloud infrastructure– Capability is user functionality interface and use

cases• NCOIC Preference Previously Was Capacity• NGA Prefers Capability Emphasis Now• Consider Implications for Three Spirals of Pilot

Source: NCOIC Geospatial Interoperability Team Telcon, December 9, 2011.

12

TIBCO Software Cloud Platforms• Business Events:

– Complex Event Processing– Enterprise Service Bus– Business Activity Monitoring

• iProcess Suite (Dynamic)– http://media.tibco.com/flash/bpm-plus/index.html

• Silver Spotfire– Interactive information visualization and analytic solutions

• https://silverspotfire.tibco.com/us/home

• Collaboration– Tibbr

• http://www.tibbr.com/

Business Events: http://media.tibco.com/flash/cep/business-events/index.html

13

Spotfire on the iPad!

http://spotfire.tibco.com/ipad.aspx

14

Spotfire Help

http://stn.spotfire.com/stn/Site/News.aspx

15

16

Spotfire World and State Boundaries

Interoperability Interface:SeeSortFilter SearchDownload (Web Player)Share (iPad)

17

Interoperability Interface• Defined: The means by which components of the smart grid can talk

to each other, for example, or by which electronic health records can be shared and added to by many parties – are an important stimulus to technology innovation and adoption. Optimally, these interfaces would be open: anyone may create products that use the interfaces without paying fees; and a public, transparent process is used to establish and revise the standards that define the interfaces.

• Spotfire Functions:– Map Settings and Manage Relations– Data Relationships– Information Designer– Federation

http://semanticommunity.info/A_NITRD_Dashboard/Designing_a_Digital_Future#Crosscutting_Themes

18

Spotfire Map Settingsand Manage Relations

19

Spotfire Data Relationships

20

Spotfire Information Designer

http://semanticommunity.info/Build_DoD_in_the_Cloud/Enterprise_Information_Web_for_Semantic_Interoperability_at_DoD/Spotfire_Information_Designer

21

Spotfire Data Federation

http://semanticommunity.info/Build_DoD_in_the_Cloud/Enterprise_Information_Web_for_Semantic_Interoperability_at_DoD/Spotfire_Data_Federation

http://ondemand.spotfire.com/public/library.aspx?folder=Users/FAMIEVL-91915/public

Search

22

Federation of Data Sets Strategy• One Table:– Two Columns

• Example: Column 1: Section and Column 2: URL– Note: A Column 3: Description could be in the URL

– Three Columns:• Example: Column 1: Subject, Column 2: Object, and Column 3: Predicate

– Note: This is the Semantic Web’s Linked Open Data Cloud as Linked Open Data for Network Analytics!

– Four Columns:• Examples: Column 1: Subject, Column 2: Attribute, Column 3: From, and

Column 4: To, or Column 1: City, Column 2: Country, Column 3: Longitude, and Column 4: Latitude– Note: This is the format for Spotfire’s Network Analytics Module developed for

the CIA

23

My Best Strategic Advice to You Is:Service Steps Tool Process

1. Provision Instead of Procure

Amazon Federal (secure) Web Services

AWS’s Migrating Applications to the Cloud

2. Linked Data on Mobile Devices for the Warfighter

MindTouch, Excel, SIRA, & Spotfire

Build DoD in the Cloud

3. Dynamic Case Management for Business Agility

Be Informed Be Structured

Demos Title URL

AWS Gov Cloud Summit II AWS Provision and Migrate Web

Army Weapons 2011 Knowledge and Capability Management Services for ASA(ALT) SOSE

Web

SemTech 2011 Washington DC

So Our Veterans Can Be Informed

Web

24

System of Systems Architecture

SSemantic Index ofLinked Data(e.g. Excel)

Dynamic Case Management (e.g. Be Informed)

Data Science Library (e.g. Spotfire)

Data Science Products (e.g. Spotfire)

25

Applying Data Science to Build a Knowledgebase

• Knowledgebase = Model + Instances• Model = Vocabulary, Taxonomy, and

Ontology/Rules• Instances = Linked Data Semantically Linked to

the Model

26

Applying Data Science to Build a Knowledgebase

• Model =– Vocabulary – Glossary in MindTouch– Taxonomy - Contents and Resources in MindTouch– Ontology/Rules in Be Informed

• Instances =– Linked Data Semantically Linked to the Model –

MindTouch, Excel and Spotfire

27

Business Process• Best Content to a Knowledgebase (MindTouch)– GEOINT 2011 Symposium, The Quint, NSG Expeditionary

Architecture, NCOIC Geospatial Interoperability Task Team, CIA World Fact Book, etc.

• Knowledgebase to Spreadsheet (Excel)– Linked Data to Subparts of the Knowledgebase

• Excel to Dashboard (Spotfire)– Data integration and interoperability interface

• Spotfire Dashboard to Dynamic Case Management (Be Informed)– Structured process for updating data in the dashboard

28

NCOIC Geospatial Interoperability Task Team

https://www.ncoic.org/apps/org/workgroup/geospatial_inter_task_team/documents.php

29

CIA World Fact Book

https://www.cia.gov/library/publications/the-world-factbook/index.html

30

Best Content to a Knowledgebase (MindTouch)

Well-defined URLs tosubparts of the documentand figure and tables.

http://semanticommunity.info/A_Quint-Cross_Information_Sharing_and_Integration

31

Best Content to a Knowledgebase (MindTouch)

http://semanticommunity.info/CIA_World_Factbook

The Entire Fact Book asLinked Open Data:About (4)References (6)Appendices (7)FAQs (1)Country Profiles (277)

32

Knowledgebase to Spreadsheet (Excel)

http://semanticommunity.info/@api/deki/files/15989/=Quint.xlsx

33

Excel to Dashboard (Spotfire)

PC Desktop

34

Excel to Dashboard (Spotfire)

Web Player

See the DataSort/Facet Search the DataDownload the DataShare the Data (iPad)

35

Spotfire to Be Informed• Be Structured Questions:

– Greenfield: Are you starting the report from scratch?– Legacy: Are you updating the report?– Dynamic Case Management: Does Spotfire need to be updated

frequently?• So the Business Process Pattern Includes:

– Greenfield, Legacy, and Dynamic Case Management• Definitions (and Rules!):

– CIA World Fact Book Definitions and Notes, etc.• Executable Model:

– Semantically Linked Taxonomy, Ontology, and Rules (started in previous slides)

36

SIRA: Builds a simple report and an ontology of the content

http://semanticommunity.info/@api/deki/files/15990/=Information_about_SIRA_regarding_TIBCO_Patterns.pdf

37

Recorded Future: Using Social Media Comments To Speculate About Future Cyber Events

http://gov.aol.com/2011/08/18/using-social-media-comments-to-speculate-about-future-cyber-even/

38

Summary

• Intelligence information can be integrated with Linked Open Data in Spotfire

• Linked Open Data supports cross-information sharing and integration app development

• The power of GEOINT can be put in the hands of the user

• The bathtub of traditional collection, analysis, and communication can be inverted