this conference brought to you by - big data · this conference brought to you by ... unified cloud...
Post on 01-May-2018
223 Views
Preview:
TRANSCRIPT
This Conference brought to you bywww.ttcus.com
Linkedin/Group:Technology Training Corporation
Technology Training Corporation
@Techtrain
Corporationwww.ttcus.com
U S A I t lli d S it C d
UNCLASSIFIEDUNCLASSIFIED
U.S. Army Intelligence and Security Command
Army Intelligence and “Big Data” 29 S t 2015
INSCOM … the Army’s Force for Dominant Intelligence
29 Sept 2015
UNCLASSIFIEDUNCLASSIFIED
Why? Big DataUNCLASSIFIED
All Sensors need Processing, Exploitation and Dissemination (PED)
UNCLASSIFIED3
Unified Cloud Data (UCD): A Joint Interagency Effort Aimed at Defining the
Unified Cloud Data (UCD)UNCLASSIFIED
• Unified Cloud Data (UCD): A Joint, Interagency Effort – Aimed at Defining the Service Technical Approach to the ICITE Framework and Data … While Solving the Global Processing, Exploitation & Dissemination (PED) Migration Challenges
• UCD: Our “Big Data” Reference Implementation of a Unified Data Layer• All Data Work Regardless of Type of Intelligence and Security Level• All Data Work, Regardless of Type of Intelligence and Security Level• All Analytics + Enrichment Processes Run against UCD: Write Once, Use Often• All Indexes Make UCD Data Discoverable to Analysts• Security Is “Baked In” – Security Markings Are Integrated at the Source Level,
Event / Document / Entity level, Down to Individual Attributes for an Entity
• UCD Pilot – Army’s Instantiation, Led by INSCOM, Assessed by Soldiers in Live Environment – To Inform Army Programs of Record• Army Mission Command and Army Cyber Pilots show Benefit of Unified Data
for Operations-Intel Convergence for the Army Operational Concept (AOC)
5UNCLASSIFIED
Open Source Information Alerts AnalystsUNCLASSIFIED
• Use Open Information – Pilot New Analyticsp y
• Train Analysts on Sources and Capabilities• Sources Change; Terms Evolve• Stay Engaged: Frequent Changes • Share Information, including Concepts of Operation
P t i t C t t f Wh t’ K• Put into Context of What’s Known • Mature Techniques to Verify and Understand:
Who, What, Why??
• Certify for Policy Compliance• Certify for Policy Compliance• Reinforce Mission and Legal Authorities• Do the Right Analysis, the Right Way
• Leverage the Enterprise:A l ti C biliti• Analytic Capabilities
• Data Approach: Pay for Data Once, Use Many Ways• Enterprise Impact: Data Retention, Storage,
Correlation, Cyber Security…
Big Data: U f l f Ti i C i
7UNCLASSIFIED
Useful for Tipping + Cueing, But Has Risks
Unified Cloud Data (UCD) – Partners & PilotsUNCLASSIFIED
• Demonstrated Value:
ldi d i+ Soldiers used UCD + Live Data+ Saved Mission Command 7+ months to pilot Ops/Intel Convergence, DTRA to pilot g , pConstellation on UCD baseline
+ Remote Mgt + Puppet deployment lets 1 SysAdminmanage multiple UCD sites
10UNCLASSIFIED
manage multiple UCD sites+ Piloted AWS GovCloud for Integration: Saved $
Implementing Unified Cloud Data (UCD)Implementing Unified Cloud Data (UCD)UNCLASSIFIED
11UNCLASSIFIED
Assessed UCD + Value of ‘Big Data’ Analytics to Inform Future Requirements
Good Feedback from Functional AssessmentUNCLASSIFIED
Good Initial Feedback: Users Want More Access + More Data
Assessed UCD + Value of Big Data Analytics to Inform Future Requirements
Improved Analyst Usability:• Ease of Use: Easy Multi-INT exploitation of Unified Data with Widgets + Workflows
• Rapid Mastery: After 3 days training, Soldiers could use UCD for mission threads
• Speed of Analysis: Soldiers used UCD to do Country Study in 30 min (1/3 time)
• Fast Data Access: Facial Recognition in seconds against 100Ks of records
Operations-Intelligence Convergence:• Improved Situational Awareness: Blue Force + Red Data in Common Operating Picture
• Pre-Deployment Checks: Soldiers easily checked New Area: Know What’s Known
Enterprise Efficiencies and Security:• Built-In Support: Self-Configurable Dashboards + Workflows Soldiers can share
• Info Sharing: Built-in Reports/Report Creation (no support needed), Coalition Info Sharing
• Enterprise Operations: Remote Admin by Fewer System Admins support multiple sites
• Improved Security: Cell-level Security, Thin Client/PKI, Separation of Roles
12UNCLASSIFIED
UCD Support of Intel FunctionsUCD Support of Intel Functions
Separate Data from Analytics
UNCLASSIFIED
Sensors
Separate Data from Analytics
Security:P
UCD handles
Data Access:User
Authorizations
Sensors • Provenance• Security Labels • Metadata Tagging• Extract Entities + Geo/Temporal Attributes• Metrics• more
many kinds of data
Data Access:Match User
Roles/Authorizations against Data Security
Data Ingestion
Data from Many Sources/Types
• Images• Audio• Video
Velocity +
Content
Authorizations
Real Time
Community Partners• Video
• Messages• Public Info• Mission
Command• Etc. Analytics
Update
Real-TimeCell-Level Security
Analyst’s Conclusions
Indexes Enrich Data
• Correlate All Data• Context-Based Data Navigation
• Analysts Enrich Correlated Data:+ Know What’s Known Now+ Helps Analysts “Connect the Dots”
• Supports ‘Big Data’ Analytics
13
• Map Reduce Analytics Enable Data Sharing
Context Based Data Navigation+ Beneficial to All Domains:+ Operations-Intel Convergence+ Medical Support+ Logistics Support
• Supports ‘Big Data’ Analytics and Multi-Discipline Fusion for AOC environments: A2AD, Megacities, Ad Hoc Response…
UNCLASSIFIED
UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED
UCD handles Separating Data from
Apps lets Analytics U S D t
many kinds of data
Use Same Data: Read Once, See Data
Many Ways
UNCLASSIFIED
UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED
UCD handles Separating Data from
Apps lets Analytics U S D t
Counter-Insider Threat
Security checks
many kinds of data
Use Same Data: Read Once, See Data
Many Ways
Security checks
UNCLASSIFIED
UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED
UCD handles Separating Data from
Apps lets Analytics U S D t
Counter-Insider Threat
Security checks
many kinds of data
Use Same Data: Read Once, See Data
Many Ways
Security checks
UCD implements full CRUD functionality: Analysts can
Create, Read, Update, Delete
UNCLASSIFIED
UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED
UCD handles Separating Data from
Apps lets Analytics U S D t
Counter-Insider Threat
Security checks
many kinds of data
Use Same Data: Read Once, See Data
Many Ways
Security checks
UCD implements full CRUD functionality: Analysts can
Create, Read, Update, Delete
UCD Unifies Data
UNCLASSIFIED
UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED
UCD handles Separating Data from
Apps lets Analytics U S D t
Counter-Insider Threat
Security checks
many kinds of data
Use Same Data: Read Once, See Data
Many Ways
Security checks
UCD implements full CRUD functionality: Analysts can
Create, Read, Update, Delete
UCD Unifies Data
Fine-Grained Data Security Markings are stored in Accumulo
UNCLASSIFIED
UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED
UCD handles Separating Data from
Apps lets Analytics U S D t
Counter-Insider Threat
Security checks
many kinds of data
Use Same Data: Read Once, See Data
Many Ways
Security checks
UCD implements full CRUD functionality: Analysts can
Create, Read, Update, Delete
UCD Unifies Data
Fi G i d D tFine-Grained Data Security Markings are stored in Accumulo
UNCLASSIFIED
Fast GPU-based Geospatialand Temporal indexing
UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED
UCD handles Separating Data from
Apps lets Analytics U S D t
Counter-Insider Threat
Security checks
many kinds of data
Use Same Data: Read Once, See Data
Many Ways
Security checks
UCD implements full CRUD functionality: Analysts can
Create, Read, Update, Delete
UCD Unifies Data
Fi G i d DIntegrated Capabilities
Fine-Grained Data Security Markings are stored in Accumulo
g p
such as Facial Recognition
UNCLASSIFIED
Fast GPU-based Geospatialand Temporal indexing
UCD Support of Intel FunctionsUCD Support of Intel FunctionsUNCLASSIFIED
UCD handles Separating Data from
Apps lets Analytics U S D t
Counter-Insider Threat
Security checks
many kinds of data
Use Same Data: Read Once, See Data
Many Ways
Security checks
UCD implements full CRUD functionality: Analysts can
Create, Read, Update, Delete
UCD Unifies Data
Integrated Capabilities Fine-Grained Data
Security Markings are stored in Accumulo
g p
such as Facial Recognition
Can Make Workflows –
UNCLASSIFIED
Fast GPU-based Geospatialand Temporal indexing
Can Make Workflows for Analyst Tasks + for
MapReduce Jobs
Conform to the Enterprise
Constraints We Must HandleConstraints We Must HandleReduce Costs
UNCLASSIFIED
Conform to the Enterprise Reduce Costs• Software Licenses• Physical Footprint• IT Supportpp• Leverage Open Source / Other
Software
Increase SecurityIncrease Security• Provenance: Track Every Interaction• PKI and Thin Client• Bastion Node Construct
Expect Change• New Kinds of Data: OSINT/Social Media
• Remote Administration: Separate System Administration from the Data
Exploit Exponential • New Kinds of Data: OSINT/Social Media• New User Needs: Heat Map• New Capabilities from Partners: WAMI
Track Extraction from NGA
Increase in Data
22
• New Technology Components: GPU Processors for Geospatial Data
UNCLASSIFIED
Protect the Data:
UCD Lessons LearnedUCD Lessons LearnedUNCLASSIFIED
Protect the Data: Build on a secure Cloud architecture with cell/object-level security, and extend security down to the weapon system to be able to Counter Insider Threat (with PKI, provenance etc.).
"Big Data” Strategy with Unified Data: Capture and triage vast, increasing amounts of data of all types, from many sources, with automated "Big Data" p g , g yp , y , ganalytics. Analytics from different providers should leverage the same correlated data: write once, use many ways.
Enterprise-Scale Remote Administration: Leverage automated build, delivery, install, configuration management, system administration, and monitoring to make operational use simple and scalable to the Enterprise level: automate routine tasks so humans focus on problems.
Leverage Prior Capabilities with Enterprise Platform: Break apart legacy stovepipe capabilities from battlefield-proven Quick Reaction Capabilities (QRCs); re-host unique components on IC ITE security architecture to ensure needed capabilities endure for the future.
Transform Capabilities Acquisition: To benefit from the innovation & speed of new capabilities, at long-term greatly reduced cost, incentivize organizations and individuals to use Open Source software, Agile business models of vendors & gov’t organizations.
Encourage and Reward Partnering: Strong INSCOM + Mission Command partnership integrated INSCOM UCD software onto MC tactical cloud servers: at NIE 15 1 showed Intel/Ops Convergence situational awareness DIL operations Bde TOC server consolidationNIE 15.1, showed Intel/Ops Convergence, situational awareness, DIL operations, Bde TOC server consolidation …
Future Ready for Capabilities Integration: Future analytic capabilities that exploit Unified Data can be rapidly integrated and (done right) can inherit security.
Need New Categories of Innovative Capability: Cyber Security and Info Assurance require new Enterprise analytics, to understand threats, attacks, system health –but we also need Innovative Capabilities + Processes to demonstrate Info Assurance (IA) and Capabilities Security.
23UNCLASSIFIED
top related