Ben Conklin, Esri

Activity Based Intelligence Discovery Intelligence for Resolving the Unknown

Implementing ABI Workflows using ArcGIS

Activity Based Intelligence

• A set of spatiotemporal analytic methods to:

- Discover correlations

- Resolve unknowns

- Understand Networks

- Develop Knowledge

- Drive Collection

...Using diverse multi-INT data sets

Monitor Research

SearchDiscover

ABI

Shift to Discovery Focus

Known Location

Unknown Location

Kn

ow

n S

ign

atu

re

Un

kno

wn

Sig

na

ture

Locations and Targets

Be

ha

vio

rs a

nd

Sig

na

ture

s

The Intelligence Cycle

Requirements

Tasking

Collection

Processing

Exploitation

Dissemination

ABI Emphasis Traditional Emphasis

Traditional Intelligence and ABI

Attribute Traditional Intel ABI

Adversary Nation-states; predictable; doctrine-based

Asymmetric threats;unpredictable; motivation-based

Signature Durable; physical; definite Non-durable; proxies

Smallest Unit Class of equipment/object Individual entity with unique identifier

Analytic Reasoning Inductive; linear Deductive; non-linear

Target model Facilities & targets; coordinate; targeted

Area of interest; population; region; incidental collection

Motivation Collection-driven Analysis-driven

Reporting Finished serial reporting In-work products; layers; files

Collection frequency Scheduled; deck based Persistent and pervasive; multi-INT

4 Pillars of ABI

ABI Pillar

SequenceNeutrality

ABI Pillar

IntegrationBefore

Exploitation

ABI Pillar

Data Neutrality

ABI Pillar

GeoreferenceTo

Discover

Application Frameworks

Real-Time Analytics

Big DataAnalytics

Geo-Enrichment

IntelligenceEnterprise

Enabling Technology

Spatial & TemporalData Environment

All Source Analysts

Geospatial Analysts

Foundation Intelligence

Real TimeReporting

AnalysisServices

HUMINT Analysts

Imagery Analysts

SIGINT Analysts

Major Categories of Intelligence

Strategic Intelligence

Guidance for

developing policies,

usually looking three to

5 years ahead

Current Intelligence

Data-to-Day events and

new development.

Possible indicators of

developments to come

Basic Intelligence

Compiling of reference

data presented in

various forms and

publications

Discovery Intelligence

Unconstrained exercise

of searching for new or

potentially unknown

information

National

Intelligence Estimate

Analysis of Competing

Hypotheses (ACH)

President’s Daily Brief

Key Assumptions

Check (KAC)

CIA World Factbook High-impact, Low

Probability Assesment

Activity-based

Intelligence

Intelligence Category

Sample Product

Methodology

Decomposition of Intel Problem for ABI

Intelligence Problem

Determine the intentions of a

near-peer state power.

Sub-Problem

What are the intentions of the

near-peer’s current leadership?

Sub-Problem

What is the current military

capability of the near-peer?

Sub-Problem

What is the current economic

situation of the near-peer?

Intelligence Problem

What is the pattern-of-life and

behaviors associated with the

state’s leadership?

Approach or Method to Address

Sub-Problem

Approach or Method to Address

Sub-Problem

ABI

Who-Where-Who & Where-Who-WhereScope the Entity/Entities of

Interest

“What is the entity or group

of entities of interest?”

Identify Relevant Locations

“Where has this entity

previously been observed?”

Examine Co-Occuring

Entities

“What other entities have

also been observed at these

locations?

Asses for Demonstrative

Correlation

“Is co-occurance indicative

of a relationshiop between

the individual entity and the

discovered entity.

Foundation

Observations

Imagery

Big Data

3D

Lidar

Real-Time

Location Integrates Intelligence DataUsing a Simple Logical Model

Creating A Common Language

Apps Distributed

Georeference to Discover

Degree Type Basis Example

First Degree Direct Metadata GPS Location “tag” on a still image

Indirect Content Text document stating individual residence

Indirect Metadata Biographic profile with a metadata tag for residence

Second Degree Indirect Metadata/Context Synthesized from both content and context of data (e.ggeoreferenced poem)

Georeference to Discover - Context

Data Type Focus Primary Purpose

Activity Discrete activities conducted by individual entities

Entity resolution/identification,pattern-of-life assessment

Context Aggregated data of any type Provides context for observed activities and entities

Biographical Attribute information of an entity Provide information to a specific entity such as age or name

Relational Information describing relationship between entities

Understand and visualize the formal and informal social networks to which an entity belongs

Data Conditioning Techniques

Source Typical Extraction Activities

Text reports Entities, events, coordinates, locations

Still imagery Buildings, roads, geographic features, vehicles, changes between frames

Motion Imagery Vehicle motion (tracks), human activities

Hyperspectral imagery

Materials

Infrared imagery Warm objects, operating equipment

Financial transactions Account numbers, identifies, amounts

Strucutred Observations and Temporal Registartion

Integration before Exploitation

Defense, Intel and National Security

Object and Activity Extraction from Motion and Still Imagery

• Still Imagery for Long Duration Activity

- Ships in Port

- Cars in Parking Lot

• Motion Imagery for rapid activity and transactions

• Machine Learning and “Neural Networks” for advanced recognition

Activity Category Candidate Activities

Single Person Digging, loitering, picking up, throwing, exploding/burning, carrying, shooting, launching, walking, limping, running, kicking, smoking, gesturing

Person-person Following, meeting, gathering, moving as a group, dispersing, shaking hands, kissing, exchanging objects, kicking, carrying together

Person-vehicle Driving, getting in/out, loading/unloading, opening trunk, crawling under car, breaking window, shooting/launching, exploding/burning, dropping off, picking up

Person-facility Entering (exiting), standing, waiting at checkpoint, evading checkpoint, climbing atop, passing thru gate, dropping off

Vehicle Accelerating, turning, stopping, overtaking/passing, exploding/burning, discharging, shooting, moving together, forming into convoys, maintaining distance

Other VIP activities (convoy, parade, receiving line, troop formation, speaking to crowds) riding animal, bicycling, etc..

Vehicle Start

ABI Lexicon

• Activity

• Events and Transactions

- Physical and logical

• Context and Biographical

• Relational

Date 03-MAR-2012

TIME 1540Z

Location X,Y

Type 4D Sedan

Color RED

Start

Vehicle Stop

Date 03-MAR-2012

TIME 1840Z

Location X,Y

Type 4D Sedan

Color RED

Stop

Disambiguation and Durability

• Entities are observed through proxies

• High Durability – Biometrics

• Low Durability – Vehicle

• Indexing – Entity to Proxy Resolution

Entity A

Height

Birthplace

Vehicle

Name

Telephone

Email

Entity and Unstructured Data Integration

Enterprise Approach to Data Integration

Defense, Intel and National Security

Desktop Apps

Server

ArcGIS

New Analytic Workflow

Real Time Analysis

Batch Analysis

Raw Feeds Alerts & Forecasts

Portal

Apps

DesktopAPIs

ArcGIS Enterprise

Enterprise Data Management, Analysis, and Mapping

ArcGIS Enterprise

Data Management, Mapping, and

Geoprocessing

Real Time

Image Management & Raster Analytics

Big Data

Access and Management

GeoEventServer

GeoAnalyticsServer

Image Server

GIS Server

Portal

Modular and Massively Scalable

• Specialized Servers

• Independently Scalable

• Flexible

Real-Time GISIntegration & exploitation of streaming data

• Integrates real-time

streaming data

into ArcGIS

• Performs continuous

processing and

real-time analytics

• Sends updates and alerts to

those who need it

where they need itArcGIS Server

GeoEvent Extension

DesktopWeb Device

Working with Real-Time DataMaking features come alive

• Import your feature layer’s schema as a GeoEvent Definition

• Connect an output to your feature service

• Configure an input to receive real-time data

• Author and publish a GeoEvent Service

• Visualize your real-time features

GeoEvent Extension

Ou

tpu

ts

Inp

uts

GeoEvent Services

ArcGIS Server

Operations Dashboard for ArcGIS

operation views

web maps

ArcGIS Online /Portal for ArcGIS

feature services

GeoEvent Definitions

Applying real-time analyticsGeoEvent Services

• A GeoEvent Service defines the flow of GeoEvents,

- The Filtering and Processing steps to perform

- what input(s) to apply them to

- and what output(s) to send the results to

Applying Real-Time AnalyticsGeoEvent Processing

You can createyour own

processors.

• You can perform continuous analytics on GeoEvents as they are received using a processor.

GeoEvent Extension

Inp

uts

Ou

tpu

ts

GeoEvent Services

Buffer Creator

Convex Hull Creator

Difference Creator

Envelope Creator

Field Calculator

Field Enricher

Field Mapper

GeoTagger

Incident Detector

Intersector

Projector

Simplifier

Symmetric Difference

Track Gap Detector

Field Reducer Union Creator

Ou

t o

f th

e B

ox

Add XYZ

Esr

iG

all

ery

Bearing

Ellipse

Event Volume Control

Extent Enricher

Field Grouper

GeoNames Lookup

Range Fan

Reverse Geocoder

Service Area Creator

Symbol Lookup

Track Idle Detector

Unit Converter

Visibility

Motion Calculator Query Report

Location SourcesSensed

Location

Volunteer Location

ObservedLocation

StayLocation

Stay Locations

• Locations where individuals spend time

• Includes Key Attributes for pattern analysis.

• Can be used at different scales

- Individual

- Neighborhood

- Regional

Attribute Description

Geometry Polygon and Point

IndividualID ID for individual at Stay Location

LocationID Unique ID for location

PreviousLocationID ID for Previous Location reported, blank if no locationID

StartTime UTC Time of location at start

EndTime UTC Time of location of departure or last report at stay location

Duration Duration at Stay Location

LocationSource Source Data for stay location id

Source Source data for the reported location

Known Location InformationSensed

Location

Volunteer Location

ObservedLocation

StayLocation

SuspiciousPlaces Stay

Locations

MonitoredLocations

Suspicious Places

• Predefined Locations for monitoring

- Home, Work, Meeting Areas

- Key POIs (Place of Worship, Restaurants, Internet Café)

Attribute Description

Geometry Polygon

LocationID Unique ID for location

LocationName Common name for the location

LocationCategory General Category for the Location

LocationActivity Type of activity typically observed at location

LocationModel Discrete,Semi-Discrete, Non-Discrete

LocationSignificance A subjective numerical value

LocationSource Source of information for the suspicious Location

DiscretenessLocation A (Movie Theater)

Show MoviesUsed for Community

MeetingsClosed Sundays

Entity Class

“Ticketholders and

Guests”

Entity Class

“Community Meeting

Attendees”

Entity Class

“Underground Cinema

Club”

Value Description Example Location

Discrete Locations restricted to a highly limited entity network that provide highly diagnostic proxy observations

Private residence

Semi-discrete Locations that maintain some degree of access control but stillhave many potential proxy-entity relationships

Restricted military installation; sporting event; office place

Non-discrete A location not unique to any one entity or network of entities at agiven time; therefore a location that has somewhat less value for disambiguation

Public market, square, or park

trackId V10987

gap true

lastReceived 1405176855553

geometry -116.93…, 33.93…

trackId V10987

gap true

lastReceived 1405176855553

geometry -116.93…, 33.93…

GeoEvent processingNotify about the absence of events

• A Track Gap Detector processor

- Detects the absence of events and alerts about the situation.

• A Track Idle Detector processor

- Detects the lack of movement even while events are received and alerts about the situation.

GapClosed

GapDetected

SuspectID V10987

Date 1405176845553

Geometry -116.93…, 33.93…

SuspectID V10987

Date 1405176855553

geometry -116.93…, 33.93…

trackId V10987

gap true

lastReceived 1405176855553

geometry -116.93…, 33.93…

trackId V10987

gap true

lastReceived 1405176855553

Geometry -116.93…, 33.93…

trackId V10987

gap false

lastReceived 1405176915553

geometry -117.123…, 36.064…

SuspectID V10987

Date 1405176915553

geometry -116.93…, 33.93…

SuspectID V10987

Date 1405176925553

geometry -116.93…, 33.93…

SuspectID V10987

Date 1405176935553

geometry -116.93…, 33.93…

SuspectID V10987

Date 1405176945553

Geometry -117.123…, 36.064…

GeoEvent ProcessingEnrich a GeoEvent with new fields

• A Cache Aware Field Calculator processor

- Uses information for the Previous Track to calculate values on the current event

- Tracks are based on Individual ID

- Allows for calculation of Previous Location and Duration Tracking

Event

Previous

Event Details

IndividualID V10987

Date 1405176845553

LocationID House

geometry -117.123…, 36.064…

IndividualID V10987

Date 1405176845553

LocationID Work

geometry -117.123…,36.064…

PreviousLocationID Work

geometry -117.123…,36.064…

Field Calculator

Cache-Aware

Alerts

Standard Reports

Tailored Reports

Visualization

Query/Drill Down

Statistical Analysis

Forecasting

Anticipatory Analytics

Something Happened

What Happened?

Where? How Many? How Often?

Where? How Many? How Often?

What are the specific causes?

Why did it happen?

What might happen?

When, where and how likely is it to happen?

Analytic Effort Required

De

cisi

on

Ad

va

nta

ge

Ga

ine

d

GeoEnrichment of Observations

Defense, Intel and National Security

Cultural Data

Landscape Data

Social Data

Observations

Big Data Spatial Analytics | Faster and Massively Scalable

Faster (10x+)

Power Outages(50+ Million)

Density

Imagery

Lidar: Bare Earth

Hot Spots

Riparian AreasSpace-Time Cube

Lidar: First Return

. . . Accessible from ArcGIS Pro and Python APILeveraging Distributed Computing and Parallel Processing

Image ServerLarge Imagery Collection

Imagery / Raster

Image Processing

Classification

Change Detection

Topo

Suitability

Density

Corridors

Distance

Proximities

Interpolation

Features / Vectors

Space-Time Analytics

Hot Spots

Density

Buffer

Summarize

Aggregation

Construct Tracks

Find Similar

Spatial Join

GeoAnalytics ServerLarge Observation Collections

Rich Collection of Analysis Tools

Summarize DataAggregate PointsSummarize NearbySummarize WithinReconstruct TracksCreate Panel

Find LocationsFind Existing LocationsFind Similar Locations

Analyze PatternsCalculate DensityFind Hot SpotsCreate Space Time Cube

Use ProximityCreate Buffers

Manage DataExtract DataJoin Features

Spatial / BI

GIS

Charts

Linked and Responsive Charts and Maps

On-the-Fly Visual Models

Integrated Spatial and Tabular Analysis

• SQL Server• Oracle• SAP HANA• Teradata

DBMSs• Excel• CSV

Local

Insights | A New Experience for Spatial Analytics

For Analysts and Data Scientists

• Visual, Intuitive, Responsive

• Exploratory Data Analysis and Visualization

4 Pillars of ABI

ABI Pillar

SequenceNeutrality

ABI Pillar

IntegrationBefore

Exploitation

ABI Pillar

Data Neutrality

ABI Pillar

GeoreferenceTo

Discover

Application Frameworks

Real-Time Analytics

Big DataAnalytics

Geo-Enrichment

IntelligenceEnterprise

Enabling Technology

Please Share Your Feedback in the App

Download the EsriEvents app and find

your event

Select the session you attended

Scroll down to “Survey”

Log in to access the survey

Complete the survey and select “Submit”

North Star Branding Visuals to ComeUpdated walk-in and walk-out slides will be provided mid-June

Show this after your presentation has ended.