international technology alliance in network & information sciences dave braines, john ibbotson,...
TRANSCRIPT
International Technology Alliance inNetwork & Information Sciences
Dave Braines, John Ibbotson, Graham White (IBM UK)
Dave Braines, John Ibbotson, Graham White (IBM UK)
SPIE Defense Security & SensingSPIE Defense Security & SensingNext Generation AnalystNext Generation Analyst
MIPS:MIPS:A service-based aid for A service-based aid for
intelligence analysisintelligence analysis
SPIE Defense Security & SensingSPIE Defense Security & SensingNext Generation AnalystNext Generation Analyst
MIPS:MIPS:A service-based aid for A service-based aid for
intelligence analysisintelligence analysis
Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The USand UK Governments are authorized toreproduce and distribute reprints forGovernment purposes notwithstandingany copyright notation hereon.
Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The USand UK Governments are authorized toreproduce and distribute reprints forGovernment purposes notwithstandingany copyright notation hereon.
AgendaAgendaAgendaAgenda
What is MIPS?
IT Support for the Intelligence Analysis Process
MIPS and the International Technology Alliance (ITA)
The MIPS Architecture and an example of its use
Meeting the MIPS Objectives
Research transition project led by DSTL (Defence Science Technology Laboratory) UKApplying emerging technologies arising from the US/UK
ITA research program• Information Fabric, Gaian Database, Controlled English (CE) & CE Store
Provide a generic service-based information processing architecture
Support the information analyst in their analytic goals• Notify new information relevant to their current task
– In any form, from any source
• Automatic processing of future information to extract meaningful task-relevant domain information
Management of Information Processing Services
What is MIPS?What is MIPS?What is MIPS?What is MIPS?
Collection IT Processing can extract, filter and
transform data into information from large volumes of data
Cognitive Processing is restricted to IT process configuration due to large volumes of data
Processing Preparation of information for
interpretation by analysts
Interpretation Human activity through hypothesis
testing and other cognitive techniques
IT processing assists analyst through information management, rules based inferencing and visualisation
MIPS is a proof of concept to investigate how analysts can configure and benefit from an improved Collect/Process/Interpret processing pipeline
On the ground Increased volume and variety of
information sources Cannot manually inspect, process &
interpret IT is not appropriate for all phases
in intelligence analysis
Processing to support Intelligence Analysis
IT supports extraction, storage,
indexing
IT supports retrieval, visualisation,
presentation & communication
Human configuration, algorithms, modeling
Human intel analysis,
hypotheses
IT Support for the Intelligence Analysis IT Support for the Intelligence Analysis ProcessProcessIT Support for the Intelligence Analysis IT Support for the Intelligence Analysis ProcessProcess
MIPS Builds on work from the US/UK MIPS Builds on work from the US/UK International Technology AllianceInternational Technology AllianceMIPS Builds on work from the US/UK MIPS Builds on work from the US/UK International Technology AllianceInternational Technology Alliance
ITA research is a US/UK collaboration between industry, academia and governmentFocused on: Network science, Decision-making & Coalition operationsFrom May 2006 – May 2016
Research is fundamental (6.1), low Technical Readiness Level (TRL) Intended outputs are papers, proof of conceptsHigher TRL transition contracts take the core ideas to progress further
Emerging technologies arising from ITA research used by MIPS: ITA Controlled English
• Consumable by humans and machines
ITA Information Fabric• A lightweight service bus middleware
Gaian Database• A Dynamic, Distributed, Federated Database
What is it? Human-friendly language but machine readable Rich semantics, broad application Reasoning engine with rationale Supports agility for dynamic, evolving situations Enables hybrid human/machine collaboration Empowers non-technical users
Examples: the person Dave works for the company IBM. if (the person P works for the company C) then (the company C employs the person P). the person pn0123 is a suspect in the crime c713 because the person pn0123 was sighted in… conceptualise an ~ intelligence report ~ I that ~ mentions ~ the suspect S.
Why does it matter? Real fusion of machine precision and human cognition Can harness “collective intelligence” Facilitate human-human communication & socialisation
Download the CE Store: http://ibm.co/RDIa53
Using human language to enable smarter human/machine processing of information
ITA Controlled EnglishITA Controlled EnglishITA Controlled EnglishITA Controlled English
A two-way message bus and set of middleware services Connects all network assets to each other and to users Provides universal access to intelligence data, processing services and applications
Implemented using a multi-hop publish/subscribe architecture Messages are efficiently propagated without duplication thereby minimising bandwidth utilisation
Policies constrain how assets can be used and configured and information shared across coalition boundaries
Same policy mechanism as is used in the Gaian Database
Information fabric can integrate assets and services In MIPS, the Information Fabric is used to integrate information processing services
A lightweight service-bus middleware designed for the edge of the network
ITA Information FabricITA Information FabricITA Information FabricITA Information Fabric
Download the fabric:http://ibm.co/13FyW9X
8
1250 node Gaian Database
DDFDs provide distributed accessto distributed heterogeneous data
sources, are highly scalable & impose low management overhead
Obtaining information from multiple heterogeneous data sources does
not scale & requires significant management overhead
N11
N1
N0
N3
N4
N5
N2
N6
N9N10
N8
N7
Query
Query
DDFD is based on “store locally & query from anywhere” principle
A dynamic distributed federated database (DDFD)
The Gaian DatabaseThe Gaian DatabaseThe Gaian DatabaseThe Gaian Database
Download Gaian:http://ibm.co/15TMSBr
Objective 1: Notify analysts when new, relevant information has arrived Defining Relevant Information Maintaining and sharing analytical goals Indexing information & notifying analysts of matches with their analytical goals Operation in a mixed environment
Objective 2: Automatic processing of new information Identifying processing services and data sets Standardised description of processing services to enable reasoning Visualising current capabilities for the user Efficient operation Abstract services Automatic information processing in a mixed environment Facilitating collaboration between analysts
MIPS Objectives & ChallengesMIPS Objectives & ChallengesMIPS Objectives & ChallengesMIPS Objectives & Challenges
Composition Analyst describes a set of services and
links them to form a network (to achieve a goal)
Network can be deployed as a set of linked and running services
Information Fabric Provides nodes for services to run on
and message bus for service interconnection
Services perform specialised information processing
Sources & Sinks
Controlled English All data produced by MIPS services is
converted to CE and optionally stored in the CE Store
Data is stored and queried using CE Some services also implemented in CE
Analyst Tools A set of tools for users to operate
within the MIPS environment
MIPS ArchitectureMIPS ArchitectureMIPS ArchitectureMIPS Architecture
Integrates multiple SharePoint repositories
Document Extractor service extracts metadata facts in CE about the documents
Report Analytics text mining service extracts entities and relationships from document text
Implemented as a set of CE-based services
Watch List Notifier looks at extracted CE about people and queries watch lists to see if a person is present on the list
Multiple types of watch list are demonstrated
Notification Sink manages notifications that can be accessed by other services and to alert users
Information generated is stored in CE store
Example: A typical information flowExample: A typical information flowExample: A typical information flowExample: A typical information flow
Challenge 1: Defining relevant information Analysts indicate relevant information by defining service compositions, or by
querying the CE Store. Compositions and queries may be saved, shared and reused
Challenge 2: Maintaining and sharing analytical goals Analysts define goals using the Fabric Service Composition Tool. Goals so defined
may be reused and shared amongst teams of analysts Goals may also be encapsulated in queries (or rules) in the CE Store which may
be saved, shared and reused
Challenge 3: Indexing information and notifying analysts of matches with their analytical goals
Indexing of information may be via a MIPS service or an external application e.g. traditional indexing within a relational database
Example provided of local and external watch list inspection, notifying people matched, with a link to the original document text about the person of interest
Challenge 4: Operation in a mixed environment The MIPS demonstrator is capable of interacting with and integrating different data
sources and using a wide range of deployed processing services
Meeting the MIPS ObjectivesMeeting the MIPS ObjectivesMeeting the MIPS ObjectivesMeeting the MIPS ObjectivesObjective 1: Notify Analysts when new, relevant information has arrived
Challenge 1: Identifying processing services and data sets The MIPS Fabric Service Composition Tool (FSCT) allows users to identify services available to them
and the data sets required and produced by the services
Challenge 2: Standardised description of processing services to enable reasoning MIPS standardises the descriptions of the services available to a user together with the information
exchange formats used between services
Challenge 3: Visualising current capabilities for the user The MIPS processing pipeline is constructed and visualised using a graphical editor; the FSCT Complex queries to the CE Store can be constructed and visualised using a CE Query Builder; a
graphical tool for creating complex queries across linked information
Challenge 4: Efficient operation MIPS uses the ITA Information Fabric as its underlying messaging middleware. The fabric’s
publish/subscribe model provides efficient routing of messages between services
Challenge 5: Abstract services Abstract services are a generalisation of a group or type of service Abstract service descriptions may be defined in MIPS before the service has been created
Challenge 6: Automatic information processing in a mixed environment MIPS is capable of handling information and services from both within and external to its
environment.
Challenge 7: Facilitating collaboration between analysts Some rudimentary support is provided in MIPS for collaboration amongst groups of analysts. Further
support is expected in future work
Meeting the MIPS ObjectivesMeeting the MIPS ObjectivesMeeting the MIPS ObjectivesMeeting the MIPS ObjectivesObjective 2: Automatic processing of new information
Email: [email protected] or [email protected]
Questions?Questions?
Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The US and UK Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.
Main links: IBM developerWorks downloads:
• Information fabric:http://ibm.co/13FyW9X
• Gaian database:http://ibm.co/15TMSBr
• CE Store:http://ibm.co/RDIa53
International Technology Alliancehttp://www.usukita.org