tasking the tweeters: obtaining actionable information from human sensors alun preece, will...
TRANSCRIPT
Tasking the Tweeters: Obtaining Actionable Information from Human Sensors
Alun Preece, Will Webberley (Cardiff)
Dave Braines (IBM UK)
Introduction
Social Media for Real-time Intel
Social Media streams as sources of actionable intelligence for Situation Awareness (SA)
• Acknowledging a human-based sensor network• A good source today: Twitter
– Real-time characteristics– Follower-based model– Open APIs
• Real-world examples from recent events:– Boston marathon bombing (US)– Lee Rigby murder (UK)
Some SA platforms emerging:• Twitcident, Apollo, ReDites, Sentinel
Mapping Social Media to DCPD
Direction - what data to collect from where
Collection• e.g. Twitter: APIs for streaming, searching sampling
(other platforms available too)• Post-filtering for noise reduction
Processing• e.g. probabilistic, NLP, sentiment, event detection• Provide semantic enrichment (contextual) for Shared
Understanding.• Detect trends, clusters, anomalies etc
Dissemination• Visualisation, alerting,
summarisation• Further querying• Direct further collections
A generic social media processing pipeline mapped to DCPD steps
Dynamic ISR asset management
Missions-and-means framework formalised as a collection of ontologies
Tasks characterised by the data needed to achieve them
type of data (visual, IR, radar etc )
“quality” rating 0 to 9 Assets rated by the data they
provide
MMF framework
NIIRS-based approach
Software tool for agile sensor-task assignment Extensible knowledge-base of sensor-task
suitability Uses existing models and frameworks to map
capabilities
Sensor Assignment to Missions (SAM)
In previous work we have defined a framework for
dynamic ISR asset management:
A Pilot Study
July 26th, 2014: Cardiff protest march
• Planned protest march in Cardiff, UK• Against Israeli incursions into Gaza• Potential for public order disruption• Approximately 2,000
people• Some limited local trouble• Evidence of protest and
activities found on Twitter– Real-time during the event– In various stages afterwards
Source: Wales online – www.walesonline.co.uk
Social Media Timeline
Timeline of the July 26th 2014 protest and its aftermath
Social Media Timeline
Timeline of the July 26th 2014 protest and its aftermath
UK-wide tweets
Social Media Timeline
Timeline of the July 26th 2014 protest and its aftermath
Verbal and physical abuse at bars [15:15]
UK-wide tweets
Social Media Timeline
Timeline of the July 26th 2014 protest and its aftermath
Verbal and physical abuse at bars [15:15]
March ends [15:40]
UK-wide tweets
Social Media Timeline
Timeline of the July 26th 2014 protest and its aftermath
Verbal and physical abuse at bars [15:15]
March ends [15:40]
Tweeting after the march
UK-wide tweets
Social Media Timeline
Timeline of the July 26th 2014 protest and its aftermath
Verbal and physical abuse at bars [15:15]
March ends [15:40]
Tweeting after the march
Police mentionsincrease after broadcast news
UK-wide tweets
Social Media Timeline
Timeline of the July 26th 2014 protest and its aftermath
Verbal and physical abuse at bars [15:15]
March ends [15:40]
Tweeting after the march
Police mentionsincrease after broadcast news
UK-wide tweets
Important:We are observing perception of the event, not the
event itself…
Practical details
Sentinel Twitter Stream Analysis• Geo-tagged tweets• Topical search terms• Mentions of local places
People on the ground• Access to live twitter (+ search)• Manually identify “key” tweets
Some issues• Generality of tweets• Crowd size estimation: “a few hundred”,
“thousands”• Very few tweets geo-tagged
The Sentinel application
Observations from the pilot
• Sweet spot for initial relevancy: Search terms + geo-spatial
• Social Media reflects perception, not reality• We are not claiming that this simple study is
representative.• Key events and activities can be detected:
– …but how early can these be found through “small signals”?
• Some issues with Social Media:– Propagation of misinformation– Detection of bias– Quantification of contextual factors
• There is the potential to inform action viathis kind of situation awareness
Modeling Tweets and Tweeters
Background: CNL for conversation
Need an appropriate form for human-machine interaction:
humans prefer natural language (NL) or images these forms are difficult for machines to
process, leading to ambiguity and miscommunication
Compromise: controlled natural language (CNL)there is a person named p1 that is known as ‘John Smith’ and is a person of
interest.
low complexity | no ambiguityITA Controlled English (CE)
Defining sources and peopleSources, e.g. a Twitter account:conceptualise a ~ twitter account ~ A that
is an online identity andis a temporal thing andhas the value L as ~ location ~ andhas the value NT as ~ number of tweets
~ andhas the web image PP as ~ profile
picture ~ andhas the value NT as ~ number of tweets
~ andhas the value NFR as ~ number of
friends ~ andhas the value NFO as ~ number of
followers ~.
there is a journalist named ‘Paul Heaney’ thatuses the twitter account ‘paulheaney67’
andworks for the media organization ‘bbc’.
People (and their derivation from a source):
…we are actually building profiles of “human sensors”.
Human Sensor profilesThe following information is available for inclusion in the human sensor profile:• All data from their Twitter profile (including location)• Who they frequently interact with• Who they talk about• Who are their influencers• Recently posted media (photos, videos)• Terms names from recent tweets• Locations from recent tweets
– Including travel to/from locations• Sentiment analysis for tweets and terms
The use of our human friendly CNL means that additional “local knowledge” can easily be added too.
e.g. “stance” – to capture some importance contextual detail
This is a dynamic social network
Talking to Moira
An example Moira query showing some elements of the tweeter model
All this information (people, sources, tweets, terms, events etc) is available in a CNL knowledge base.
The Moira agent is able to access this and support conversation with human team members…
Tasking Tweeters
Defining ISR tasks
From our previous work:
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
Defining ISR tasks
From our previous work:
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
The “action” – what you are trying to achieve
Defining ISR tasks
From our previous work:
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
The “action” – what you are trying to achieve
What you are trying to do, e.g. “detect”, “localize”
Defining ISR tasks
From our previous work:
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
The “action” – what you are trying to achieve
What you are trying to do, e.g. “detect”, “localize”
From a predefined ISR ontology
Defining ISR tasks
From our previous work:
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
The “action” – what you are trying to achieve
What you are trying to do, e.g. “detect”, “localize”
From a predefined ISR ontology
From a gazetteer or similar
Defining ISR tasks
From our previous work:
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
The “action” – what you are trying to achieve
What you are trying to do, e.g. “detect”, “localize”
From a predefined ISR ontology
From a gazetteer or similar To establish temporal
bounds
Defining ISR tasks
From our previous work:
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
The “action” – what you are trying to achieve
What you are trying to do, e.g. “detect”, “localize”
From a predefined ISR ontology
From a gazetteer or similar To establish temporal
bounds
For simple resource scheduling
Defining Social Media ISR tasks
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
Defining Social Media ISR tasks
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
Direction:• The search terms (topics)
are derived from the “detectable”
• The spatial extent from the “spatial area”
Defining Social Media ISR tasks
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
Direction:• The search terms (topics)
are derived from the “detectable”
• The spatial extent from the “spatial area”
Collection:Stream-processing of tweets based on “direction” phase.
Defining Social Media ISR tasks
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
Direction:• The search terms (topics)
are derived from the “detectable”
• The spatial extent from the “spatial area” Processing:
The required “intelligence capability” determines the type of processing:• “localization” – derive
location data from tweets or tweeter
• “detection” – use existing event detection algorithms.
Collection:Stream-processing of tweets based on “direction” phase.
Defining Social Media ISR tasks
conceptualise the task T~ requires ~ the intelligence
capability IC and ~ is looking for ~ the detectable thing DT and
~ operates in ~ the spatial area SA and
~ operates during ~ the time period TP and
~ is ranked with ~ the task priority PR.
Direction:• The search terms (topics)
are derived from the “detectable”
• The spatial extent from the “spatial area” Processing:
The required “intelligence capability” determines the type of processing:• “localization” – derive
location data from tweets or tweeter
• “detection” – use existing event detection algorithms.
Collection:Stream-processing of tweets based on “direction” phase.
Dissemination:Alerting (or otherwise) via contextual application such as Sentinel, or agent such as Moira.
Identifying “key tweeters”
• In practice “key tweeters” emerge:– Use spatial terms: they want people to know where they
are– Use terms/hashtags: they want their tweets to be found– Social network: who are they and who they connect to
• From these we can determine:– Whether they are in a “position to know”– Their skills in Twitter usage– Their influence and reach
• All of this helps buildknowledge of trust andinformation quality
Findings so far• Existing ISR task representation can drive Twitter collection• Human & machine agents can use this information in
many ways• The Moira agent helps us to interact with the knowledge
base:– Engage the system in a conversation– Assert new local knowledge– Extend the model– Invoke additional functions such as
“fact extraction”
Use of the “stance” relationship in a conversation with Moira
An example of fact extraction from tweet text using Moira
Wrapping up
Related work
• Conversational interaction:– Bi-directional chains for
ISR pipelines– Humans and machines
in collaboration
• Experiments with Human subjects:– Using the Moira interface– Crowd-sourced Situational Understanding– Combine Human input and physical sensors– Handling incomplete and conflicting information– Use of relevancy criteria to minimise resource utility
Some conclusions
• Streamed insight from Social Media could be incorporated into traditional ISR asset management.
• This could be streamlined through:– Automatic assignment of assets (for stream processing)– Automatic identification of Social Media collections
• Lots of issues:– e.g. misinformation and coordinated rumours
• Awareness improves potential for action:– Early countering strategies, opportunities for
community intervention
• Limitations and opportunities:– We have focused on text-based analysis– Imagery potential: image processing,
face detection, object recognition etc
Tasking the Tweeters: Obtaining Actionable Information from Human Sensors
SPIE DSS 2015 – Ground/Air Multisensor Interoperability, Integration & Networking for Persistent ISR IV
Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The US and UK Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.
Development of the Sentinel platform was funded by the European Commission under the project “Tackling Radicalisation in Dispersed Societies (TaRDiS)”, and the ESRC via the project “After Woolwich: Social Re- actions on Social Media” (ES/L008181/1). Cardiff University provided funding for the pilot study examining community impacts of the NATO Summit.
We thank Kieran Evans and David Rogers (Cardiff University) for setting up the data collection pipeline for the pilot study in Section 2 and assistance with the data analysis. We thank Darren Shaw (IBM Emerging Technology Services, UK) for creating the tweeter locator service in Section 3. Valuable insights on policing and community reaction to events such as the ones featured in our pilot study were provided by Martin Innes, Colin Roberts and Sarah Tucker (Cardiff Universities Police Science Institute, http://www.upsi.org.uk).