1

SERENDIPITI SEnsoR ENricheD

Information Prediction and InTegratIon

2

the effect by which one accidentally discovers something fortunate, especially while looking for something entirely unrelated. (wikipedia.org)

Serendipity n.

Scoperta (Italian)Serendipiteit (Dutch)Vrozené štěstí (Czech)

/ˌsɛ.rɛn.ˈdɪp.ə.ti/

3

Today’s Agenda

VisionObjectivesTechnology

Partners

4

Vision What happens when you aggregate partial

observations?

• Maps and ship captains– No single human has been to

every point on a map– Cartographers resolved partial

observations from ship captains– Many needed, potentially

conflicting– Slowly, there emerged a map of

the world

• Can we do something similar to learn something new about our cities?

5

Objectives• Aim? Stimulate and integrate research groups

from currently fragmented research areas, creating synergies at the cross-over points.

• Approach? By fostering long-term relationships between research groups, based around people, and laying the foundations for a Virtual Centre of Excellence (VCE).

• In what areas? Real-time, large-scale data analysis and inference for fusing semantic information and predicting events.

• How? By harvesting, mining, correlating and clustering extremely large, highly dynamic, very noisy, contradictory and incomplete information from multiple sources including: tweets, logs, RSS, web-sources, mobile texts, web-cams, CCTV and other publicly-available multimedia archives.

6

What’s New in SERENDIPITI?

• Real-time

• Prediction (events)

• Multimodal

• Noisy, errorsome data

• Novel applications

7

How do we do this?

• Sensing– Aggregate diverse sources of information from

the real and online worlds

• Analysis– Extract stable spatio-temporal patterns of

human activity – Track these over time

• Use Case Scenarios– City Planning– Journalist (e.g. see Appendix)– Police

8

Physical

Online

How do we do this?

SensorBase

9

How do we do this?

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

Online

10

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

OnlineMachine

learning and data mining

How do we do this?

11

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

OnlineMachine

learning and data mining

How do we do this?

SERENDIPITIPlatform

12

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

OnlineMachine

learning and data mining

How do we do this?

SERENDIPITIPlatform

Planning (City)

Journalist

Police

Use Case Scenarios

13

Partners & Roles

Patricia Ho-Hune

Alan Smeaton, Noel O’Connor, Barry Smyth

Giovanni Tummarello, John Breslin, Paul Buitelaar

Keith van Rijsbergen, Joemon Jose, Mark Girolami

Ebroul Izquierdo

Maarten de Rijke, Arnold Smeulders

Vojtech Svatek

ERCIM

DCU-CLARITY

DERI

GLA

QMUL

UvA

UEP

0

1

2

3

4

5

6

14

Partners & RolesEU Project Management

Analysis of multimodal sensor data, real-time integration of physical & online sources, organisation & management of online sources

Semantic text analysis/mining, large-scale semantic search/indexing, linked data, social semantics, online communities

Multimedia information retrieval, formal models, mining information from large data sets, event detection, information fusion, machine learning

Correlating/mining media and textual data, sensor base, automatic (CCTV-based) event analysis

Focused crawling, wrapper induction, mining social media, information extraction, data integration, cross-media mining and information fusion, machine learning

Mining rich associations from large databases, information extraction from the Web, semantic and social Web technology, effectiveness of ICT

ERCIM

DCU-CLARITY

DERI

GLA

QMUL

UvA

UEP

0

1

2

3

4

5

6

15

1 2 3 4 5 6DCU QMULDERI GLA UvA UEP

PARTNERS

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

OnlineMachine

learning and data mining

SERENDIPITIPlatform

Planning (City)

Journalist

Police

Use Case Scenarios

16

1 2 3 4 5 6DCU QMULDERI GLA UvA UEP

PARTNERS

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

OnlineMachine

learning and data mining

SERENDIPITIPlatform

Planning (City)

Journalist

Police

Use Case Scenarios1

34

5

4

63 5

236

12

56

123456

17

Work Packages

• WP1 – Management• WP2 – Integration of Organisation & People

• WP6 – Applications & Infrastructure Sharing

• WP7 – Outreach (Spreading Excellence)

Joint Program of Activities• WP3 – Real-time Large-scale Data

Analysis• WP4 – Semantic Information Fusion• WP5 – Inference & Event Prediction

18

Implementation

Planning (City)

Journalist

Police

User Group Data Provision Board

Industrial Advisory Board

Wikimedia Foundation

Boards.ie

TW

S+V

EPA/MI/Met

Ken Wood

Milek Bover

E. Aarts

Carto Ratti

One other !

19

Tangible Outputs

• Provision of mini projects to tackle unforeseen research topics

• Industrial placements of research staff• Academic exchanges• Contribution to standardisation efforts• Public software repositories and

tools/data access through the SERENDIPITI platform

• Outreach to other targeted projects

20

Impact

• SERENDIPITI : “SEnsoR ENricheD Information Prediction and InTegratIon”

• People - Research - Outreach - Platform

• Large-scale, semantic urban computing