an introduction to open data

104
AN INTRODUCTION TO OPEN DATA Sally Jenkinson - Fronteers - Amsterdam - 09.10.2015 @sjenkinson | [email protected]

Upload: sally-jenkinson

Post on 16-Feb-2017

34.862 views

Category:

Data & Analytics


0 download

TRANSCRIPT

Page 1: An introduction to open data

AN INTRODUCTION

TO OPEN DATASally Jenkinson - Fronteers - Amsterdam - 09.10.2015 @sjenkinson | [email protected]

Page 2: An introduction to open data

[email protected] | @sjenkinson

Digital solutions architect & consultant Records Sound the Same Ltd

Sally Jenkinson

Page 3: An introduction to open data
Page 4: An introduction to open data

DATA

Page 5: An introduction to open data

OPEN DATA

Page 6: An introduction to open data

“Big data”

@sjenkinson

Page 7: An introduction to open data

90% of the world’s total data has been created within the last 2 years

!

(IBM, 2014)

@sjenkinson

Page 8: An introduction to open data
Page 9: An introduction to open data

I ♡ DATA

Page 10: An introduction to open data

@sjenkinson

Page 11: An introduction to open data
Page 12: An introduction to open data
Page 13: An introduction to open data

@sjenkinson

Page 14: An introduction to open data

sallyjenkinson.co.uk/labs/teatracker

Page 15: An introduction to open data

BUT…

Page 16: An introduction to open data
Page 17: An introduction to open data

“You agree to maintain your apps

and your systems in accordance with

industry standard quality levels…”

Page 18: An introduction to open data
Page 19: An introduction to open data

DATA SHARING

Page 20: An introduction to open data
Page 21: An introduction to open data

WHAT IS OPEN DATA?

Page 22: An introduction to open data
Page 23: An introduction to open data

Open data and content can be freely used, modified, and shared by anyone for any purpose.

opendefinition.org

Page 24: An introduction to open data
Page 25: An introduction to open data

Re-publish

Derive new content or data

Make money by selling products

Charge a fee for access

Page 26: An introduction to open data

Make money by selling products

Charge a fee for access

Page 27: An introduction to open data

“We observed that often people think of open data as a specific ‘kind’ of data –

something separate and distinct from the data they use day-to-day in their

organisation or team – rather than a choice about how people publish data.”

theodi.org/blog/closed-shared-open-data-whats-in-a-name

Page 28: An introduction to open data

theodi.org/guides/publishers-guide-open-data-licensing | theodi.org/guides/reusers-guide-open-data-licensing

Public domain (CC0) Attribution (CC-by) Attribution & share-alike (CC-by-sa)

OPEN LICENCES FOR CREATIVE CONTENT

Page 29: An introduction to open data

theodi.org/guides/publishers-guide-open-data-licensing | theodi.org/guides/reusers-guide-open-data-licensing

Public domain (PDDL) Attribution (ODC-by) Attribution & share-alike (ODbL)

OPEN LICENCES FOR DATABASES

Page 30: An introduction to open data

theodi.org/guides/publishers-guide-open-data-licensing | theodi.org/guides/reusers-guide-open-data-licensing

Open Government Licence OS Open Licence etc

OTHER OPEN LICENCES

Page 31: An introduction to open data

WHERE CAN I GET IT FROM?

Page 32: An introduction to open data

wiki.dbpedia.org

Page 33: An introduction to open data
Page 34: An introduction to open data

musicbrainz.org

Page 35: An introduction to open data

earthquake.usgs.gov/earthquakes/search/

Page 36: An introduction to open data

plaidplug.com

Page 37: An introduction to open data

data.id/dataset/daftar-titik-reklame-di-dki-jakarta/resource/361ce01f-34ed-4e00-a204-6062c7b9ad64

Page 38: An introduction to open data

web.archive.org/web/20150520175645/http://137.189.35.203/WebUI/CatDatabase/catData.html

Page 39: An introduction to open data

vision.stanford.edu/aditya86/ImageNetDogs/

Page 40: An introduction to open data

{"gilded":0,"author_flair_text":"Male","author_flair_css_class":"ma

le","retrieved_on":1425124228,"ups":3,"subreddit_id":"t5_2s30g","edited":false,"controversial

ity":0,"parent_id":"t1_cnapn0k","subreddit":"AskMen","body":"I can't agree with passing the blame, but I'm glad to hear it's at least helping you with the anxiety. I went the other direction and started taking responsibility for everything. I had to realize that people make mistakes

including myself and it's gonna be alright. I don't have to be shackled to my mistakes and I don't have to be

afraid of making them. ","created_utc":"1420070668","downs":0,"score":

3,"author":"TheDukeofEtown","archived":false,"distinguished":null,"id":"cnasd6x","score_hidden":false,"name":"t1_c

nasd6x","link_id":"t3_2qyhmp"}

x ~1.7 billion

reddit.com/r/datasets/comments/3bxlg7/i_have_every_publicly_available_reddit_comment/

Page 41: An introduction to open data

♥github.com/caesar0301/awesome-public-datasets

Page 42: An introduction to open data

CONSUMING OPEN DATA

Page 43: An introduction to open data

@sjenkinson

Page 44: An introduction to open data
Page 45: An introduction to open data

d3js.org

Page 46: An introduction to open data

MORE THAN WEBSITES

Page 47: An introduction to open data
Page 48: An introduction to open data

iquantny.tumblr.com/post/92116352544/mapping-nyc-hydrant-revenue-upper-easts-19th

Page 49: An introduction to open data
Page 50: An introduction to open data
Page 51: An introduction to open data

Generating value & making savings

@sjenkinson

Page 52: An introduction to open data

+$3 trillion / year

mckinsey.com/insights/business_technology/open_data_unlocking_innovation_and_performance_with_liquid_information

open data

Page 53: An introduction to open data

Transparency

@sjenkinson

Page 54: An introduction to open data

“…within two years chemical emissions nationwide (at least as reported, and

presumably also in fact) had decreased by 40 percent.

!

Some companies were launching policies to bring their emissions down

by 90 percent, just because of the release of previously sequestered

information.”

maban.co.uk/80

Page 55: An introduction to open data
Page 56: An introduction to open data

DATA & USER EXPERIENCES

Page 57: An introduction to open data

“How far do you live from your workplace? Chances are, you'd answer that question in minutes rather than miles. !

An hour on the bus tells us a lot more than 47 miles. That's why we made Mapumental. !

Given any start point or destination, it'll show everywhere within the chosen commute time, by public transport.”

mapumental.com/services/travel-time

Page 58: An introduction to open data

“How accessible is your nearest school, post office, or GP’s surgery?

!

In Wales, that’s not always a simple question: the country’s mountainous landscapes, rural

populations, and sometimes infrequent bus services can mean that those without cars are rather cut off from public service provision.”

mapumental.com/services/accessibility

Page 59: An introduction to open data

“Just how quickly could fire engines reach a given postcode in case of a fire?

!

It’s a question that’s pivotal to decisions made by both the emergency services and

the insurance industry.”

mysociety.org/2013/04/22/fire-fire-mapumental-and-fire-engine-journey-times

Page 60: An introduction to open data

Improved efficiency Improved effectiveness Impact measurement

@sjenkinson

Page 61: An introduction to open data

Improved or new private products or services & innovation

@sjenkinson

Page 62: An introduction to open data

NOT JUST DIGITAL

Page 63: An introduction to open data

opensensors.io

Page 64: An introduction to open data

DOUG MCCUNEdougmccune.com

Page 65: An introduction to open data
Page 66: An introduction to open data
Page 67: An introduction to open data
Page 68: An introduction to open data
Page 69: An introduction to open data

STEFANIE POSAVECstefanieposavec.co.uk

Page 70: An introduction to open data

“Air Transformed is a series of wearable data objects that communicate this physical burden in different ways. Though seemingly decorative, they are based entirely on open air quality data

from Sheffield, UK, a former steelmaking city and notorious for its bad air.”

stefanieposavec.co.uk/data/#/airtransformed

Page 71: An introduction to open data
Page 72: An introduction to open data
Page 73: An introduction to open data
Page 74: An introduction to open data

Participation & self-empowerment

@sjenkinson

Page 75: An introduction to open data

LINKED DATA

Page 76: An introduction to open data

New knowledge from combined data sources and patterns in large

data volumes

@sjenkinson

Page 77: An introduction to open data
Page 78: An introduction to open data
Page 79: An introduction to open data
Page 80: An introduction to open data
Page 81: An introduction to open data

Misrepresentation

Page 82: An introduction to open data
Page 83: An introduction to open data

tylervigen.com/spurious-correlations

Page 84: An introduction to open data

tylervigen.com/spurious-correlations

Page 85: An introduction to open data

Combining data sets & licences

clipol.org/tools/compatibility

Page 86: An introduction to open data

PUBLISHING OPEN DATA

Page 87: An introduction to open data

“There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know

there are some things we do not know. But there are also unknown unknowns – the

ones we don't know we don't know.”

en.wikipedia.org/wiki/There_are_known_knowns

Page 88: An introduction to open data
Page 89: An introduction to open data

STEP ONE Identification & planning

@sjenkinson

Page 90: An introduction to open data

Clear licensing & usage information

Structure & quality

A plan for support

@sjenkinson

Page 91: An introduction to open data
Page 92: An introduction to open data

Accuracy

Page 93: An introduction to open data

STEP TWO Extracting & cleaning

@sjenkinson

Page 94: An introduction to open data

Data privacy & the individual

Page 95: An introduction to open data

openrefine.org

Page 96: An introduction to open data

STEP THREE Sharing

@sjenkinson

Page 97: An introduction to open data

FIVE STAR DATA5stardata.info

Page 98: An introduction to open data

★ Make your data available on the web (in whatever format) under an open license.

★★ Make it available as structured data (e.g., Excel instead of image scan of a table).

★★★ Use non-proprietary formats (e.g., CSV instead of Excel).

★★★★ Use URIs to denote things, so that people can point at your data.

★★★★★ Link your data to other data to provide context.

Page 99: An introduction to open data

OPEN DATA CERTIFICATEScertificates.theodi.org

Page 100: An introduction to open data
Page 101: An introduction to open data

IN CONCLUSION…

Page 102: An introduction to open data

1. Choose open data 2. Publish your data 3. Link it 4. Use standards 5. Promote freedom 6. Do some good 7. Be creative

Page 103: An introduction to open data
Page 104: An introduction to open data

@sjenkinson !

[email protected] !

recordssoundthesame.com

THANK YOU. Thank you to these lovely people for making their content open:

Linking Open Data cloud diagram 2014, by Max Schmachtenberg, Christian Bizer, Anja Jentzsch and Richard Cyganiak - lod-cloud.net

The Data Spectrum - theodi.org/data-spectrum

Doug McCune - dougmccune.com

Stefanie Posavec - stefanieposavec.co.uk

Data abstract painting - flickr.com/photos/rachubarama/2709346242

IE Market Share vs Murder Rate - imgur.com/47D7zGq

Troy Marusek - flickr.com/photos/troymars/9113025616

The Roof of Wales - flickr.com/photos/stray_croc/4743302841

Fire Wall - flickr.com/photos/epleitez/1714341218

Money - flickr.com/photos/mikephotoart/12839909303

cc - flickr.com/photos/kalexanderson/7175627336

RDF - flickr.com/photos/gertcha/8292978031

Small Parts - flickr.com/photos/oskay/2156889157/

Hydrant - flickr.com/photos/pamhule/4677109732/

Upsala Glacier Retreat - flickr.com/photos/nasamarshall/10726540434/