playful explorations of public and personal data - oscon data 2011
DESCRIPTION
It’s easy to find and create data. But what are you going to do with it? Can I ask the world complex questions such as what’s the local crime rate, distance to metro, or rating of my local school? Can you combine these all together to rate houses you may want to buy? And how do you then connect back to your government and local businesses to engage in collaborative decision making.This talk with discuss how you should consider users and their personal interactions with data and information. We’ll also peel back the covers on how open source tools such as HBase, Cascading, Geos and Polymaps handle analyzing and streaming realtime data to maps and visualizations both on the web and to mobile devices.To illustrate what’s possible, we’ll dive through GeoCommons, a large online community of data sharing and community analytics that uses open source mapping visualization, Hadoop analysis, and mobile interfaces to provide this to the world. Users can even build and socialize their own analysis methods to share their expert knowledge with other users. We’ll also review how global organizations like the World Bank and United Nations are using these tools to connect with citizens in developing countries to empower them to make decisions on building investment and understanding how climate science may affect their areas.TRANSCRIPT
Pub
lic &
Personal
DataPlayful
Explorations
Andrew Turner@ajturner
Data
Open Government Data
flickr: jrparis
of the world80%live within cell coverage
flickr: jrparis
of the world80%live within cell coverage
and of the50%world own a mobile phone
flickr: jrparis
http://jess3.com/geosocial-universe-2/
unread POI’s 1000+
Black Friday, 2010
Social Media Trajectorieshttp://geocommons.com/maps/89371#
Personal Data
How to deal with this data? flickr: ejpphoto
Approachable, relevant, accessible
Sharing Visualizations http://flowingdata.com
Tools are often too difficult.
Basic Principles
Open Access
Basic Principles
Open Access
Visualize
Basic Principles
Open Access
Visualize
Analyze
Basic Principles
Power of Place
The role of mapsMaps Reflect & Define Reality
Waldseemüller Uniersalis Cosmographia
The role of mapsMaps Reflect & Define Reality“America”
Waldseemüller Uniersalis Cosmographia
Story Mapping
Navigation Marshall Islands stick chart19th century
Travel Guides
Spatial Operations
Buffer, Intersect, AggregationCorrelation, Prediction, Custom
Tools: geos, JTS, PostGIS
Spa$al Aggrega$on: Group data by arbitrary geographic boundaries
(Proprietary and confiden0al)
Spa$al Aggrega$on: Group data by arbitrary geographic boundaries
(Proprietary and confiden0al)
Spa$al Aggrega$on: Group data by arbitrary geographic boundaries
(Proprietary and confiden0al)
Spa$al Correla$on: Rela0onship between mul0ple datasets by geography
(Proprietary and confiden0al)
Spa$al Correla$on: Rela0onship between mul0ple datasets by geography
(Proprietary and confiden0al)
Spa$al Correla$on: Rela0onship between mul0ple datasets by geography
(Proprietary and confiden0al)
Spa$al Buffer and Distance Filter: Radius buffer from a point, line or polygon
(Proprietary and confiden0al)
Spa$al Buffer and Distance Filter: Radius buffer from a point, line or polygon
(Proprietary and confiden0al)
Spa$al Buffer and Distance Filter: Radius buffer from a point, line or polygon
(Proprietary and confiden0al)
Intersec$on: Overlap between two polygonal areas, lines or points
(Proprietary and confiden0al)
Intersec$on: Overlap between two polygonal areas, lines or points
(Proprietary and confiden0al)
Intersec$on: Overlap between two polygonal areas, lines or points
(Proprietary and confiden0al)
Intersec$on: Overlap between two polygonal areas, lines or points
(Proprietary and confiden0al)
Intersec$on: Overlap between two polygonal areas, lines or points
Use Cases:• Locate that a social media men0ons happened within a geo-‐fence
• Track mobile traffic entering a zip code
(Proprietary and confiden0al)
User-‐Defined Analysis: Custom equa0ons via intui0ve wizard interface
(Proprietary and confiden0al)
User-‐Defined Analysis: Custom equa0ons via intui0ve wizard interface
(Proprietary and confiden0al)
User-‐Defined Analysis: Custom equa0ons via intui0ve wizard interface
(Proprietary and confiden0al)
Traffic Accidents http://geocommons.com/maps/77512
Virtual Vehicle www.virtualvehicle.co
Crash ‘Likelihood’ Density www.virtualvehicle.co
Making it revelantflickr: qwertyuiop
neogeography |nē'ō-jē-ŏg'rə-fē|
geographical techniques and tools used for personal activities or for utilization by a non-expert group of users;
Psychogeography
Playful Explorations
http://geocommons.com
http://geocommons.com
67k+ unique datasetsbillions of features.
590,000+ geographic attributes
Acetate Layers
http://developer.geoiq.com
= 174,049,705,000
= 174,049,705,000
well... close enough
Open Access
Open Access
Visualize
Open Access
Visualize
Analyze
Citizens
Foursquare Crime
Citizens
Government
Social Government
Citizens
Government
Media
Journalists
Journalists
Social Media Indicatorshttp://geocommons.com/maps/51058
Social Media Indicatorshttp://geocommons.com/maps/51058
Citizens
Science
Media
Government
Research
Citizens
Science
MediaDevelopers
Government
Using Appcelerator’s open-source cross-mobile platform technology, mobile applications can quickly integrate with GeoIQ to access and create new data from the field.
GeoIQ & Appcelerator
GeoPace
HBase
GEOS
GeoPacing
GeoPace
Action
Results
Search
Loc(lat,lon)
HBase
GEOSstart
GeoPacing
GeoPace
Action
Results
Search
Loc(lat,lon)
HBase
GEOSstart
GeoPacing
GeoPace
Action
Results
Search
Loc(lat,lon)
HBase
GEOSstart
Cascading
GeoPacing
GeoPace
Action
Results
Search
Loc(lat,lon)
HBaseDB
DB
DB
GEOSstart
Cascading
GeoPacing
GeoPace
Action
Results
Search
Loc(lat,lon)
HBaseDB
DB
DB
GEOSstart
Cascading
MapMapMapMapMapMapMap
GeoPacing
Appcelerator
http://appcelerator.geoiq.com
Mobile Analysis on Mobile Devices
Citizens
Organizations
Science
MediaDevelopers
Enterprise
Social Media Dashboards MongoDBPolymaps
Organizations
Science
MediaDevelopers
Enterprise
Citizens
Hotness http://geocommons.com/maps/89561
Health http://geocommons.com/maps/77282
Traffic & Caffeine http://geocommons.com/maps/82402
More Technical Information
Open Geospatial Libraries for RubyChris HelmWednesday @ 5PM
