opening “big data challenge” data: some insights on our role in the story
TRANSCRIPT
Opening “Big Data Challenge” data: some insights on our role in the story
#openbigdata #bigdatachallenge
what was Telecom Italia “Big Data Challenge”?
December 2014
http://www.telecomitalia.com/tit/en/bigdatachallenge.html
“The Big Data Challenge contest is designed to stimulate the creation and development of innovative technological ideas in the field of Big Data.”
December 2014
there was a lot of interesting data, from 10 different data providers
(most refers to the period from November 2013 to December 2013)
http://www.telecomitalia.com/tit/en/bigdatachallenge/contest/infografica.html
December 2014
SpazioDati was the technological partner hosting the data distribution platform, dandelion.eu.
we took the raw data from the 10 different data providers…
December 2014
we cleaned up and normalized that data, to make it really REUSABLE, like a LEGO block…
December 2014
wherever possible, we built an API on that data…
http://apievangelist.com/2014/03/31/should-government-provide-download-or-api-of-government-data-resources/
December 2014
http://blog.spaziodati.eu/en/2014/10/21/spaziodati-at-iswc-2014-visit-our-booth-research-plans-available/
we used our internal data curation workflow to publish the data on dandelion.eu
December 2014
http://blog.spaziodati.eu/en/2014/07/24/using-openrefine-to-perform-text-mining-on-your-data-food-for-thoughts/
starting from OpenRefine to clean up the data easily, for example
December 2014
* reconcile and clean up the data* align the data model to our internal ontologies, using RDF skeletons
* export the RDF modelled using our rules
some examples
December 2014
Milan and Trento telephone grids
https://dandelion.eu/datagems/SpazioDati/milano-grid/description/
https://dandelion.eu/datagems/SpazioDati/milanotoday/description/
Milan and Trento press articles
Social Pulse in Milan
https://dandelion.eu/datagems/SpazioDati/social-pulse-milano/data/#?q={"$sort":[{"field":"entities","order":"desc"}],"$offset":0}
https://dandelion.eu/datagems/SpazioDati/social-pulse-trentino/data/#?q={"$sort":[{"field":"entities","order":"desc"}],"$offset":0}
Social Pulse in Trento
wherever useful, we linked the data to our internal
“Administrative Regions” dataset..
December 2014
beta
https://dandelion.eu/datagems/SpazioDati/administrative-regions/api/?$limit=10&$offset=0
December 2014
December 2014
from MilanoToday articles to Administrative regions
https://dandelion.eu/datagems/SpazioDati/milanotoday/description/
December 2014
that’s the idea: having more and more
CONTEXTUAL DATA
and now, on December 2014…
December 2014
http://theodi.fbk.eu/openbigdata/#team
we are not alone in this journey
December 2014
Big Data and Open Data: an old story…
http://www.opendatanow.com/2013/11/new-big-data-vs-open-data-mapping-it-out/#.VJhGqcAAJB
December 2014
Unleash your creativity: start playing with
#openbigdata
@SpazioDati
#bigdatachallenge
https://dandelion.eu/datamine/open-big-data/
December 2014