freebase workshop, december 2009

Freebase WorkshopJamie TaylorNew York City, December 2009

Code available at: http://dev.mqlx.com/~jamie/nyc2009

Building a MQL Query

MQL

[{ "name" : null, "type" : "/tv/tv_program"}]

"type":"/type/object" is assumed"name" = /type/object/name

http://www.freebase.com/app/queryeditor

MQL

{"id":"/en/ncis",

"name" : null, "type" : "/tv/tv_program"}

MQL

{"id":"/en/ncis",

"name" : null, "type" : "/tv/tv_program",

"program_creator":[ ]}

"/program_creator" = /tv/tv_program/program_creator

MQL

{"id":"/en/ncis",

"name" : null, "type" : "/tv/tv_program",

"program_creator":[{ "id":null, "name":null }]}

TVPerformance

Mark Harmon

Jethro Gibbs{

"id":"/en/ncis", "name" : null, "type" : "/tv/tv_program",

"program_creator":[{ "id":null, "name:null }],"regular_cast":[{

"actor":null, "character":null

}]}

{"id":"/en/ncis",

"name" : null, "type" : "/tv/tv_program",

"program_creator":[{ "id":null, "name:null }],"regular_cast":[{

"actor":null, "character":null

}]}

TVPerformance

Mark Harmon

Jethro Gibbs{

"id":"/en/ncis", "name" : null, "type" : "/tv/tv_program",

"program_creator":[{ "id":null, "name:null }],"regular_cast":[{

"actor": {"id":null, "name":null}, "character":null

}]}

{"id":"/en/ncis",

"name" : null, "type" : "/tv/tv_program",

"program_creator":[{ "id":null, "name:null }],"regular_cast":[{

"actor": {"id":null, "name":null}, "character":null

}],"spin_offs":[ ]

}

{"id":"/en/ncis",

"name" : null, "type" : "/tv/tv_program",

"program_creator":[{ "id":null, "name:null }],"regular_cast":[{

"actor": {"id":null, "name":null}, "character":null

}],"spin_offs":[{"id":null, "name":null,

"air_date_of_first_episode":null}]}

Accessing Freebase Services via PHP

Service Requests in PHP

• Using cURL to make external service requests

$topicid = "/en/ncis";$widgeturl = "http://www.freebase.com/widget/topic?id= $topicid&mode=i&panes=image,article_props";

$ch = curl_init();curl_setopt($ch, CURLOPT_URL, $widgeturl);curl_setopt($ch, CURLOPT_HEADER, 0);curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);$resultstr = curl_exec($ch);curl_close($ch);

print $resultstr;

Accessing External Services via MQL Extensions

A Graph of Entities

A Graph of Services

http://www.myspace.com/shakira

http://www.facebook.com/shakira

http://twitter.com/shakira

http://www.daylife.com/topic/Shakira

http://www.bestbuy.com/site/She+Wolf…

http://www.guardian.co.uk/music/shakira

http://www.last.fm/music/Shakira

http://www.netflix.com/RoleDisplay/Shakira/20046629

eMQL: MQL Extensions

• Request data from other service providers• Services accessed transparently within MQL query

• Use Freebase Keys/Properties in external service request

• Uses:

• real-time data services

• Stock quotes, sensor data

• premium services

• requests can specify private "api keys" for the service

• fine grained & authoritative data sources

eMQL Query

{  "id":   "/en/ibm",  "type": "/business/company",  "ticker_symbol": [{    "stock_exchange": null,    "ticker_symbol":null  }]}

ticker

"ibm"

{  "id":   "/en/ibm",  "type": "/business/company",  "ticker_symbol": [{    "stock_exchange": null,    "ticker_symbol":null, "quote":null  }]}

eMQL Query

ticker

"ibm"

{  "id":   "/en/ibm",  "type": "/business/company",  "ticker_symbol": [{    "stock_exchange": null,    "ticker_symbol":null, "quote":null  }]}

eMQL Query

ticker

"ibm"

"pre"

"fetch"

eMQL Service Request• Pre

• Modify MQL query to provide additional information necessary to complete request

• e.g., retrieve specific namespace keys

• Fetch• Produce (retrieve, calculate, format) value for the

eMQL property

• Reduce (optional)• Create a result that looks across all results returned

in the query

• Help

• Document how the property is used

MQL Read Service http://api.freebase.com/api/service/mqlread?query= {"query":{"id":"/en/ncis", "name":null}}

http://api.freebase.com/api/service/mqlread?queries= {"q0":{"query":{"id":"/en/ncis", "name":null}}} {"q1":{"query":{"id":"/en/blade_runner", "name":null}}}

MQL Requests in PHP

$topicid = "/en/ncis";$simplequery = array('id'=>$topicid, 'name'=>null);$queryarray = array('q1'=>array('query'=>$simplequery)); #query envelope

$jsonquerystr = json_encode($queryarray);$mqlurl = "http://www.freebase.com/api/service/mqlread?queries=". $jsonquerystr;

$ch = curl_init();curl_setopt($ch, CURLOPT_URL, $mqlurl);curl_setopt($ch, CURLOPT_HEADER, 0);curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);$resultstr = curl_exec($ch);curl_close($ch); $resultarray = json_decode($resultstr, true); #true:give us an array

$topicname = $resultarray["q1"]["result"]["name"]

{"id":"/en/ncis", "name":null}

Demo eMQL Extensions

stock quote

@tag references

TV Episode Adapter

{  "id":"/en/ncis",  "/user/jamie/extension/tv_episode": null}

"name":"Faith""airdate": "Dec/15/2009"

"id":"/authority/tvrage/episode/1064868143"

Query:

DesiredData:

TV Rage Episode Adapter

Show ID@4628Show Name@NCISShow URL@http://www.tvrage.com/NCISPremiered@2003Started@Sep/23/2003Ended@Latest Episode@07x09^Child's Play^Nov/24/2009Next Episode@07x10^Faith^Dec/15/2009RFC3339@2009-12-15T20:00:00-5:00GMT+0 NODST@1260921600Country@USAStatus@Returning SeriesClassification@ScriptedGenres@Action | Crime | Drama | Military/WarNetwork@CBSAirtime@Tuesday at 08:00 pmRuntime@60

http://services.tvrage.com/tools/quickinfo.php?show=NCIS

Tim

e

TV

Ep

iso

de

Ad

ap

ter

eM

QL D

rive

r

pre

updated MQLquery

MQ

L Q

ue

ry

run MQLquery

MQLresults

fetch

Extend MQL query to retrieveTV Rage Key (program identifier)

TV

Ra

ge

Se

rvic

e

get seriesdata

next episodenumber,

name,date

nextepisode

URL

get episodedata

query dispatched

format results

results

returnqueryresults

TV RageeMQL

AdapterLife Cycle

Using Foreign Identifiers to locate Freebase Topics

• /authority Namespace• Organizations managing stable entity identifiers

• /source - identifiers only appearing in URLs

• Keys can be for URLs to other sites• And vice-versa!

URLs and Freebase Keys

• http://www.imdb.com/title/tt0083658• /authority/imdb/title/tt0083658• http://www.rottentomatoes.com/alias?type=imdbid&s=0083658

• http://dbpedia.org/resource/Blade_Runner• /wikipedia/en/Blade_Runner• http://en.wikipedia.org/wiki/Blade_Runner

• http://musicbrainz.org/artist/2c4dae8c-e591-49e0-9c5a-62b310a15788.html• /authority/musicbrainz/2c4dae8c-

e591-49e0-9c5a-62b310a15788• http://www.bbc.co.uk/music/artists/2c4dae8c-

e591-49e0-9c5a-62b310a15788

http://ids.freebaseapps.com

Using Freebase RDF URIs

http://rdf.freebase.com/ns/<freebase-key>

• Performs content negotiation (HTTP ACCEPT Header)

• Produces HTML for standard browsers

• Produces RDF if you ask for it

• Easily translated to MQL style Freebase identifiers

http://rdf.freebase.com/ns/en.ncis = /en/ncis

Entity Extractors

• Zemanta• Freebase RDF URIs• Wikipedia Links• IMDB identifiers

• Orchestr8• Freebase RDF URIs• DBPedia RDF URIs• MusicBrainz identifiers• Crunchbase identifiers

• OpenCalais• Freebase RDF URIs• DBPedia RDF URIs

Getting Started++• Freebase Documentation Hub

• http://www.freebase.com/docs

• Developer Mailing List• http://freebase.markmail.org/search/?q=list:com.freebase.developers

• Schema/Modeling assistance• Data Modeling Mailing List

• http://freebase.markmail.org/search/?q=list:com.freebase.data-modeling

• Real Time help on IRC• Freenode #freebase

• Freebase Happenings• http://blog.freebase.com