Importing linked life science databases into Neo4j

Simon Jupp

Sample Phenotypes and Ontologies Team

European Bioinformatics Institute

jupp@ebi.ac.uk

Purpose of the workshop

• Introduce two alternate graph models • RDF graphs• Property graph

• Demonstrate a simple data integration use-case

• Show how Neo4j data import tools can be used to rapidly import life science data from public APIs

• Example Cypher for querying biological data

• Introduction to Neo4j sandboxes, Apoc procedures and tips for creating your own Neo4j guide

Some biological questions

“Differentially expressed genes in adult mice, bred in oxygen rich vs oxygen poor environments? Of this set, which biological processes (GO) are enriched?”

“Where are genes with antigen binding function differentially expressed, which disease and which associated pathways?”

“Get metformin associated pathways with differentially expressed genes, find any proteins that are targets for known diabetes drugs”

How do you go about answering these kinds of questions?...

… you go to the data

Literature & ontologies•Experimental Factor Ontology•Gene Ontology•BioStudies•Europe PMC

Chemical biology•ChEBI•ChEMBL•SureChEMBL

Molecular structures•Protein Data Bank in Europe•Electron Microscopy Data Bank

Gene, protein & metabolite expression•Expression Atlas•Metabolights•PRIDE•RNA Central

Protein sequences, families & motifs•InterPro•Pfam•UniProt

Genes, genomes & variation•Ensembl •Ensembl Genomes•GWAS Catalog•Metagenomics portal

Systems•BioModels•BioSamples•Enzyme Portal•IntAct•Reactome

Molecular Archives•European Nucleotide Archive•European Variation Archive•European Genome-phenome Archive•ArrayExpress

Data integration challenges • Heterogeneous formats and identifiers

• We invest heavily in mapping and cross-linking resources, but it’s still hard to integrate and query across internal/external resources. • Lots of effort doing mapping, each groups duplicate these

efforts

Standardise data publishing

• What is we could standardise the way we publish data? • Global identification systems (so we can identify the things

in our data)• Common semantics (talking about the same things)• A common query language to the data

Original vision of the Web

Information Management: A Proposal, Tim Berners-Lee, CERN, March 1989, May 1990, http://www.w3.org/History/1989/proposal.html

Relations

“Things”

Vocabularies

Early Web

Semantic Web ( or Linked Data)

"The Semantic Web is a webby way to link data"

“Turning the web into a global API”

“The existing web links documents, the semantic web links data”

“Shared meaning through ontologies”

The Linking Open Data cloud 2017

http://lod-cloud.net

RDF is for describing graphs

• 1995-2004 W3C develop specification for a vocabulary for Web meta-data called Resource Description Framework (RDF)

http://en.wikipedia.org/wiki/Barack_Obama

WebDocument

Structured data Publishing data as a graph

dbpedia:Barack_Obama

Human President of the United States

Honolulu

1961-08-04

birthplace

birthdate

position_heldtype

Anatomy of a triple statement

• All triples are composed of a subject, predicate and an object

Barack Obama

Honolulu

birth placeSubject

Predicate

Object

Identify things on the web

• Build on existing Web technology• global identifiers for resources (things) using URIs• URIs should resolve

http://dbpedia.org/page/Barack_Obama

http://dbpedia.org/page/Honolulu

http://dbpedia.org/property/birthPlace

Subject

Predicate

Object

Turning relational data to RDF –EBI Gene Expression Atlas database

Relational Data to RDF graph conversion•Give “things” URIs•Type “things” with ontologies•Link “things” to other related “things”

Stardog

Apache Jena

SesameVirtuoso

Allegrograph

OWLIM

Storing and querying RDF

• Optimized databases for RDF data

• SPARQL query language

Querying RDF with SPARQL

• W3C standard query language for querying RDF data• Query language for matching graph patterns in RDF• SPARQL endpoints – common API to query RDF data• ”Get all presidents of the united states?” from

https://query.wikidata.org/

PREFIX position_held: http://www.wikidata.org/prop/direct/P39PREFIX potus: http://www.wikidata.org/entity/Q11696SELECT ?label WHERE {

?subject position_held: potus: . ?subject rdfs:label ?label . filter (lang(?label) = "en")

}

RDF and the Property graphRDF graphs

dbpedia:Barack_Obama

Human President of the United States

Honolulu

“1961-08-04”xsd:datetime

birthplace

birthdate

position_heldtype

Every statement adds a new edge to the graphAll nodes are resources (with URIs) or literals (with types)

Property graphs (Neo4j)

“Barack_Obama”xsd:string

name

dbpedia:Barack_Obama{name: “Barack Obama”Type: “Human”}

Nodes and edges have internal structure

Honolulu

Birthplace{ Birthdate: “1961-08-04” }

Working with RDF and Neo4j• RDF great for publishing data

• SPARQL gives flexible access to query data

• But…• RDF schemas are often (necessarily) complex

• Expose the full underlying data semantics

• RDF comes with baggage that can be turn off for newomers

• Neo4j is for graphs• Easier to grasp for begniners• Powerful query language (Cypher)• Excellent third-party tools, community and developer

integrations

Working with RDF and Neo

• In this tutorial we will harness the publishing power of RDF• Combine with the simplicity and querying power of Neo4j

• Use Neo4j data import tools to rapidly import data from public SPARQL endpoints• Simplify the graph schema to fit a specific use case

Use-case

• Build a simple graph of gene-disease and drug-disease associations• Data from public resources (Ensembl, GWAS, ChEMBL)

Setup for workshop

• Sandbox Neo4j instance from https://neo4j.com/sandbox-v2/ • Optionally run your own local installation, but you’ll need

Apoc procedures installed to run

• Run the Neo4j guide

:play https://guides.neo4j.com/life-science-import