a lot of data and toolsusers.ics.forth.gr/~fgeo/files/hobbit17night.pdf · holistic benchmarking of...

http://project-hobbit.eu @hobbit_project HOLISTIC BENCHMARKING of Big Linked Data This work was supported by a grant from the EU H2020 Framework Programme provided for the project HOBBIT (GA no. 688227). V. Papakonstantinou 1 , T. Saveta 1 , G. Flouris 1 , I. Fundulaki 1 and A.-C. Ngonga Ngomo 2 {papv, jsaveta, fgeo, fundul}@ics.forth.gr, [email protected] #1 FORTH-ICS, #2 Paderborn University What is a Benchmark? Benchmarks are sets of tests to assess the performance of computer systems. A lot of Data and Tools • Which tool(s) should I use for my business use case? • Where are the current bottlenecks? • Which steps of the data lifecycle are critical? • Which solutions are available on the market? • Which key performance indicators are relevant? • How well should tools perform? • How do existing solutions perform w.r.t. relevant indicators? FORTH Tasks in HOBBIT Versioning benchmark tests the ability of versioning systems to efficiently manage evolving datasets and queries evaluated across the multiple versions of said datasets. Spatial Benchmark tests the performance of systems that deal with topological relations proposed in the state of the art Dimensionally Extended nine Intersection Model (DE-9IM). Linking/ Fusion Enrichment Quality Analysis Evolution/ Repair Search/ Explore Extraction Storage/ Querying Revision/ Authoring Data Lifecycle Data Collection TomTom, USU, AGT, IMEC, INFAI, ... Measure Collection Benchmark Creation Linking Benchmark KPIs Tasks KPIs Tasks KPIs Tasks KPIs Tasks KPIs Tasks Storage, Query Performance, precision, recall, f-measure, ... Versioning Benchmark Streaming Benchmark LIMES QUAD Virtuoso Challenges Participants/Community HOBBIT Platform Reports … …

Upload: others

Post on 08-Jun-2020

3 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

http://project-hobbit.eu@hobbit_project

HOLISTIC BENCHMARKING

of Big Linked Data

This work was supported by a grant from the EU H2020 Framework Programme provided for the project HOBBIT (GA no. 688227).

V. Papakonstantinou1, T. Saveta1, G. Flouris1, I. Fundulaki1 and A.-C. Ngonga Ngomo2

{papv, jsaveta, fgeo, fundul}@ics.forth.gr, [email protected] #1 FORTH-ICS, #2 Paderborn University

What is a Benchmark?Benchmarks are sets of tests to assess the performance of computer systems.

A lot of Data and Tools

• Which tool(s) should I use for my business use case?• Where are the current bottlenecks?• Which steps of the data lifecycle are critical?• Which solutions are available on the market?• Which key performance indicators are relevant?• How well should tools perform?• How do existing solutions perform w.r.t. relevant

indicators?

FORTH Tasks in HOBBIT

Versioning benchmark tests the ability ofversioning systems to efficiently manageevolving datasets and queries evaluatedacross the multiple versions of saiddatasets.

Spatial Benchmark tests the performanceof systems that deal with topologicalrelations proposed in the state of the artDimensionally Extended nine IntersectionModel (DE-9IM).

Linking/ Fusion EnrichmentQuality Analysis

Evolution/ Repair

Search/ Explore

ExtractionStorage/ Querying

Revision/ Authoring

Data Lifecycle

Data Collection

TomTom, USU, AGT,

IMEC, INFAI, ...

Measure Collection

Benchmark Creation

Linking Benchmark