a lot of data and toolsusers.ics.forth.gr/~fgeo/files/hobbit17night.pdf · holistic benchmarking of...
Post on 08-Jun-2020
3 Views
Preview:
TRANSCRIPT
http://project-hobbit.eu@hobbit_project
HOLISTIC BENCHMARKING
of Big Linked Data
This work was supported by a grant from the EU H2020 Framework Programme provided for the project HOBBIT (GA no. 688227).
V. Papakonstantinou1, T. Saveta1, G. Flouris1, I. Fundulaki1 and A.-C. Ngonga Ngomo2
{papv, jsaveta, fgeo, fundul}@ics.forth.gr, axel.ngonga@upb.de #1 FORTH-ICS, #2 Paderborn University
What is a Benchmark?Benchmarks are sets of tests to assess the performance of computer systems.
A lot of Data and Tools
• Which tool(s) should I use for my business use case?• Where are the current bottlenecks?• Which steps of the data lifecycle are critical?• Which solutions are available on the market?• Which key performance indicators are relevant?• How well should tools perform?• How do existing solutions perform w.r.t. relevant
indicators?
FORTH Tasks in HOBBIT
Versioning benchmark tests the ability ofversioning systems to efficiently manageevolving datasets and queries evaluatedacross the multiple versions of saiddatasets.
Spatial Benchmark tests the performanceof systems that deal with topologicalrelations proposed in the state of the artDimensionally Extended nine IntersectionModel (DE-9IM).
Linking/ Fusion EnrichmentQuality Analysis
Evolution/ Repair
Search/ Explore
ExtractionStorage/ Querying
Revision/ Authoring
Data Lifecycle
Data Collection
TomTom, USU, AGT,
IMEC, INFAI, ...
Measure Collection
Benchmark Creation
Linking Benchmark
KPIsTasks
KPIsTasks
KPIsTasksKPIs
TasksKPIsTasks
Storage, Query
Performance,precision, recall,
f-measure, ...
Versioning Benchmark
Streaming Benchmark
LIMES
QUAD
Virtuoso
Challenges
Participants/Community
HOBBIT Platform
Reports
… …
top related