Workshop ReportBenchmarking Linked Data

Axel-Cyrille Ngonga NgomoIrini Fundulaki

Anastasia Krithara

Horizon 2020GA No 688227

01/12/2016–30/11/2018

ISWC 2016Kobe, Japan

November 10, 2016Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 1 / 5

Core Questions

Goal of BLINKProvide a forum where the community can gather to address these challenges.

Core questions1 Developers: How good is my tool?2 Vendors: Who is my tool good for?3 Users: Which tool(s) should I use for my

application?Many questions

1 Tools⇒ Bottlenecks, solved problems, distinctive

features2 Benchmarks

⇒ Novel bechmarks, limitations, updates3 Performance indicators

⇒ Limitations, correlations, innovations4 Comparative studies

⇒ Across benchmarks, fields of application

Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 2 / 5

Core Questions

Goal of BLINKProvide a forum where the community can gather to address these challenges.

Core questions1 Developers: How good is my tool?2 Vendors: Who is my tool good for?3 Users: Which tool(s) should I use for my

application?Many questions

1 Tools⇒ Bottlenecks, solved problems, distinctive

features2 Benchmarks

⇒ Novel bechmarks, limitations, updates3 Performance indicators

⇒ Limitations, correlations, innovations4 Comparative studies

⇒ Across benchmarks, fields of application

Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 2 / 5

Highlights

Insights1 Scalable application stacks for Linked Data in demand (Keynote)2 Increasing number of realistic benchmarks (Kotsev et al., Spasic et al.)3 Benchmarking as motor for improvements (Saveta et al., BioASQ, GERBIL)

Positive1 Emerging benchmarking platforms (GERBIL QA, HOBBIT)2 Increase of tool performance with benchmarks (OAEI, BioASQ, GERBIL)3 Uptake of methodology-based and data-driven benchmark design

Todo1 Need for benchmark repository (CKAN)2 Community effort necessary for uptake of more benchmarks (HOBBIT)3 Untackled real-world challenges (benchmarks coming up)

Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 3 / 5

Highlights

Insights1 Scalable application stacks for Linked Data in demand (Keynote)2 Increasing number of realistic benchmarks (Kotsev et al., Spasic et al.)3 Benchmarking as motor for improvements (Saveta et al., BioASQ, GERBIL)

Positive1 Emerging benchmarking platforms (GERBIL QA, HOBBIT)2 Increase of tool performance with benchmarks (OAEI, BioASQ, GERBIL)3 Uptake of methodology-based and data-driven benchmark design

Todo1 Need for benchmark repository (CKAN)2 Community effort necessary for uptake of more benchmarks (HOBBIT)3 Untackled real-world challenges (benchmarks coming up)

Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 3 / 5

Highlights

Insights1 Scalable application stacks for Linked Data in demand (Keynote)2 Increasing number of realistic benchmarks (Kotsev et al., Spasic et al.)3 Benchmarking as motor for improvements (Saveta et al., BioASQ, GERBIL)

Positive1 Emerging benchmarking platforms (GERBIL QA, HOBBIT)2 Increase of tool performance with benchmarks (OAEI, BioASQ, GERBIL)3 Uptake of methodology-based and data-driven benchmark design

Todo1 Need for benchmark repository (CKAN)2 Community effort necessary for uptake of more benchmarks (HOBBIT)3 Untackled real-world challenges (benchmarks coming up)

Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 3 / 5

That’s all Folks!

@hobbit_project

Axel NgongaAKSW Research Group

Augustusplatz 10, Room P90504109 Leipzig, Germany

ngonga@informatik.uni-leipzig.de@NgongaAxel

Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 4 / 5

Acknowledgment

This work was supported by grants from the EU H2020 Framework Programmeprovided for the project HOBBIT (GA no. 688227).

Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 5 / 5