hobbit a brief overview · 2019-07-03 · hobbit a brief overview...

HOBBITA Brief Overview

Michael Röder, Axel-Cyrille Ngonga Ngomo

DICE research groupInstitute of applied Informatics, Leipzig, Germany

University Paderborn, Germany(Horizon 2020, GA No 688227)

BDV PPP SummitRiga, June 27th, 2019

Michael Röder (InfAI) Project Overview 17/01/2019 1 / 21

IntroductionA Lot of Data

11http://www.ibmbigdatahub.com/infographic/four-vs-big-data


http://www.ibmbigdatahub.com/infographic/four-vs-big-data

IntroductionA Lot of Tools

2

2https://cdn.datafloq.com/cms/os_big_data_open_source_tools-v2.pngMichael Röder (InfAI) Project Overview 17/01/2019 3 / 21

https://cdn.datafloq.com/cms/os_big_data_open_source_tools-v2.png

IntroductionA Lot of Tools

33https://hackernoon.com/great-power-great-responsibility-the-2018-big-data-ai-landscape-

6a35bcf34f7f?gi=3d49c063deb8


IntroductionOne Question

Which tool(s) should I usefor my use case?


IntroductionMany Questions

Where are the current bottlenecks?Which steps of the data lifecycle arecritical?Which solutions are available on themarket?Which key performance indicatorsare relevant?How well should tools perform?How do existing solutions performw.r.t. relevant indicators?


IntroductionHOBBIT

Research project from 2015 – 2018(Horizon 2020, GA No 688227)Focus on Big Linked DataCover the business-critical steps ofthe Linked Data lifecycleUsed by a growing number ofcompaniesMature and maturing technologies


IntroductionAims

1 Gathered real requirementsFocussed on industrial requirementsGathered relevant performance indicatorsGathered relevant performance thresholdsGathered real datasets

2 Developed benchmarks based on real data3 Provided universal benchmarking platform

Comparable resultsHosted as a free-to-use online instance

4 Periodic benchmarking challenges andreporting

5 Created an association (Special Group 7 ofTask Force 6)


IntroductionAims


2 Developed benchmarks based on real data

3 Provided universal benchmarking platformComparable resultsHosted as a free-to-use online instance




IntroductionAims


2 Developed benchmarks based on real data3 Provided universal benchmarking platform

Comparable resultsHosted as a free-to-use online instance




Section 1

Project Highlights


Project HighlightsHOBBIT platform

PlatformController

Data Generator

Task Generator

Data Generator

Data Generator

Task Generator

Task Generator

Front End

Benchmarked System

data flowcreates component

StorageAnalysis

BenchmarkController

Evaluation Module

Eval. Storage

User ManagementRepository

Scalable open-source benchmarking platformLocal, distributed and remote (AWS) deploymentFirst FAIR platform for benchmarking Big Linked Data in a holistic manner


Project HighlightsProject Highlights

? 5 mimicking algorithms? 52 benchmarks? 200+ systems? 14 challenges

DEBS GC 2017 and 2018? 300+ users? 13K+ experiments


Section 2

Benchmarking Machine Learning


Benchmarking Machine LearningSML Benchmark v1 for DEBS GC 2017

The task: find anomalies in molding machine sensor data to predictmaintenance intervals (predictive maintenance).Mimicking algorithm based on real dataData was streamed as in the real worldParticipants had to use Markov Models to identify anomalies14 Participants, 7 made it into the last round



The task: predictions about ship routes based on AIS dataSpatio-temporal streaming dataPredictions for vessels’ destinations and arrival times


Section 3

Future Directions


Future Directions

KnowGraphs (Innovative Training Networks (ITN))4 years, starting in October 201915 Early-Stage Researchers (ESRs)HOBBIT will be used as central benchmarking platformFurther datasets will be integrated (e.g., UICML datasets)

RAKI (BMWi project)3 years, starting in September 2019HOBBIT will be used for evaluation

More projects pending

→ Further development of the HOBBIT platformHOBBIT is open for the community! Benchmarks, systems, datasets can beaddedNot limited to linked data


Future Directions

KnowGraphs (Innovative Training Networks (ITN))4 years, starting in October 201915 Early-Stage Researchers (ESRs)HOBBIT will be used as central benchmarking platformFurther datasets will be integrated (e.g., UICML datasets)

RAKI (BMWi project)3 years, starting in September 2019HOBBIT will be used for evaluation

More projects pending→ Further development of the HOBBIT platform

HOBBIT is open for the community! Benchmarks, systems, datasets can beaddedNot limited to linked data


Thank you

HOBBIT offersScalable benchmarkingBased on real world data in anExtendable,Open source platformFollowing the FAIR data principles

http://project-hobbit.eu/https://dice-research.org/about/


http://project-hobbit.eu/

https://dice-research.org/about/

hobbit a brief overview · 2019-07-03 · hobbit a brief overview...

Documents