ditas project presentation v1.0
TRANSCRIPT
![Page 1: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/1.jpg)
DITAS Project
Introduction
![Page 2: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/2.jpg)
Context
![Page 3: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/3.jpg)
Data-Intensive Applications
Easy to implement
scalable solutions
Data management is
totally demanded to the
cloud providers
Robust solutions
Latency could be
significant
No control over data
management
(security/privacy
compliance checking)
Total control over the
data management
Data latency reduced
Higher investments
No scalability
on Cloud on the Edge
![Page 4: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/4.jpg)
Mixing the two
Cloud On premise
Scalability
Reliability
Latency
Security
Compliance
Scalability
Reliability
Latency
Security
Compliance
DITAS
![Page 5: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/5.jpg)
DITAS Objectives
1. To improve the productivity of developers
2. To enhance the data management in mixed cloud/fog
environment
3. To improve the execution of data-intensive applications
through data movement strategies
4. To provide an innovative execution environment for data-
intensive applications
5. To maximize the impact on the market of developers and
adopters of data-intensive applications
![Page 6: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/6.jpg)
What is a DIA?
We call an application data-intensive if data is
its primary challenge:
the quantity of data,
the complexity of data,
or the speed at which it is changing [5]
Two perspectives:
Data production
Data consumption
![Page 7: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/7.jpg)
Our position
Source [5]
![Page 8: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/8.jpg)
Our case studies
Characteristic Industry 4.0 e-Health
Data size Depending on the sensors attached to
the machine, it generates 100MB –
600MB per day
GigaByte if characters, Terabyte if
images
Complexity of analysis TBD Taxonomy problems and source
hetherogeneity increase the analysis
complexity
Heterogeneity of data Low High
Number of data sources Between 1 and 3 Several , depending upon the
domain; on average 5 to 7 sources
Distribution of data Not distributed Depending upon the localization of
the data sources. For multicentric
clinical studies the sources may be
located in different , geographically
distinct institutions
Timeliness of processing Needs can vary from near real-time to
hours depending on the
service/application
No online query are required,
therefore the response time is not an
issue for complex analysis.
![Page 9: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/9.jpg)
s
s
s
ss
s
Our model
Store
Acquire
Consume
Dismiss
help you on your daily tasks on Data
Management
![Page 10: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/10.jpg)
Our model
Data sources
Data format
Transmission
protocols
s
s
s
ss
sStore
Acquire
Consume
Dismiss
help you on your daily tasks on Data
Management
![Page 11: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/11.jpg)
Our model
Where to store
DFS with IoT
Security/privacy
s
s
s
ss
sStore
Acquire
Consume
Dismiss
help you on your daily tasks on Data
Management
![Page 12: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/12.jpg)
Our model
Data analytics /
operational processes
Lambda/Kappa
architecture
s
s
s
ss
sStore
Acquire
Consume
Dismiss
help you on your daily tasks on Data
Management
![Page 13: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/13.jpg)
Our model
Delete
Freezing
s
s
s
ss
sStore
Acquire
Consume
Dismiss
help you on your daily tasks on Data
Management
![Page 14: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/14.jpg)
Dismiss
Our model
Data movement
is covered in all
the phasesStore
Acquire
Consume
Dismiss
help you on your daily tasks on Data
Management
![Page 15: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/15.jpg)
About the results
![Page 16: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/16.jpg)
Expected results
DITAS-SDK
To design and implement data-intensive applications
DITAS Execution Environment (DITAS-EE)
To run data-intensive applications where data movement is
working behind the stage
Data and computation movement techniques
To satisfy the developer requirements
Strategies for application improvement
To be sure that data movement strategies are properly
working
![Page 17: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/17.jpg)
Solution at a glance
DITAS
SDK
Device and cloud manager
Identity manager
Application deployment
manager
DITAS Execution
EnvironmentDITAS Execution
Environment
DITAS Execution
Environment
DITAS Execution
Environment
![Page 18: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/18.jpg)
DITAS as a service…
DITAS developer
PaaS
(design) (execution)
IaaS• Hide the complexity of the infrastructure
• Where the application is deployed
• What the developer uses
• Supports the design of DIA
• Enable the data movement
![Page 19: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/19.jpg)
IaaS in DITAS
IaaS
Common Accessible Framework
Virtual access to
data in the
federated storedData in the cloud store
![Page 20: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/20.jpg)
PaaS
PaaS in DITAS (Design) -
SDK
Data Utility definitionData Security and
Privacy
Virtual Data Container Definition
Application Profiling and Deployment strategies)
![Page 21: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/21.jpg)
PaaS
PaaS in DITAS (Execution
Environemnt – DitasEE)
Data movement
strategies
Data analytics
Execution Engine
Data movement enactor
Data monitoring
![Page 22: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/22.jpg)
About the work plan
![Page 23: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/23.jpg)
Milestones
General architecture
design and case studies
refinement
Detailed component architecture
First releaseFinal release and mature
market analysis
Final validation, joint
sustainability, and established
sustainability body
M6MS1 M12MS2 M18MS3 M30MS4 M36MS5
WP1 WP2
WP3
WP4
WP1
WP2
WP3
WP4
WP5
WP2
WP3
WP4
WP5
WP5
Iteration 1 Iteration 2
![Page 24: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/24.jpg)
How DITAS can help your
workflow
![Page 25: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/25.jpg)
Phase 1 – system knowledge
Device profiling
How to access
What is able to do
Data profiling
Which are the data
consumed/produced
Characteristics of the
data IaaS
![Page 26: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/26.jpg)
Phase 2 - development
Developer defines the data-processing pipeline (process) Data are seen as VDC
Agnostic from the platform (unless data does not exist)
Developer defines the QoS At which granularity
(process/function)
DITAS
SDK
IaaS
PaaS
![Page 27: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/27.jpg)
Phase 3 – deployment
Based on the system system knowledge functions are properly deployed
Balance between cost/benefits deployment
Questions: Do all the nodes know
the complete process?
Do we need to have a central node?
Application deployment
manager
DITAS Execution
EnvironmentDITAS Execution
Environment
![Page 28: Ditas Project Presentation v1.0](https://reader031.vdocuments.net/reader031/viewer/2022022415/58ee6b6a1a28abad778b45f7/html5/thumbnails/28.jpg)
Phase 3 – execution
(adaptive)
When data processing is in place data and computation can be moved Move data among
devices
Move computation among devices
Monitoring Centralized/distributed
Application deployment
manager
DITAS Execution
EnvironmentDITAS Execution
Environment