i/o research @ gppd ufrgs · gppd ufrgs francieli zanon boito rodrigo virote kassick philippe o. a....
TRANSCRIPT
![Page 1: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/1.jpg)
I/O Research @ GPPD UFRGS
Francieli Zanon BoitoRodrigo Virote Kassick
Philippe O. A. NavauxFederal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
Yves DenneulinINRIA – LIG - University of Grenoble, France
![Page 2: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/2.jpg)
High Performance Computing
Weather forecast, seismic simulations, DNA sequencing, …
… they need to access and write data to files that are shared by all processes.
![Page 3: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/3.jpg)
Parallel File Systems
● Allow the access to shared files by all processing nodes
● Parallel access to data (focus on high performance)
● Transparent access: applications do not need to know
where the file is
● Examples: Lustre, PVFS (OrangeFS)
![Page 4: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/4.jpg)
Parallel File Systems
File System
Meta-data Server
Data Server
Client
Data Server Data Server
![Page 5: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/5.jpg)
Data Striping
Data Server Data Server Data Server
File
Data striping
![Page 6: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/6.jpg)
Data Striping
Data Server Data Server Data Server
File
0 21
3
Data striping
![Page 7: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/7.jpg)
Parallel Access
Data Server Data Server Data Server
Client
Parallel Access
File?
![Page 8: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/8.jpg)
Parallel Access
Data Server Data Server Data Server
Client
021
3
Parallel Access
![Page 9: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/9.jpg)
Parallel Access
Data Server Data Server Data Server
Client
021
3
File
Parallel Access
![Page 10: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/10.jpg)
Performance Issues
● Some access patterns are known to present
poor performance:
● Small and sparse accesses
● Small files, large number of files
● “Out-or-order” accesses (by offset order)
● Accesses not aligned with the stripe size
● Concurrency on the access
● ...
![Page 11: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/11.jpg)
I/O Optimizations
● Several techniques try to adapt the
applications' access patterns to improve
performance.
● Collective I/O
● Requests reordering and aggregation
● I/O forwarding and offloading
● ...
![Page 12: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/12.jpg)
Example: I/O Scheduling
● Applications access concurrently the shared
file system infrastructure
● I/O Scheduling: schedule requests to the file
system in order to minimize interference
![Page 13: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/13.jpg)
I/O Scheduling
Server
Application A
Client 0
Client 1
Client 2
Application B
Client 3
Client 4 File A File B
A0A1A2
B0B1
Concurrent access
![Page 14: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/14.jpg)
I/O Scheduling
Server
Application A
Client 0
Client 1
Client 2
Application B
Client 3
Client 4 File A File B
A0A1A2
B0B1
A0A1A2 B0B1
Access pattern at the server does not present good performance
Interference
![Page 15: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/15.jpg)
I/O Scheduling
Server
File A File B
A0A1A2 B0B1
A0A1A2B0B1The scheduler improves performance through requests reordering and aggregation
![Page 16: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/16.jpg)
I/O Research @ GPPD UFRGS
![Page 17: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/17.jpg)
AGIOS
● I/O Scheduling Library developed in our
research group
● Easily included in parallel file systems
● aIOLi algorithm for scheduling
![Page 18: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/18.jpg)
AGIOS
![Page 19: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/19.jpg)
Application-aware
● The file system servers do not have information
about the application
● Information is lost on the I/O stack
● Optimizations could be smarter with
information about access patterns
![Page 20: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/20.jpg)
AGIOS
● AGIOS = Application-guided I/O Scheduler
● Include information about the application on the
scheduler
● Through traces from previous executions
● Use information to guide the scheduler's choices
● Wait for incoming requests in order to aggregate
more
[Boito et al. 2013]
![Page 21: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/21.jpg)
AGIOS – Performance Improvements
● Aggregations ~21% bigger
● Performance improvements of ~25% on average
● Over the base scheduling algorithm (aIOLi)
● 46.3% on average over not using the scheduler
[Boito et al. 2013]
![Page 22: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/22.jpg)
I/O Optimizations
● Several optimizations, like the scheduler, work
on the assumption that contiguous accesses
are better than non-contiguous.
● Developed for hard disks
● Seek costs
![Page 23: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/23.jpg)
Solid State Drive (SSD)
● Non-volatile flash-based (mostly) storage
● No moving components
● more resistance to physical shocks
● less noise
● less heat dissipation
● less energy consumption
● Difference between sequential and random
accesses becomes less important
![Page 24: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/24.jpg)
Sequential to Random Ratio
Read Ratio Write Ratio
HDD 143.7 66.8
SSD1
11.0 3.1
SSD2
9.2 328.0
SSD3
2.4 151.6
SSD4
1.1 1.3
SSD5
3.2 1.5
Table from [Rajimwale, Prabhakaran and Davis 2009]
● The sequential to random
access time ratio is not
always smaller on SSDs
than on HDDs
● We cannot easily make
assumptions about their
performance
![Page 25: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/25.jpg)
Ongoing Research @ GPPD
● Storage devices profiling
● Use information about the devices in order to
select optimizations that will improve
performance
![Page 26: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/26.jpg)
Ongoing Research @ GPPD
● Hybrid storage infrastructures
● Larger, slower devices for storage space
● Smaller, faster devices for access speed
● Management is done by the file system
![Page 27: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/27.jpg)
Final Remarks
![Page 28: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/28.jpg)
Final Remarks
● I/O is an important issue on the path to exascale
● Applications have their performance impaired by I/O
operations
● Performance depends on the access pattern
● Several optimizations try to adjust the access
pattern
![Page 29: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/29.jpg)
Final Remarks
● With new technologies, assumptions on
devices' performance cannot be easily made
● All layers benefit from more information in
order to make better decisions
![Page 30: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/30.jpg)
I/O Research @ GPPD UFRGS
Francieli Zanon BoitoRodrigo Virote Kassick
Philippe O. A. NavauxFederal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
Yves DenneulinINRIA – LIG - University of Grenoble, France
![Page 31: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/31.jpg)
AGIOS – Performance Improvements
0
5
10
15
small large small largecontiguous non-contigous
0
1
2
3
4
5
write
read
AGIOS (base) AGIOS + predict
-0.1%32%
-5%
31%
35%
24%
30%17%
15%13%
28% 34%
19% 22%
[Boito et al. 2013]
![Page 32: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/32.jpg)
aIOLi scheduling algorithm
R00
R10
R2128K
R30
R432K
Requests of 32KBoffset
Step 1
[Lebre et al. 2006]
![Page 33: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/33.jpg)
aIOLi scheduling algorithm
Step 1
R00
R10
R2128K
R30
R432K File 1
File 2
File 3
Sort requests by type, offset and insert in queue
![Page 34: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/34.jpg)
aIOLi scheduling algorithm
Step 1
R00
R10
R2128K
R30
R432K File 1
File 2
File 3
Quantum = 0Q=0 Q=0 Q=0
Q=0
Q=0
![Page 35: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/35.jpg)
aIOLi scheduling algorithm
Step 1
R00
R10
R2128K
R30
R432K File 1
File 2
File 3
Perform aggregations Q=0 Q=0
Q=0
Q=0
![Page 36: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/36.jpg)
aIOLi scheduling algorithm
Step 1
R00
R10
R2128K
R30
R432K File 1
File 2
File 3
Quanta are increased by a fixed value
Q=32K Q=32K
Q=32K
Q=32K
![Page 37: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/37.jpg)
aIOLi scheduling algorithm
Step 1
R00
R10
R2128K
R30
R432K File 1
File 2
File 3
Q=32K Q=32K
Q=32K
Q=32K
Select request
- by offset order- FIFO between queues- quantum is large enough for the request size
![Page 38: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/38.jpg)
aIOLi scheduling algorithm
Step 1
R00
R2128K
R30
R432K File 1
File 2
File 3
Q=32K Q=32K
Q=32K
R10
Execution
![Page 39: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/39.jpg)
aIOLi scheduling algorithm
Step 2
R00
R2128K
R30
R432K File 1
File 2
File 3
Q=32K Q=32K
Q=32K
R10
Execution
R5160K
![Page 40: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/40.jpg)
aIOLi scheduling algorithm
R00
R2128K
R30
R432K File 1
File 2
File 3
Q=32K Q=32K
Q=32K
R10
Execution
R5160K
Step 2
![Page 41: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/41.jpg)
aIOLi scheduling algorithm
R00
R2128K
R30
R432K File 1
File 2
File 3
Q=32K Q=32K
Q=32K
R10
Execution
R5160K
Step 2
![Page 42: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/42.jpg)
aIOLi scheduling algorithm
R00
R2128K
R30
R432K File 1
File 2
File 3
Q=64K Q=64K
Q=64K
R10
Execution
R5160K
Step 2
![Page 43: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/43.jpg)
aIOLi scheduling algorithm
R00
R2128K
R30
R432K
File 1
File 2
File 3
Q=64K
Q=64K
R10
Execution
R5160K
Step 2
![Page 44: I/O Research @ GPPD UFRGS · GPPD UFRGS Francieli Zanon Boito Rodrigo Virote Kassick Philippe O. A. Navaux Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil Yves](https://reader036.vdocuments.net/reader036/viewer/2022090606/605beb27db3378413464fc9f/html5/thumbnails/44.jpg)
aIOLi scheduling algorithm
File 1
File 2
File 3
Execution
...