optimizing fuse for cloud storage · vault 2015 maxim patlasov linux kernel developer, parallels...
TRANSCRIPT
![Page 1: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/1.jpg)
Vault 2015
Maxim Patlasov Linux Kernel Developer, Parallels Inc.
Optimizing FUSE for Cloud Storage
![Page 2: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/2.jpg)
2
Agenda
1. Parallels Cloud Storage 2. FUSE concept 3. FUSE optimizations 4. Performance achieved 5. Future improvements
![Page 3: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/3.jpg)
Parallels Cloud Storage
![Page 4: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/4.jpg)
Parallels Hypervisor Virtualization
• OS flexibility • HW emulation • Bare metal installation
![Page 5: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/5.jpg)
Parallels Containers
• More efficient memory management
• More efficient caching reduces I/O
• CT resource management • Easy migration • Easy backups&snapshots
![Page 6: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/6.jpg)
6
Storage requirements
Key requirements for VM and Container’s needs: • Strong consistency • High performance • Fault tolerance • Fast recovery • Address all space from any node • Commodity hardware • In-flight reconfiguration and update
![Page 7: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/7.jpg)
7
Parallels Cloud Storage solution
Key decisions made: • Optimize for big files • Union of all local storages • Replication for fault tolerance • Keep data and metadata separately • Multiple metadata servers
![Page 8: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/8.jpg)
8
PStorage architecture
![Page 9: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/9.jpg)
FUSE
![Page 10: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/10.jpg)
10
FUSE framework
Kernel FUSE
/fuse /dev/fuse
Client application FUSE daemon
req req ack ack
![Page 11: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/11.jpg)
11
FUSE: Containers on PStorage
Kernel FUSE
/fuse /dev/fuse
FUSE daemon
Client application
per-container FS
virtual block device
image file
libpstorage
Parallels Cloud Storage
MDS CS CS
CS MDS
MDS
![Page 12: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/12.jpg)
FUSE optimizations
![Page 13: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/13.jpg)
13
Asynchronous direct IO
io_submit
Application: io_submit(&iocb1); io_submit(&iocb2);
Before: After:
kernel fuse
fuse daemon
actual IO
io_submit
kernel fuse
fuse daemon
actual IO time
io_submit
kernel fuse
io_submit
kernel fuse
fuse daemon
actual IO
![Page 14: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/14.jpg)
14
Synchronous direct IO
write
Application: fd = open(O_DIRECT); write(fd, buf, 1<<20);
Before: After:
kernel fuse: 128K
fuse daemon
actual IO . . .
kernel fuse: 128K
fuse daemon
actual IO time
8x:
write
kernel fuse: 8 x 128K
fuse daemon
actual IO
![Page 15: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/15.jpg)
15
Writeback cache
write
Application: buffered write(fd, buf, 1<<20); Before: After:
kernel fuse: 128K
fuse daemon
actual IO . . .
kernel fuse: 128K
fuse daemon
actual IO
time
8x:
write
kernel fuse: populate page cache
fuse daemon
actual IO
kernel writeback
kernel fuse: 8 x 128K
Key benefits: • Lower latency of write(2) • Parallel processing writeback
![Page 16: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/16.jpg)
16
Performance Comparison :: HW
iSCSI SAN Storage DELL EqualLogic PS6510E
48 SATA Disks: 1TB 7200rpm (Seagate ST31000524NS)
30 SATA Disks: 2TB 7200rpm (Seagate ST2000DM001)
+ 10 SSD for caching (Intel SSD 520)
x1 HW SAN EQL PS6510E x10 compute nodes
Network: 10Gbit (Dell Force10 S4810)
Network: 10Gbit (Brocade FastIron SuperX SX-F42XG)
vs.
(FUSE based) Parallels Cloud Storage
![Page 17: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/17.jpg)
17
PCS FASTER than
HW SAN
Just 10 nodes PCS cluster
faster than DELL EQL SAN
($97000) in most workloads
![Page 18: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/18.jpg)
FUSE: what’s next?
![Page 19: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/19.jpg)
19
FUSE: future improvements
Kernel FUSE
/fuse
/dev/fuse
FUSE daemon 0 App 0
… FUSE daemon 1
FUSE daemon N App 1
App M …
queue 0 queue 1 queue N …
• Variable message size (currently 128K) • Eliminate global lock • Multi-queue • CPU and NUMA affinity
![Page 20: Optimizing FUSE for Cloud Storage · Vault 2015 Maxim Patlasov Linux Kernel Developer, Parallels Inc. Optimizing FUSE for Cloud Storage](https://reader030.vdocuments.net/reader030/viewer/2022040419/5e00d647c4986c457e67da44/html5/thumbnails/20.jpg)
Q&A
20