improving jffs2 ram usage and performance elc europe 2007 alexey korolev
TRANSCRIPT
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Agenda
• JFFS2 in brief
• JFFS2 efficiency on large volumes– Market data on flash densities– Volume size test data– Why JFFS2 is inefficient on large volumes
• Improving JFFS2 RAM usage and performance within existing architecture– Proposed improvement – File direct I/O as a fast solution to reduce RAM usage and increase
performance– Test data– Requirements and limitations– Current status and availability– Deferred write and JFFS2 RAM buffer as a RAM usage reduction prototype
• Q&A
Verdana regular 7pt.
Legal text goes here
Flash Products Group
JFFS2 in brief
• Journaling Flash File System version 2
• Supported in Linux for years since 2001.
• Works with NOR, NOR-ECC, NOR with programming regions, NAND and Dataflash
• Widely used in many configurations to provide FS interface to memory technology devices.
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Forecasted Flash Densities
2007 2008 2009 2010
NOR 25.8 31.5 40.2 48.8
NAND 46 79 146 239
0
50
100
150
200
250
300
2007 2008 2009 2010
NOR
NAND
Cellular Handsets Forecasted NOR and NAND Densities in MB
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Volume size test data
Volume used size (MB)
0
0.5
1
1.5
2
2.5
3
3.5
20MB 40MB 60MB 80MB 100MB 120MB 140MB 160MB 180MB
MB
JF
FS
2 R
AM
us
age
(M
B)
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Volume size test data
File size (MB)
0
0.5
1
1.5
2
2.5
3
3.5
2 4 6 8 10 12 14 16 18
Sec
File
op
en
tim
e (
se
c)
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Why Does JFFS2 Become Inefficient on Large Data Volumes?
JFFS2 assumes keeping nodes (fragment) descriptions in memory since the moment of getting file info.
1. JFFS2 RAM usage increases linearly as the number of nodes increases (Every data node takes 40bytes of RAM)
2. If a file has many fragments, file open time increases significantly because JFFS2 needs to read every header from flash
This is the architectural scalability issue of JFFS2
It is not possible to completely solve it without a complete file system redesign, however there are workarounds
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Improving RAM Usage and File Open Time Within Existing JFFS2 Architecture
Our approach to reducing RAM usage and decreasing file open time is conceptually simple (but effective):
Reduce the number of JFFS2 fragments
Let’s look at how this can be done…
Flash Products Group
Using File Direct I/O to Reduce RAM Usage and Increase Performance
Lets look at the file write procedure
Application requests towrite data
VFS
64KB
JFFS2file_ops.write()
do_sync_write()
add to VFS pg cache
JFFS2write_page()0-4KB
4-8KB
…
56-64KB
16 requests16 fragments
Flash Products Group
Using File Direct I/O to Reduce RAM Usage and Increase Performance
One way to reduce number of nodes (fragments) is to provide direct file IO interface support in JFFS2 and increase JFFS2 granularity
Application requests towrite data
VFS
64KB
JFFS2file_ops.write()
Write one fragment of data
Verdana regular 7pt.
Legal text goes here
Flash Products Group
20MB 40MB60MB 80MB
100MB120MB
140MB160MB
0
0.5
1
1.5
2
2.5
3
JFFS2 direct IO JFFS2
Test ResultsJFFS2 RAM Usage for Different Volume Sizes
Volume Size
JF
FS
2 R
AM
us
age
(M
B)
3.5X
Verdana regular 7pt.
Legal text goes here
Flash Products Group
2 4 6 8 10 12 14 16 18 20
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
JFFS2 direct IO JFFS2
Test Results
Stat Time Performance for Different File Sizes
File Size (MB)
Sta
t Tim
e(s
ec)
>4X
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Requirements and Limitations
• Application should write data in large chunks• If applications write data in small chunks, fragments won’t
be combined until garbage collector touches it • This model not effective if there are tons of tiny files • Avoiding VFS page cache use could have positive and negative
consequences• Using the file open flag is recommended*
• For media files it avoids wasting page cache space and this could positively affect on system performance
• For databases, logs, and documents, using the page cache is highly recommended
* We have patch providing reading through VFS cache while direct write
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Current status and availability
• The File direct I/O feature is ready for use (Linux-2.6.20)
• It can be downloaded from www.sourceforge.net/mtd-mods
• The patch size is <50Kb and can be easily ported to other JFFS2 versions
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Deferred Write and RAM Buffer as a RAM Usage Reduction Prototype
Application requests towrite data
VFS
6KB
JFFS2file_ops.write()
do_sync_write()
JFFS2Write RAM buffer
4KB
2KB
1KB
…
Merge nodes
Large chunks
Sync()JFFS2 flush RAM buffer
1KB
4KB
…
Compared to Direct IO, deferred write is effective even if applications write data in small chunks.
16
Verdana regular 7pt.
Legal text goes here
Flash Products Group
Conclusion
Patches to provide Direct I/O in JFFS2 are ready
+ Significantly reduce RAM usage and open time+ Easy applicable- Has restrictions on large write chunks
17
Intel Confidential Flash Products Group
Q & A
Questions ?
Link: http://sourceforge.net/mtd-modsmailto: [email protected]