YAFFS: Difference between revisions

YAFFS
Developer(s)	Charles Manning
Full name	Yet Another Flash File System
Other
Supported; operating systems	Android, Firefox OS, Linux, Windows CE, pSOS, RTEMS, eCos, ThreadX, LCOS

Browse history interactively

← Previous edit Next edit →

Content deleted Content added

VisualWikitext

Inline

Revision as of 20:53, 18 March 2024

Yaffs (Yet Another Flash File System) is a file system designed and written by Charles Manning for the company Aleph One.

Yaffs1 was the first version of this file system and was designed for the then-current NAND chips with 512 byte page size (+ 16 byte spare (OOB; Out-Of-Band) area). Work started in 2002, and it was first released later that year. The initial work was sponsored by Toby Churchill Ltd, and Brightstar Engineering.

These older chips also generally allow 2 or 3 write cycles per page.^[1] YAFFS takes advantage of this: dirty pages are marked by writing to a specific spare area byte. Newer NAND flash chips have larger pages, first 2K pages (+ 64 bytes OOB), later 4K, with stricter write requirements. Each page within an erase block (128 kilobytes) must be written to in sequential order, and each page must be written only once.^{[citation needed]}

Designing a storage system that enforces a "write once rule" ("write once property") has several advantages.^[2]

YAFFS2 was designed to accommodate these newer chips. It was based on the YAFFS1 source code, with the major difference being that internal structures are not fixed to assume 512 byte sizing, and a block sequence number is placed on each written page. In this way older pages can be logically overwritten without violating the "write once" rule. It was released in late 2003.

YAFFS is a robust log-structured file system that holds data integrity as a high priority. A secondary YAFFS goal is high performance. YAFFS will typically outperform most alternatives.^[3] It is also designed to be portable and has been used on Linux, WinCE, pSOS, RTEMS, eCos, ThreadX, and various special-purpose OSes. A variant 'YAFFS/Direct' is used in situations where there is no OS, embedded OSes or bootloaders: it has the same core filesystem but simpler interfacing to both the higher and lower level code and the NAND flash hardware.

The YAFFS codebase is licensed both under the GPL and under per-product licenses available from Aleph One.

Yaffs is locked on a per-partition basis at a high level, allowing only one thread to write at any given time.^[4]

YAFFS1

There is no special procedure to initialize a YAFFS filesystem beyond simply erasing the flash memory. When a bad block is encountered, YAFFS follows the smart media scheme of marking the fifth byte of the block's spare area. Blocks marked as such remain unallocated from then on.^{[clarification needed]}

To write file data, YAFFS initially writes a whole page (chunk in YAFFS terminology) that describes the file metadata, such as timestamps, name, path, etc. The new file is assigned a unique object ID number; every data chunk within the file will contain this unique object ID within the spare area. YAFFS maintains a tree structure in RAM of the physical location of these chunks. When a chunk is no longer valid (the file is deleted, or parts of the file are overwritten), YAFFS marks a particular byte in the spare area of the chunk as ‘dirty’. When an entire block (32 pages) is marked as dirty, YAFFS can erase the block and reclaim the space. When the filesystem's free space is low, YAFFS consolidates a group of good pages onto a new block. YAFFS then reclaims the space used by dirty pages within each of the original blocks.

When a YAFFS system mounts a NAND flash device, it must visit each block to check for valid data by scanning its spare area. With this information it then reconstitutes the memory-resident tree data structure.

YAFFS2

YAFFS2 is similar in concept to YAFFS1, and shares much of the same code; the YAFFS2 code base supports YAFFS1 data formats through backward compatibility. The main difference is that YAFFS2 needs to jump through significant hoops to meet the "write once" requirement of modern NAND flash.^[5]

YAFFS2 marks every newly written block with a sequence number that is monotonically increasing. The sequence of the chunks can be inferred from the block sequence number and the chunk offset within the block. Thereby when YAFFS2 scans the flash and detects multiple chunks that have identical ObjectIDs and ChunkNumbers, it can choose which to use by taking the greatest sequence number. For efficiency reasons YAFFS2 also introduces the concept of shrink headers. For example, when a file is resized to a smaller size, YAFFS1 will mark all of the affected chunks as dirty - YAFFS2 cannot do this due to the "write once" rule. YAFFS2 instead writes a "shrink header", which indicates that a certain number of pages before that point are invalid. This lets YAFFS2 reconstruct the final state of the filesystem when the system reboots.

YAFFS2 uses a more abstract definition of the NAND flash allowing it to be used with a wider variety of flash parts with different geometries, bad block handling rules etc.

YAFFS2 later added support for checkpointing, which bypasses normal mount scanning, allowing very fast mount times. Performance will vary, but mount times of 3 seconds for 2 GB have been reported.^{[citation needed]}

References

^ Erasing a flash erase block sets all of its bits to 1s, and writing a write block (smaller than an erase block, but possibly bigger than a filesystem block) sets selected bits to 0s. One or two further writes to the block could be sustained if the bits being written to 0 were previously 1s in the write block. Writing a 0 to a bit which was already 0 risked making the 0 "stick", i.e. multiple erases could be needed to return the bit to a 1.^{[citation needed]} Needless to say, this multiple-write practice was not generally tested and guaranteed by flash vendors, and cannot work at all on non-SLC flash technologies.
^ Jack B. Dennis; Guang R. Gao; and Vivek Sarkar. "Collaborative Research: Programming Models and Storage System for High Performance Computation with Many-Core Processors". p. 4
^ "Flash filesystem benchmarks Linux 3.1".
^ "How Yaffs works | Yaffs - A Flash File System for embedded use". yaffs.net. Retrieved 2024-03-18.
^ "YAFFS 2 Specification and Development Notes".

http://www.yaffs.net/yaffs-history

External links

Official website
Introducing YAFFS, the first NAND-specific flash file system
Unyaffs: A simple program to unpack YAFFS2 images.
yaffs2utils: Utilities to create/extract a YAFFS2 image on Linux.

[1] Erasing a flash erase block sets all of its bits to 1s, and writing a write block (smaller than an erase block, but possibly bigger than a filesystem block) sets selected bits to 0s. One or two further writes to the block could be sustained if the bits being written to 0 were previously 1s in the write block. Writing a 0 to a bit which was already 0 risked making the 0 "stick", i.e. multiple erases could be needed to return the bit to a 1.^{[citation needed]} Needless to say, this multiple-write practice was not generally tested and guaranteed by flash vendors, and cannot work at all on non-SLC flash technologies.

[2] Jack B. Dennis; Guang R. Gao; and Vivek Sarkar. "Collaborative Research: Programming Models and Storage System for High Performance Computation with Many-Core Processors". p. 4

[3] "Flash filesystem benchmarks Linux 3.1".

[4] "How Yaffs works | Yaffs - A Flash File System for embedded use". yaffs.net. Retrieved 2024-03-18.

[5] "YAFFS 2 Specification and Development Notes".

[1]

[2]

[3]

[4]

[5]

@@ Line 1: / Line 1: @@
+{{Short description|File system}}
 {{Refimprove|date=October 2009}}
 {{Time-context|date=August 2013}}
@@ Line 22: / Line 23: @@
 | compression =
 | encryption =
-| OS =[[Android (operating system)|Android]], [[Firefox OS]], [[Linux]], [[Microsoft Windows CE|Windows CE]], [[pSOS (real-time operating system)|pSOS]], [[eCos]], [[ThreadX]]
+| OS =[[Android (operating system)|Android]], [[Firefox OS]], [[Linux]], [[Microsoft Windows CE|Windows CE]], [[pSOS (real-time operating system)|pSOS]], [[RTEMS]], [[eCos]], [[ThreadX]], LCOS
 }}
-'''Yaffs''' ('''[[Yet another|Yet Another]] Flash [[File system|File System]]''') was designed and written by Charles Manning, of [[Whitecliffs]], [[New Zealand]], for the company Aleph One.
+'''Yaffs''' ('''Yet Another Flash File System''') is a [[file system]] designed and written by Charles Manning for the company Aleph One.
-Yaffs1 was the first version of this file system and was designed for the then-current NAND chips with 512 [[byte]] page size (+ 16 byte spare (OOB;Out-Of-Band) area). Work started in 2002, and it was first released later that year. The initial work was sponsored by Toby Churchill Ltd, and Brightstar Engineering.
+Yaffs1 was the first version of this file system and was designed for the then-current NAND chips with 512 [[byte]] page size (+ 16 byte spare (OOB; Out-Of-Band) area). Work started in 2002, and it was first released later that year. The initial work was sponsored by Toby Churchill Ltd, and Brightstar Engineering.
-These older [[Integrated circuit|chips]] also generally allow 2 or 3 write cycles per page.<ref>Erasing a flash erase block sets all of its bits to 1s, and writing a write block (smaller than an erase block, but possibly bigger than a filesystem block) sets selected bits to 0s. One or two further writes to the block could be sustained if the bits being written to 0 were previously 1s in the write block. Writing a 0 to a bit which was already 0 risked making the 0 "stick", i.e. multiple erases could be needed to return the bit to a 1. Needless to say, this multiple-write practice was not generally tested and guaranteed by flash vendors, and cannot work at all on non-[[Multi-level_cell#Single-level_cell|SLC]] flash technologies.</ref> YAFFS takes advantage of this: [[dirty page]]s are marked by writing to a specific spare area byte. Newer NAND flash chips have larger pages, first 2K pages (+ 64 bytes OOB), later 4K, with stricter write requirements.  Each page within an erase block (128 kilobytes) must be written to in sequential order, and each page must be written only once.
+These older [[integrated circuit|chips]] also generally allow 2 or 3 write cycles per page.<ref>Erasing a flash erase block sets all of its bits to 1s, and writing a write block (smaller than an erase block, but possibly bigger than a filesystem block) sets selected bits to 0s. One or two further writes to the block could be sustained if the bits being written to 0 were previously 1s in the write block. Writing a 0 to a bit which was already 0 risked making the 0 "stick", i.e. multiple erases could be needed to return the bit to a 1.{{Citation needed|reason=the "flash memory" article never mentions this; do you have a reference?|date=May 2018}} Needless to say, this multiple-write practice was not generally tested and guaranteed by flash vendors, and cannot work at all on non-[[Multi-level cell#Single-level cell|SLC]] flash technologies.</ref> YAFFS takes advantage of this: [[dirty page]]s are marked by writing to a specific spare area byte. Newer NAND flash chips have larger pages, first 2K pages (+ 64 bytes OOB), later 4K, with stricter write requirements.  Each page within an erase block (128 kilobytes) must be written to in sequential order, and each page must be written only once.{{Citation needed|reason=the "flash memory" article never mentions this; do you have a reference?|date=May 2018}}
+Designing a storage system that enforces a "write once rule" ("write once property") has several advantages.<ref>
+Jack B. Dennis; Guang R. Gao; and Vivek Sarkar.
+[http://www.capsl.udel.edu/pub/doc/memos/memo088.pdf "Collaborative Research: Programming Models and Storage System for High Performance Computation with Many-Core Processors"].
+p. 4
+</ref>
 YAFFS2 was designed to accommodate these newer chips. It was based on the YAFFS1 source code, with the major difference being that internal structures are not fixed to assume 512 byte sizing, and a block sequence number is placed on each written page.  In this way older pages can be logically overwritten without violating the "write once" rule. It was released in late 2003.
-YAFFS is a robust [[log-structured file system]] that holds [[data integrity]] as a high priority. A secondary YAFFS goal is high performance. YAFFS will typically outperform most alternatives.<ref>{{cite web| title=Flash filesystem benchmarks Linux 3.1 |url=http://elinux.org/Flash_Filesystem_Benchmarks_3.1}}</ref> It is also designed to be portable and has been used on [[Linux]], [[WinCE]], [[pSOS (real-time operating system)|pSOS]], [[eCos]], [[ThreadX]], and various special-purpose OSes. A variant 'YAFFS/Direct' is used in situations where there is no OS, embedded OSes or bootloaders: it has the same core filesystem but simpler interfacing to both the higher and lower level code and the NAND flash hardware.
+YAFFS is a robust [[log-structured file system]] that holds [[data integrity]] as a high priority. A secondary YAFFS goal is high performance. YAFFS will typically outperform most alternatives.<ref>{{cite web| title=Flash filesystem benchmarks Linux 3.1 |url=http://elinux.org/Flash_Filesystem_Benchmarks_3.1}}</ref> It is also designed to be portable and has been used on [[Linux]], [[WinCE]], [[pSOS (real-time operating system)|pSOS]], [[RTEMS]], [[eCos]], [[ThreadX]], and various special-purpose OSes. A variant 'YAFFS/Direct' is used in situations where there is no OS, embedded OSes or bootloaders: it has the same core filesystem but simpler interfacing to both the higher and lower level code and the NAND flash hardware.
 The YAFFS codebase is licensed both under the [[GNU General Public License|GPL]] and under per-product licenses available from Aleph One.
+Yaffs is locked on a per-partition basis at a high level, allowing only one thread to write at any given time.<ref>{{Cite web |title=How Yaffs works {{!}} Yaffs - A Flash File System for embedded use |url=https://yaffs.net/documents/how-yaffs-works |access-date=2024-03-18 |website=yaffs.net}}</ref>
 ==YAFFS1==
 There is no special procedure to [[Disk formatting|initialize]] a YAFFS filesystem beyond simply erasing the flash memory.  When a bad block is encountered, YAFFS follows the smart media scheme of marking the fifth byte of the block's spare area.  Blocks marked as such remain unallocated from then on.{{clarify|date=October 2010}}
-To write file data, YAFFS initially writes a whole page (chunk in YAFFS terminology) that describes the file [[metadata]], such as [[timestamp]]s, name, path, etc.  The new file is assigned a unique object ID number; every data chunk within the file will contain this unique object ID within the spare area.  YAFFS maintains a tree structure in [[RAM]] memory of the physical location of these chunks.  When a chunk is no longer valid (the file is deleted, or parts of the file are overwritten), YAFFS marks a particular byte in the spare area of the chunk as ‘dirty’.  When an entire block (32 pages) is marked as dirty, YAFFS can erase the block and reclaim the space.  When the filesystem's free space is low, YAFFS consolidates a group of good pages onto a new block.  YAFFS then reclaims the space used by dirty pages within each of the original blocks.
+To write file data, YAFFS initially writes a whole page (chunk in YAFFS terminology) that describes the file [[metadata]], such as [[timestamp]]s, name, path, etc.  The new file is assigned a unique object ID number; every data chunk within the file will contain this unique object ID within the spare area.  YAFFS maintains a tree structure in [[random-access memory|RAM]] of the physical location of these chunks.  When a chunk is no longer valid (the file is deleted, or parts of the file are overwritten), YAFFS marks a particular byte in the spare area of the chunk as ‘dirty’.  When an entire block (32 pages) is marked as dirty, YAFFS can erase the block and reclaim the space.  When the filesystem's free space is low, YAFFS consolidates a group of good pages onto a new block.  YAFFS then reclaims the space used by dirty pages within each of the original blocks.
 When a YAFFS system [[Mount (computing)|mount]]s a [[NAND flash]] device, it must visit each block to check for valid data by scanning its spare area.  With this information it then reconstitutes the memory-resident tree data structure.
@@ Line 46: / Line 55: @@
 YAFFS2 is similar in concept to YAFFS1, and shares much of the same code; the YAFFS2 code base supports YAFFS1 data formats through backward compatibility.  The main difference is that YAFFS2 needs to jump through significant hoops to meet the "write once" requirement of modern NAND flash.<ref>{{cite web| title=YAFFS 2 Specification and Development Notes |url=http://www.yaffs.net/yaffs-2-specification-and-development-notes}}</ref>
-YAFFS2 marks every newly written block with a sequence number that is [[Monotonic function|monotonically increasing]]. The sequence of the chunks can be inferred from the block sequence number and the chunk offset within the block.   Thereby when YAFFS2 scans the flash and detects multiple chunks that have identical ObjectIDs and ChunkNumbers, it can choose which to use by taking the greatest sequence number.  For efficiency reasons YAFFS2 also introduces the concept of shrink headers.  For example when a file is resized to a smaller size, YAFFS1 will mark all of the affected chunks as dirty - YAFFS2 cannot do this due to the "write once" rule.  YAFFS2 instead writes a "shrink header", which indicates that a certain number of pages before that point are invalid.  This lets YAFFS2 reconstruct the final state of the filesystem when the system reboots.
+YAFFS2 marks every newly written block with a sequence number that is [[Monotonic function|monotonically increasing]]. The sequence of the chunks can be inferred from the block sequence number and the chunk offset within the block.   Thereby when YAFFS2 scans the flash and detects multiple chunks that have identical ObjectIDs and ChunkNumbers, it can choose which to use by taking the greatest sequence number.  For efficiency reasons YAFFS2 also introduces the concept of shrink headers.  For example, when a file is resized to a smaller size, YAFFS1 will mark all of the affected chunks as dirty - YAFFS2 cannot do this due to the "write once" rule.  YAFFS2 instead writes a "shrink header", which indicates that a certain number of pages before that point are invalid.  This lets YAFFS2 reconstruct the final state of the filesystem when the system reboots.
 YAFFS2 uses a more abstract definition of the NAND flash allowing it to be used with a wider variety of flash parts with different geometries, bad block handling rules etc.
-YAFFS2 later added support for [[checkpointing]], which bypasses normal mount scanning, allowing very fast mount times. Performance will vary, but mount times of 3 seconds for 2 [[gigabyte|GB]] have been reported.{{citationneeded|date=June 2014}}
+YAFFS2 later added support for [[checkpointing]], which bypasses normal mount scanning, allowing very fast mount times. Performance will vary, but mount times of 3 seconds for 2 [[gigabyte|GB]] have been reported.{{citation needed|date=June 2014}}
 ==See also==
-{{Portal|Free software}}
+{{Portal|Free and open-source software}}
 * [[List of file systems]]
 * [[JFFS]]
@@ Line 59: / Line 68: @@
 * [[UBIFS]]
 * [[LogFS]]
+* [[NILFS]], a New Implementation of a Log-structured File System
 * [[Open NAND Flash Interface Working Group]]
@@ Line 66: / Line 76: @@
 ==External links==
-*{{Official website|http://www.yaffs.net/}}
+*{{Official website|http://www.yaffs.net}}
-*[http://web.archive.org/web/00000000000000/http://www.linuxdevices.com/articles/AT9680239525.html Introducing YAFFS, the first NAND-specific flash file system]
+*[https://web.archive.org/web/20020924095833/http://www.linuxdevices.com/articles/AT9680239525.html Introducing YAFFS, the first NAND-specific flash file system]
 * [http://code.google.com/p/unyaffs/ Unyaffs]: A simple program to unpack YAFFS2 images.
 * [http://code.google.com/p/yaffs2utils/ yaffs2utils]: Utilities to create/extract a YAFFS2 image on Linux.