Flash Filesystems

Flashfilesystems
Flash filesystems
MichaelOpdenacker ThomasPetazzoni FreeElectrons
Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec20,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/flashfilesystems Corrections,suggestions,contributionsandtranslationsarewelcome!
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
TheMTDsubsystem
MTD:MemoryTechnologyDevices(flash,ROM,RAM)
Linuxfilesysteminterface MTDUsermodules
UBI yaffs2 jffs2 Chardevice Blockdevice
FlashTranslationLayers forblockdeviceemulation Caution:patentedalgorithms!
Readonlyblockdevice
FTL
NFTL
INFTL
MTDChipdrivers
NORflash NANDflash DiskOnChipflash
RAMchips Blockdevice ROMchips Virtualmemory Virtualdevicesappearing asMTDdevices
Memorydevices hardware
MTDdevices
MTDdevicesarevisiblein/proc/mtd ThemtdchardrivercreatesacharacterdeviceforeachMTD deviceofthesystem
Usuallynamed/dev/mtdX,major90.Evenminorsforreadwrite access,oddminorsforreadonlyaccess Provideioctl()toeraseandmanagetheflash Usedbythemtdutils
ThemtdblockdrivercreatesablockdeviceforeachMTDdevice ofthesystem
Usuallynamed/dev/mtdblockX,major31.Minoristhenumber oftheMTDdevice Allowsread/writeblocklevelaccess.Butbadblocksarenot handled,andnowearlevelingisdoneforwrites.
3
MTDpartitioning
MTDdevicesareusuallypartitioned
Itallowstousedifferentareasoftheflashfordifferentpurposes: readonlyfilesystem,readwritefilesystem,backupareas, bootloaderarea,kernelarea,etc.
Unlikeblockdevices,whichcontainstheirownpartitiontable,the partitioningofMTDdevicesisdescribedexternally
Hardcodedintothekernelcode Specifiedthroughthekernelcommandline
EachpartitionbecomesaseparateMTDdevice
Differentfromblockdevicelabeling(hda3,sda2) /dev/mtd1iseitherthesecondpartitionofthefirstflashdevice,or thefirstpartitionofthesecondflashdevice
4
DefinitionofMTDpartitions
MTDpartitionsaredefinedinthekernel,intheboarddefinitions: arch/arm/machat91/boardusba9263.cexample:
staticstructmtd_partition__initdataek_nand_partition[]={ { .name="LinuxKernel", .offset=0, .size=SZ_16M, }, { .name="RootFS", .offset=MTDPART_OFS_NXTBLK, .size=120*SZ_1M, }, { .name="FS", .offset=MTDPART_OFS_NXTBLK, .size=120*SZ_1M, } }; 5
ModifyingMTDpartitions(1)
MTDpartitionscanfortunatelybedefined throughthekernelcommandline. FirstneedtofindthenameoftheMTDdevice. Lookatthekernellogatboottime: NANDdevice:ManufacturerID:0xec,ChipID: 0xda(SamsungNAND256MiB3,3V8bit) Scanningdeviceforbadblocks Baderaseblock2000at0x0fa00000 Creating3MTDpartitionson"atmel_nand": 0x000000000x01000000:"LinuxKernel" 0x010000000x08800000:"RootFS" 0x088000000x10000000:"FS"
6
ModifyingMTDpartitions(2)
Youcannowusethemtdpartskernelbootparameter Example:
mtdparts=atmel_nand:2m(kernel)ro,1m(rootfs)ro,(data)
We'vejustdefined3partitionsintheatmel_nanddevice:
kernel(2M) rootfs(1M) data
Partitionsizesmustbemultipleoftheeraseblocksize. Youcanusesizesinhexadecimaltoo.Rememberthebelowsizes: 0x20000=128k,0x100000=1m,0x1000000=16m roliststhepartitionasreadonly isusedtousealltheremainingspace.

7
mtdutils
AsetofutilitiestomanipulateMTDdevices
mtdinfotogetdetailedinformationaboutaMTDdevice flash_erasealltocompletelyeraseagivenMTDdevice flashcptowritetoNORflash nandwritetowritetoNANDflash UBIutilities Flashfilesystemimagecreationtools:mkfs.jffs2, mkfs.ubifs
Usuallyavailableasthemtdutilspackageinyourdistribution Seehttp://www.linuxmtd.infradead.org/
8
jffs2
SupportsbothNANDandNORflash Today'sstandardfilesystemforMTDflash Nicefeatures:ontheflycompression(savesstorage spaceandreducesI/O),powerdownreliable,wear levelingandECC. Drawbacks:doesn'tscalewell Mounttimedependingonfilesystemsize: thekernelhastoscanthewholefilesystemat mounttime,toreadwhichblockbelongstoeach file. NeedtousetheCONFIG_JFFS2_SUMMARYkernel optiontostoresuchinformationinflash.This dramaticallyreducesmounttime(from16sto0.8s fora128MBpartition). Standardfile API JFFS2 filesystem MTDdriver
Flashchip
9
jffs2Howtouse
OntheLinuxtarget NeedeitherthemtdutilsfromtheMTDproject,ortheir embeddedvariantsfromBusybox Eraseandformatapartitionwithjffs2: flash_eraseallj/dev/mtd2 Mountthepartition: mounttjffs2/dev/mtdblock2/mnt/flash Fillthecontentsbywriting (copyingfromNFSorfromexternalstorage) Otherpossibility:useajffs2image(seenextpagetoproduceit): flash_eraseall/dev/mtd2 nandwritep/dev/mtd2rootfs.jffs2
10
Howtocreateajffs2image
mkfs.jffs2commandavailableinthemtdutilspackage. Caution:unlikesomemkfscommands,itdoesn'tcreatea filesystem,butafilesystemimage. First,findtheeraseblocksizefromUbootnandinfo:
Device0:NAND256MiB3,3V8bit,sectorsize128KiB
Thencreatetheimageonyourworkstation: mkfs.jffs2padnocleanmarkers eraseblock=128drootfs/orootfs.jffs2 Thepadoptionpadsthejffs2imagecontents untiltheendofthefinaleraseblock. Itisfineifthejffs2imageissmallerthantheMTDpartition. Thejffs2filesystemwillusetheentirepartitionanyway. ThenocleanmarkersoptionisforNANDflashonly.

11
Mountingajffs2imageonyourhost
Usefultoeditjffs2imagesonyourdevelopmentsystem MountinganMTDdeviceasaloopdeviceisabitcomplextask. Here'sanexampleforjffs2,foryourreference: Firstfindtheeraseblocksizeusedtocreatethejffs2image. Let'sassumeitis128KiB(131072bytes). Createablockdevicefromtheimage losetup/dev/loop0root.jffs2 EmulateanMTDdevicefromablockdevice, usingtheblock2mtdkernelmodule modprobeblock2mtdblock2mtd=/dev/loop0,131072 Finally,mountthefilesystem(create/mnt/jffs2ifneeded) mounttjffs2/dev/mtdblock0/mnt/jffs2
12
Initializingjffs2partitionsfromUboot
Youmaynotwanttohavemtdutilsonyourtarget! CreateaJFFS2imageonyourworkstation IntheUBootprompt: Downloadthejffs2imagetoRAMwithtftp OrcopythisimagetoRAMfromexternalstorage (UbootunderstandsFATfilesystemsandsupportsUSBstorage) FlashitinsideanMTDpartition (exactinstructionsdependingonflashtype,NORorNAND, reusetheinstructionsusedtoflashyourkernel).Makesuretowriteonly thesizeoftheimage,notmore! Ifyoubootonajffs2rootfilesystem,addroot=/dev/mtdblock<x>and rootfstype=jffs2totheLinuxcommandlinearguments. Limitation:needtosplitthejffs2imageinseveralchunks ifbiggerthantheRAMsize.
13
yaffs2
http://www.yaffs.net/ MainlysupportsNANDflash Nocompression Wearleveling,ECC,powerfailureresistant Fastboottime Codeavailableseparatelythroughgit (DualGPL/Proprietarylicense fornonLinuxoperatingsystems)
MTDdriver Standardfile API YAFFS2 filesystem
Flashchip
14
yaffs2Howtouse
Eraseapartition: flash_eraseall/dev/mtd2 Thefilesystemisautomaticallyformattedatthefirstmount: mounttyaffs2/dev/mtdblock2/mnt/flash Imagescanbecreatedwithmkyaffstool,fromyaffsutils http://www.aleph1.co.uk/cgibin/viewvc.cgi/yaffs/utils/
15
UBI(1)
UnsortedBlockImages http://www.linuxmtd.infradead.org/doc/ubi.html VolumemanagementsystemontopofMTDdevices. Allowstocreatemultiplelogicalvolumes andspreadwritesacrossallphysicalblocks. Takescareofmanagingtheeraseblocksandwear leveling.Makesfilesystemeasiertoimplement.
16
UBI(2)
UBI Logical EraseBlocks MTD Physical EraseBlocks
Volume1
Volume2
LEB
LEB
LEB
LEB
LEB
LEB
LEB
PEB PEB PEB PEB PEB PEB PEB PEB PEB

Freeblock Freeblock
17
UBIFS
http://www.linuxmtd.infradead.org/doc/ubifs.html Thenextgenerationofthejffs2filesystem, fromthesamelinuxmtddevelopers. AvailableinLinux2.6.27 WorksontopofUBIvolumes Hasanoticeablemetadataoverheadonvery smallpartitions(4M,8M)
Standardfile API UBIFS UBI MTDdriver
Flashchip
18
UBIPreparation
Eraseyourflashpartitionwhilepreservingyourerasecounters ubiformat/dev/mtd1 Seehttp://www.linuxmtd.infradead.org/faq/ubi.htmlifyoufaceproblems Needtocreatea/dev/ubi_ctrlchardevice(ifyoudon'thaveudev) ThisspecialcharacterdeviceisusedbyotherUBIutilities Majorandminornumberallocatedinthekernel.Findthesenumbers in/sys/class/misc/ubi_ctrl/dev(e.g.:10:63) Orrunubinfo: UBIversion:1 CountofUBIdevices:1 UBIcontroldevicemajor/minor:10:63 PresentUBIdevices:ubi0
Thesestepsaredoneonceforall
19
UBIAttaching
AttachUBItoone(ofseveral)oftheMTDpartitions: ubiattach/dev/ubi_ctrlm1 Thiscommandcreatestheubi0device,whichrepresentthefull UBIspacestoredonMTDdevice1
FindthemajorandminornumbersusedbyUBI: cat/sys/class/ubi/ubi0/dev(e.g.253:0) CreatetheUBIdevicefile: mknod/dev/ubi0c2530 ThisUBIspacecancontainseveralvolumes
20
UBIVolumemanagement
Volumecreationwithubimkvol
ubimkvol/dev/ubi0Ntests116MiB ubimkvol/dev/ubi0Ntestm(maxavailablesize) Thevolumeisthenidentifiedasubi0:testforthemount/umount commands
Volumeremovalwithubirmvol
ubirmvol/dev/ubi0Ntest
21
UBIFSHowtouse
WhenaUBIvolumeiscreated,creatinganemptyUBIFS filesystemisjustamatterofmountingit
mounttubifsubi0:test/mnt/flash
ImagesofUBIFSfilesystemscanbecreatedusingthe mkfs.ubifsutility
mkfs.ubifsm512e128KiBc100r/opt/img ubifs.img CanbewrittentoaUBIvolumeusingubiupdatevolandthe /dev/ubiX_Ydevices
ImagesofafullUBIspace,containingseveralvolumescanbe createdusingtheubinizeutility
CanbewrittentoarawMTDusingnandwrite
22
SquashFS
http://squashfs.sourceforge.net/ Filesystemforblockstorage,soitdoesn'tsupporttheMTDAPI. However,asitisreadonly,itworksfinewithmtdblock,aslong astheflashdoesn'thaveanybadblocks Youcanuseitforthereadonlysectionsinyourfilesystem.
23
SquashFSHowtouse
Verysimple! Onyourworkstation,createyourfilesystemimage: mksquashfsrootdirrootdir.sqfs Caution:iftheimagealreadyexistsremoveitfirst, orusethenoappendoption. Eraseyourflashpartition: flash_eraseall/dev/mtd2 Makeyourfilesystemimageavailabletoyourdevice (NFS,copy,etc.)andflashyourpartition: ddif=rootdir.sqfsof=/dev/mtdblock2 Mountyourfilesystem: mounttsquashfs/dev/mtdblock2/mnt/flash
24
Ourbenchmarks
jffs2 Dramaticallyoutperformedbyubifsin mostaspects. Hugemount/boottimeunless CONFIG_SUMMARYisused. yaffs2 Alsooutperformedbyubifs. Maynotfitallyourdata Uglyfileremovaltime (poordirectoryupdate performance?) Memoryusagenotscaling ubifsleavesnoreason tosticktoyaffs2.
25
ubifs Greatperformanceinallcorner cases. SquashFS Bestornearbestperformance inallreadonlyscenarios.
Fullbenchmarkdetailson http://freeelectrons.com/pub/conferences/2008/elce/flashfilesystems.pdf
Conclusions
Convertyourjffs2partitionstoubifs! Itmayonlymakesensetokeepjffs2 forMTDpartitionssmallerthan10MB, incasesizeiscritical. Noreasonlefttouseyaffs2insteadofjffs2? YoumayalsouseSquashFStosqueezemore stuffonyourflashstorage.Advisabletouseiton topofUBI,toletallflashsectorsparticipateto wearleveling. SquashFS MTDblock MTDAPI UBI MTDdriver Flashchip
26
Issueswithflashbasedblockstorage
Flashstoragemadeavailableonlythroughablockinterface. Hence,nowaytoaccessalowlevelflashinterface andusetheLinuxfilesystemsdoingwearleveling. Nodetailsaboutthelayer(FlashTranslationLayer)theyuse. Detailsarekeptastradesecrets,andmayhidepoor implementations. Hence,itishighlyrecommendedtolimitthenumberofwritesto thesedevices.
27
Reducingthenumberofwrites
Ofcourse,donotuseyourflashstorageasswaparea (rareinembeddedsystemsanyway) Mountyourfilesystemsasreadonly,orusereadonlyfilesystems (SquashFS),wheneverpossible. KeepvolatilefilesinRAM(tmpfs) Usethenoatimemountoption,toavoidupdatingthefilesystem everytimeyouaccessafile.Oratleast,ifyouneedtoknowwhether fileswerereadaftertheirlastchange,usetherelatimeoption (defaultsettingsinceLinux2.6.30). Don'tusethesyncmountoption(commitswritesimmediately).Use thefsync()systemcallforperfilesynchronization. Youmaydecidetodowithoutjournaledfilesystems.Theycausemore writes,butarealsomuchmorepowerdownresistant(tradeoff).
28
Usefulreading
IntroductiontoJFFS2andLogFS: http://lwn.net/Articles/234441/ NiceUBIpresentationfromToshiba: http://freeelectrons.com/redirect/celfubi.html Documentationonthelinuxmtdwebsite: http://www.linuxmtd.infradead.org/
29
Filesystemchoicesummary
Volatile Volatile data? data?
No
Readonly Readonly files? files?
No Storagetype Storagetype
Block
Contains Contains flash? flash?
Yes MTD Yes choosesquashfs choosesquashfs
Yes
No
noatimeoption noatimeoption
chooseext2 chooseext2
Choosetmpfs Choosetmpfs
chooseUBIFS chooseUBIFS orJFFS2 orJFFS2
Chooseext3orext4 Chooseext3orext4
SeeDocumentation/filesystems/inkernelsourcesfordetails aboutallavailablefilesystems.
30
PracticallabFlashfilesysetms
Creatingpartitionsinyour internalflashstorage. Formatingthemainpartitionwith SquashFSonmtdblock. Usingjffs2forsystemdata.
31
Relateddocuments
Allourtechnicalpresentations onhttp://freeelectrons.com/docs Linuxkernel Devicedrivers Architecturespecifics EmbeddedLinuxsystemdevelopment

Howtohelp
Youcanhelpustoimproveandmaintainthisdocument... Bysendingcorrections,suggestions,contributionsand translations Byaskingyourorganizationtoorderdevelopment,consulting andtrainingservicesperformedbytheauthorsofthese documents(seehttp://freeelectrons.com/). Bysharingthisdocumentwithyourfriends,colleagues andwiththelocalFreeSoftwarecommunity. Byaddinglinksonyourwebsitetoouronlinematerials, toincreasetheirvisibilityinsearchengineresults.
Linuxkernel Linuxdevicedrivers Boardsupportcode Mainstreamingkernelcode Kerneldebugging EmbeddedLinuxTraining Allmaterialsreleasedwithafreelicense! UnixandGNU/Linuxbasics Linuxkernelanddriversdevelopment RealtimeLinux,uClinux Developmentandprofilingtools Lightweighttoolsforembeddedsystems Rootfilesystemcreation Audioandmultimedia Systemoptimization
FreeElectrons
Ourservices
CustomDevelopment Systemintegration EmbeddedLinuxdemosandprototypes Systemoptimization Applicationandinterfacedevelopment Consultingandtechnicalsupport Helpindecisionmaking Systemarchitecture Systemdesignandperformancereview Developmenttoolandapplicationsupport Investigatingissuesandfixingtoolbugs

Flash Filesystems

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Flash Filesystems

Uploaded by

Copyright:

Available Formats

Flashfilesystems

Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec20,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/flashfilesystems Corrections,suggestions,contributionsandtranslationsarewelcome!

FlashTranslationLayers forblockdeviceemulation Caution:patentedalgorithms!

RAMchips Blockdevice ROMchips Virtualmemory Virtualdevicesappearing asMTDdevices

Partitionsizesmustbemultipleoftheeraseblocksize. Youcanusesizesinhexadecimaltoo.Rememberthebelowsizes: 0x20000=128k,0x100000=1m,0x1000000=16m roliststhepartitionasreadonly isusedtousealltheremainingspace.

UBI Logical EraseBlocks MTD Physical EraseBlocks

PEB PEB PEB PEB PEB PEB PEB PEB PEB

Standardfile API UBIFS UBI MTDdriver

ubifs Greatperformanceinallcorner cases. SquashFS Bestornearbestperformance inallreadonlyscenarios.

Volatile Volatile data? data?

Readonly Readonly files? files?

Contains Contains flash? flash?

Yes MTD Yes choosesquashfs choosesquashfs

chooseUBIFS chooseUBIFS orJFFS2 orJFFS2

Creatingpartitionsinyour internalflashstorage. Formatingthemainpartitionwith SquashFSonmtdblock. Usingjffs2forsystemdata.

Allourtechnicalpresentations onhttp://freeelectrons.com/docs Linuxkernel Devicedrivers Architecturespecifics EmbeddedLinuxsystemdevelopment

You might also like