bmap: Simplified bmap configs, reduced to one LFS3_F_GBMAP flag
TLDR: This drops the idea of different bmap strategies/modes, and sorts
out most of the compile-time/runtime conditional bmap interactions.
---
Motivation: Benchmarking (at least up to the 32-bit word limit) has
shown the bmap will unlikely be a significant bottleneck, even on large
disks. The largest disks tend to be NAND, and NAND's ridiculous block
size limits pressure on block allocation.
There are still concerns for areas I haven't measured yet:
- SD/eMMC/FTL - Small blocks, so more pressure on block allocation. In
theory the logical block size can be artificially increased, but this
comes with a granularity tradeoff.
- I've only measured throughput, latency is a whole other story.
However, users have reported lfs3_fs_gc is useful for mitigating this,
so maybe latency is less of a concern now?
But while there may still be room for improvement via alternative bmap
strategies, the risk a concerning amount of complexity. Yes,
configuration gets more complicated, but the real issue is any bmap
strategies that try to track _deallocations_ (the original idea being
treediffing) risk falling leaking blocks if all cases aren't covered.
The current "bmap cache" strategy strikes a really nice balance where it
reduces _amortized_ block allocation -> ~O(log n) without RAM, while
retaining the safe, bug-resistant, single-source-of-truth properties
that come with lookahead-based allocation.
---
So, long story short, dropping other strategies, and now the presence of
the bmap is a boolean flag.
This is also the first format-specific flag:
- Define LFS3_BMAP to enable the bmap logic, but note by default the
bmap will still not be used.
- Define LFS3_YES_BMAP to force the bmap to be used.
- With LFS3_BMAP, passing LFS3_F_GBMAP to lfs3_format will include the
on-disk block-map.
- No flag is needed during mount, the presence of the bmap is determined
by the on-disk wcompat flags (LFS3_WCOMPAT_GBMAP). This also prevents
rw mounting if the bmap is not supported, but rdonly mounting is
allowed.
- Users can check if the bmap is in use via lfs3_fs_stat, which reports
LFS3_I_GBMAP in the flags field.
There's still some missing pieces, but these will be a bit more
involved:
- lfs3_fs_grow needs to be made bmap aware!
- We probably want something like lfs3_fs_mkgbmap and lfs3_fs_rmgbmap to
allow converting between bmap backed/not-backed filesystem images.
Code changes minimal:
code stack ctx
before: 37172 2352 684
after: 37172 (+0.0%) 2352 (+0.0%) 684 (+0.0%)
code stack ctx
bmap before: 38844 2456 800
bmap after: 38852 (+0.0%) 2456 (+0.0%) 800 (+0.0%)
This commit is contained in:
Christopher Haster
+5
-23
@@ -73,11 +73,7 @@ FLAGS = [
|
||||
('F_CKMETA', 0x00001000, "Check metadata checksums" ),
|
||||
('F_CKDATA', 0x00002000, "Check metadata + data checksums" ),
|
||||
|
||||
('F_BMAPMODE', 0x03000000, "On-disk block-map mode" ),
|
||||
('^_BMAPNONE', 0x00000000, "Don't use the bmap" ),
|
||||
('^_BMAPCACHE', 0x01000000, "Use the bmap to cache lookahead scans" ),
|
||||
('^_BMAPVFR', 0x02000000, "Use the bmap in VFR mode" ),
|
||||
('^_BMAPIFR', 0x03000000, "Use the bmap in IFR mode" ),
|
||||
('F_GBMAP', 0x01000000, "Use the global on-disk block-map" ),
|
||||
|
||||
# Filesystem mount flags
|
||||
('M_MODE', 1, "Mount's access mode" ),
|
||||
@@ -98,12 +94,6 @@ FLAGS = [
|
||||
('M_CKMETA', 0x00001000, "Check metadata checksums" ),
|
||||
('M_CKDATA', 0x00002000, "Check metadata + data checksums" ),
|
||||
|
||||
('M_BMAPMODE', 0x03000000, "On-disk block-map mode" ),
|
||||
('^_BMAPNONE', 0x00000000, "Don't use the bmap" ),
|
||||
('^_BMAPCACHE', 0x01000000, "Use the bmap to cache lookahead scans" ),
|
||||
('^_BMAPVFR', 0x02000000, "Use the bmap in VFR mode" ),
|
||||
('^_BMAPIFR', 0x03000000, "Use the bmap in IFR mode" ),
|
||||
|
||||
# GC flags
|
||||
('GC_MKCONSISTENT',0x00000100, "Make the filesystem consistent" ),
|
||||
('GC_LOOKAHEAD', 0x00000200, "Populate lookahead buffer" ),
|
||||
@@ -128,15 +118,11 @@ FLAGS = [
|
||||
('I_CKMETA', 0x00001000, "Metadata checksums not checked recently" ),
|
||||
('I_CKDATA', 0x00002000, "Data checksums not checked recently" ),
|
||||
|
||||
('I_BMAPMODE', 0x03000000, "On-disk block-map mode" ),
|
||||
('^_BMAPNONE', 0x00000000, "Mounted with LFS3_M_BMAPNONE" ),
|
||||
('^_BMAPCACHE', 0x01000000, "Mounted with LFS3_M_BMAPCACHE" ),
|
||||
('^_BMAPVFR', 0x02000000, "Mounted with LFS3_M_BMAPVFR" ),
|
||||
('^_BMAPIFR', 0x03000000, "Mounted with LFS3_M_BMAPIFR" ),
|
||||
('I_GBMAP', 0x01000000, "Global on-disk block-map in use" ),
|
||||
|
||||
('i_INMTREE', 0x00030000, "Committing to mtree" ),
|
||||
('i_INMODE', 0x00030000, "Btree commit mode" ),
|
||||
('^_INMTREE', 0x00010000, "Committing to mtree" ),
|
||||
('^_INBMAP', 0x00020000, "Committing to bmap" ),
|
||||
('^_INGBMAP', 0x00020000, "Committing to gbmap" ),
|
||||
|
||||
# Traversal flags
|
||||
('T_MODE', 1, "The traversal's access mode" ),
|
||||
@@ -203,11 +189,7 @@ FLAGS = [
|
||||
('WCOMPAT_RDONLY', 0x00000002, "Writing is disallowed" ),
|
||||
('WCOMPAT_DIR', 0x00000010, "Directory file types in use" ),
|
||||
('WCOMPAT_GCKSUM', 0x00001000, "Global-checksum in use" ),
|
||||
('WCOMPAT_GBMAP', 0x00006000, "Global block-map in use" ),
|
||||
('^_GBMAPNONE', 0x00000000, "Gbmap not in use" ),
|
||||
('^_GBMAPCACHE', 0x00002000, "Gbmap in cache mode" ),
|
||||
('^_GBMAPVFR', 0x00004000, "Gbmap in VFR mode" ),
|
||||
('^_GBMAPIFR', 0x00006000, "Gbmap in IFR mode" ),
|
||||
('WCOMPAT_GBMAP', 0x00002000, "Global on-disk block-map in use" ),
|
||||
('wcompat_OVERFLOW',
|
||||
0x80000000, "Can't represent all flags" ),
|
||||
|
||||
|
||||
Reference in New Issue
Block a user