Redesigned the inlined topology of files, now using geoxylic btrees
As a part of the general redesign of files, all files, not just small files, can inline some data directly in the metadata log. Originally, this was a single piece of inlined data or an inlined tree (shrub) that effectively acted as an overlay over the block/btree data. This is now changed so that when we have a block/btree, the root of the btree is inlined. In effect making a full btree a sort of extended shrub. I'm currently calling this a "geoxylic btree", since that seems to be a somewhat related botanical term. Geoxylic btrees have, at least on paper, a number of benefits: - There is a single lookup path instead of two, this simplifies code a bit and decreases lookup costs. - One data structure instead of two also means lfsr_file_t requires less RAM, since all of the on-disk variants can go into one big union. Though I'm not sure this is very significant vs stack/buffer costs. - The write path is much simpler and has less duplication (it was difficult to deduplicate the shrub/btree code because of how the shrub goes through the mdir). In this redesign, lfsr_btree_commit_ leaves root attrs uncommitted, allowing lfsr_bshrub_commit to finish the job via lfsr_mdir_commit. - We don't need to maintain a shrub estimate, we just lazily evict trees during mdir compaction. This has a side-effect of allowing shrubs to temporarily grow larger than shrub_size before eviction. NOTE THIS (fundamentally?) DOESN'T WORK - There is no awkwardly high overhead for small btrees. The btree root for two-block files should be able to comfortably fit in the shrub portion of the btree, for example. - It may be possible to also make the mtree geoxylic, which should reduce storage overhead of small mtrees and make better use of the mroot. All of this being said, things aren't working yet. Shrub eviction during compaction runs into a problem with a single pcache -- how do we write the new btrees without dropping the compaction pcache? We can't evict btrees in a separate pass becauce their number is unbounded...
This commit is contained in:
@@ -4295,7 +4295,7 @@ code = '''
|
||||
assert(i <= 2*N);
|
||||
|
||||
lfsr_binfo_t binfo;
|
||||
int err = lfsr_btraversal_read(&lfs, &btree, &traversal, &binfo);
|
||||
int err = lfsr_btree_traversalread(&lfs, &btree, &traversal, &binfo);
|
||||
assert(!err || err == LFS_ERR_NOENT);
|
||||
if (err == LFS_ERR_NOENT) {
|
||||
break;
|
||||
@@ -4447,7 +4447,7 @@ code = '''
|
||||
assert(i <= 2*N);
|
||||
|
||||
lfsr_binfo_t binfo;
|
||||
int err = lfsr_btraversal_read(&lfs, &btree, &traversal, &binfo);
|
||||
int err = lfsr_btree_traversalread(&lfs, &btree, &traversal, &binfo);
|
||||
assert(!err || err == LFS_ERR_NOENT);
|
||||
if (err == LFS_ERR_NOENT) {
|
||||
break;
|
||||
|
||||
Reference in New Issue
Block a user