Reverted raw-byte comparisons for rbyd/btree namelookups
Implementing raw-byte name comparisons ended up having more negative
effects on implementation requirements than I thought it would:
1. We would never actually concatenate the did + name, as that would
require dynamic memory. Instead we need to express the concatenated
relationship using our internal lfsr_data_t representation.
I thought this wouldn't be too bad since we already have a
concatenated lfsr_data_t representation, but:
1. It was limited in scope, specifically only lfsr_data_prog was
supported. It's actually not even possible to implement
lfsr_data_read (I think) since we can't mutate the indirect
lfsr_data_ts.
2. It's not actually required. We really only use our concatenated
representation to coalesce file fragments. You could in theory
omit this representation at the cost of not being able to limit
inlined shrub overhead.
Asking all future littlefs implementations to implement a
concatenated data representation (or dynamically allocate D:) for the
basic task of file-name lookup is sort of a big ask.
2. A readonly implementation suddenly needs a toleb128 function.
Which is an unexpected implication of requiring raw-byte leb128
comparisons for file-name lookup.
3. Raw-byte comparisons require that dids are always stored in their
canonical encoding (smallest leb128), though this is probably a good
idea anyways.
And for what? A theoretical future-planned feature (content-tree)?
Let's think about the hypothetical content-tree for a second:
1. It's an advanced, opt-in feature. Which means higher code/storage-cost
should be expected.
2. Basicall all littlefs implementations need file-name lookup, so
keeping file-name lookup cheap is a much higher priority than the
opt-int content-tree.
3. Worst case, the content-tree, and any future named trees, can just
set did=0. This will cost one byte per name (and may leave room for
future extensions).
So I'm reverting this for now.
There is still time before stabilization, so if it becomes clear there
is a better way to implement name lookups, we can still change this.
(Optimistically, the content-tree may be implemented before
stabilization, since it currently looks like it's required for data
redundancy).
Code changes:
code stack
before: 34292 2896
after: 34028 (-0.8%) 2896 (+0.0%)
This commit is contained in:
+10
-23
@@ -131,18 +131,6 @@ def fromleb128(data):
|
||||
return word, i+1
|
||||
return word, len(data)
|
||||
|
||||
def toleb128(word):
|
||||
data = []
|
||||
word &= 0xffffffff
|
||||
while True:
|
||||
b, word = word & 0x7f, word >> 7
|
||||
if word:
|
||||
data.append(b | 0x80)
|
||||
else:
|
||||
data.append(b | 0x00)
|
||||
break
|
||||
return bytes(data)
|
||||
|
||||
def fromtag(data):
|
||||
data = data.ljust(4, b'\0')
|
||||
tag = (data[0] << 8) | data[1]
|
||||
@@ -880,7 +868,7 @@ class Rbyd:
|
||||
return True, -1, 0, None
|
||||
|
||||
# lookup by name
|
||||
def namelookup(self, name):
|
||||
def namelookup(self, did, name):
|
||||
# binary search
|
||||
best = (False, -1, 0, 0)
|
||||
lower = 0
|
||||
@@ -894,14 +882,16 @@ class Rbyd:
|
||||
# treat vestigial names as a catch-all
|
||||
if ((tag == TAG_NAME and rid-(w-1) == 0)
|
||||
or (tag & 0xff00) != TAG_NAME):
|
||||
did_ = 0
|
||||
name_ = b''
|
||||
else:
|
||||
name_ = data
|
||||
did_, d = fromleb128(data)
|
||||
name_ = data[d:]
|
||||
|
||||
# bisect search space
|
||||
if name_ > name:
|
||||
if (did_, name_) > (did, name):
|
||||
upper = rid-(w-1)
|
||||
elif name_ < name:
|
||||
elif (did_, name_) < (did, name):
|
||||
lower = rid + 1
|
||||
|
||||
# keep track of best match
|
||||
@@ -913,12 +903,12 @@ class Rbyd:
|
||||
return best
|
||||
|
||||
# lookup by name with this rbyd as the btree root
|
||||
def btree_namelookup(self, f, block_size, name):
|
||||
def btree_namelookup(self, f, block_size, did, name):
|
||||
rbyd = self
|
||||
bid = 0
|
||||
|
||||
while True:
|
||||
found, rid, tag, w = rbyd.namelookup(name)
|
||||
found, rid, tag, w = rbyd.namelookup(did, name)
|
||||
done, rid_, tag_, w_, j, d, data, _ = rbyd.lookup(rid, TAG_STRUCT)
|
||||
|
||||
# found another branch
|
||||
@@ -935,9 +925,6 @@ class Rbyd:
|
||||
|
||||
# lookup by name with this rbyd as the mroot
|
||||
def mtree_namelookup(self, f, block_size, did, name):
|
||||
# concatenate did + name
|
||||
name = toleb128(did) + name
|
||||
|
||||
# have mtree?
|
||||
done, rid, tag, w, j, d, data, _ = self.lookup(-1, TAG_MTREE)
|
||||
if not done and rid == -1 and tag == TAG_MTREE:
|
||||
@@ -949,7 +936,7 @@ class Rbyd:
|
||||
|
||||
# lookup our name in the mtree
|
||||
mbid, tag_, mw, data = mtree.btree_namelookup(
|
||||
f, block_size, name)
|
||||
f, block_size, did, name)
|
||||
if tag_ != TAG_MDIR:
|
||||
return False, -1, 0, None, -1, 0, 0
|
||||
|
||||
@@ -973,7 +960,7 @@ class Rbyd:
|
||||
mdir = self
|
||||
|
||||
# lookup name in our mdir
|
||||
found, rid, tag, w = mdir.namelookup(name)
|
||||
found, rid, tag, w = mdir.namelookup(did, name)
|
||||
return found, mbid, mw, mdir, rid, tag, w
|
||||
|
||||
# iterate through a directory assuming this is the mtree root
|
||||
|
||||
Reference in New Issue
Block a user