scripts: Fixed O(n^2) slicing in Rbyd.fetch
Do you see the O(n^2) behavior in this loop?
j = 0
while j < len(data):
word, d = fromleb(data[j:])
j += d
The slice, data[j:], creates a O(n) copy every iteration of the loop.
A bit tricky. Or at least I found it tricky to notice. Maybe because
array indexing being cheap is baked into my brain...
Long story short, this repeated slicing resulted in O(n^2) behavior in
Rbyd.fetch and probably some other functions. Even though we don't care
_too_ much about performance in these scripts, having Rbyd.fetch run in
O(n^2) isn't great.
Tweaking all from* functions to take an optional index solves this, at
least on paper.
---
In practice I didn't actually find any measurable performance gain. I
guess array slicing in Python is optimized enough that the constant
factor takes over?
(Maybe it's being helped by us limiting Rbyd.fetch to block_size in most
scripts? I haven't tested NAND block sizes yet...)
Still, it's good to at least know this isn't a bottleneck.
This commit is contained in:
@@ -21,20 +21,23 @@ def openio(path, mode='r', buffering=-1):
|
||||
else:
|
||||
return open(path, mode, buffering)
|
||||
|
||||
def fromleb128(data):
|
||||
def fromleb128(data, j=0):
|
||||
word = 0
|
||||
for i, b in enumerate(data):
|
||||
word |= ((b & 0x7f) << 7*i)
|
||||
d = 0
|
||||
while j+d < len(data):
|
||||
b = data[j+d]
|
||||
word |= (b & 0x7f) << 7*d
|
||||
word &= 0xffffffff
|
||||
if not b & 0x80:
|
||||
return word, i+1
|
||||
return word, d+1
|
||||
d += 1
|
||||
return word, len(data)
|
||||
|
||||
def dbg_leb128s(data):
|
||||
lines = []
|
||||
j = 0
|
||||
while j < len(data):
|
||||
word, d = fromleb128(data[j:])
|
||||
word, d = fromleb128(data, j)
|
||||
lines.append((
|
||||
' '.join('%02x' % b for b in data[j:j+d]),
|
||||
word))
|
||||
|
||||
Reference in New Issue
Block a user