Reworked test_runner/bench_runner to evaluate define permutations lazily
I wondered if walking in Python 2's footsteps was going to run into the same issues and sure enough, memory backed iterators became unweildy. The motivation for this change is that large ranges in tests, such as iterators over seeds or permutations, became prohibitively expensive to compile. This meant more iteration moving into tests with more steps to reproduce failures. This sort of defeats the purpuse of the test framework. The solution here is to move test permutation generation out of test.py and into the test runner itself. The allows defines to generate their values programmatically. This does conflict with the test frameworks support of sets of explicit permutations, but this is fixed by also moving these "permutation sets" down into the test runner. I guess it turns out the closer your representation matches your implementation the better everythign works. Additionally the define caching layer got a bit of tweaking. We can't precalculate the defines because of mutual recursion, but we can precalculate which define/permutation each define id maps to. This is necessary as otherwise figuring out each define's define-specific permutation would be prohibitively expensive.
This commit is contained in:
@@ -38,17 +38,18 @@ enum test_flags {
|
||||
typedef uint8_t test_flags_t;
|
||||
|
||||
typedef struct test_define {
|
||||
intmax_t (*cb)(void *data);
|
||||
intmax_t (*cb)(void *data, size_t i);
|
||||
void *data;
|
||||
size_t permutations;
|
||||
} test_define_t;
|
||||
|
||||
struct test_case {
|
||||
const char *name;
|
||||
const char *path;
|
||||
test_flags_t flags;
|
||||
size_t permutations;
|
||||
|
||||
const test_define_t *defines;
|
||||
size_t permutations;
|
||||
|
||||
bool (*filter)(void);
|
||||
void (*run)(struct lfs_config *cfg);
|
||||
|
||||
Reference in New Issue
Block a user