-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Weird memory usage of random-find benchmark #23
Comments
Can't seem to reproduce it when just running
EDIT: Also doesn't occur when running the full set of benchmarks, I still get ~2400 consistently. |
Note that you need to poke every page (i.e. memset all the bytes) otherwise it won't count towards the RSS. |
Changed the added code to: // allocate 100MiB to attempt to skew the results to check if the backfill process is included
var dummy = try arena.alloc(u8, 100 * 1024 * 1024);
std.mem.set(u8, dummy, 1);
std.debug.print("last byte is {}\n", .{dummy[dummy.len - 1]}); and verified that Still got the same result: 2404 |
Ok, finally got something. I downloaded the
for So, it's the EDIT: Double checked this by adding the |
This seems to be the relevant note from
|
I've been trying to measure the Although what I've been doing is unrelated here, this might be a good read for you: https://jkz.wtf/random-linux-oddity-1-ru_maxrss since it mentions parts of the linux source where the metrics are maintained. |
The
maxrss
results of therandom-find
has some anomalous spikes for unknown reasons:From IRC:
I've tried testing the theory that the memory usage of the backfill script itself is being included and I ultimately don't think that's what's happening.
To test this, I added:
to
backfill.zig
and then ran the benchmarks once withcollect-measurements.zig
and then once withbackfill.zig
for the same commit (e498fb155051f548071da1a13098b8793f527275
). Here's themaxrss
ofrandom-find
for each:Interestingly, values ~2400 were only gotten by any
gotta-go-fast
benchmarking during the original runs, and it seems like somehow all results since then (most done by the backfill script AFAIK) seem to be getting larger and larger (for no discernible reason, as the benchmark and the hash map implementation haven't changed much in that time). Also, if you look at the graph, there are actually quite a few of these mini-spikes which are also presumably in line with where the backfill script was run previously.So, no clue what's going on here still, but hopefully I've narrowed it down slightly. Will probably try running the backfill script locally with just
random-find
next, and seeing if I can reproduce the increasingmaxrss
results.The text was updated successfully, but these errors were encountered: