Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Another Stable B-Tree. #90

Draft
wants to merge 22 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions collections/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ We use the same random number generator with fixed seed to ensure that all colle
the same elements, and the queries are exactly the same. Below we explain the measurements of each column in the table:

* generate 1m. Insert 1m Nat64 integers into the collection. For Motoko collections, it usually triggers the GC; the rest of the column are not likely to trigger GC.
* max mem. For Motoko, it reports `rts_max_heap_size` after `generate` call; For Rust, it reports the Wasm's memory page * 32Kb.
* max mem. For Motoko, it reports `rts_max_heap_size` after `generate` call; For Rust, it reports the Wasm's memory page * 64Kb; For stable benchmarks, it reports the region size of the stable memory storing the map.
* batch_get 50. Find 50 elements from the collection.
* batch_put 50. Insert 50 elements to the collection.
* batch_remove 50. Remove 50 elements from the collection.
* upgrade. Upgrade the canister with the same Wasm module. For non-stable benchmarks, the map state is persisted by serializing and deserializing states into stable memory. For stable benchmarks, the upgrade takes no cycles, as the state is already in the stable memory.
* upgrade. Upgrade the canister with the same Wasm module. For non-stable benchmarks, the map state is persisted by serializing and deserializing states into stable memory. For stable benchmarks, the upgrade only needs to initialize the metadata, as the state is already in the stable memory.

## **💎 Takeaways**

Expand All @@ -33,6 +33,7 @@ the same elements, and the queries are exactly the same. Below we explain the me
> * The stable benchmarks are much more expensive than their non-stable counterpart, because the stable memory API is much more expensive. The benefit is that they get fast upgrade. The upgrade still needs to parse the metadata when initializing the upgraded Wasm module.
> * `hashmap` uses amortized data structure. When the initial capacity is reached, it has to copy the whole array, thus the cost of `batch_put 50` is much higher than other data structures.
> * `btree` comes from [mops.one/stableheapbtreemap](https://mops.one/stableheapbtreemap).
> * `btree_stable` comes from [github.com/sardariuss](https://github.com/sardariuss/MotokoStableBTree).
> * `zhenya_hashmap` comes from [mops.one/map](https://mops.one/map).
> * `vector` comes from [mops.one/vector](https://mops.one/vector). Compare with `buffer`, `put` has better worst case time and space complexity ($O(\sqrt{n})$ vs $O(n)$); `get` has a slightly larger constant overhead.
> * `hashmap_rs` uses the `fxhash` crate, which is the same as `std::collections::HashMap`, but with a deterministic hasher. This ensures reproducible result.
Expand Down
4 changes: 4 additions & 0 deletions collections/motoko/dfx.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@
"vector": {
"type": "motoko",
"main": "src/vector.mo"
},
"btreemap_stable": {
"type": "motoko",
"main": "src/btreemap_stable.mo"
}
},
"defaults": {
Expand Down
1 change: 1 addition & 0 deletions collections/motoko/mops.template.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ stableheapbtreemap = "1.3.0"
map = "9.0.0"
splay = "0.1.0"
vector = "0.2.0"
StableBTree = "https://github.com/matthewhammer/MotokoStableBTree"
55 changes: 55 additions & 0 deletions collections/motoko/src/btreemap_stable.mo
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import Nat64 "mo:base/Nat64";
import Iter "mo:base/Iter";
import Option "mo:base/Option";
import Region "mo:base/Region";

import Random "random";
import Profiling "../../../utils/motoko/Profiling";

import BytesConverter "mo:StableBTree/bytesConverter";
import Map "mo:StableBTree/btreemap";
import Memory "mo:StableBTree/memory";

actor {
stable let profiling = Profiling.init();
stable var region = Region.new();

var map = Map.init<Nat64, Nat64>(
Memory.RegionMemory(region),
BytesConverter.NAT64_CONVERTER,
BytesConverter.NAT64_CONVERTER,
);

let rand = Random.new(null, 42);

public func generate(size : Nat32) : async () {
let rand = Random.new(?size, 1);
let iter = Iter.map<Nat64, (Nat64, Nat64)>(rand, func x = (x, x));
for ((k, v) in iter) {
ignore map.insert(k, v);
};
};
public query func get_mem() : async (Nat, Nat, Nat) {
let size = Nat64.toNat(Region.size(region)) * 65536;
(size, size, size)
};
public func batch_get(n : Nat) : async () {
for (_ in Iter.range(1, n)) {
let x = Option.get<Nat64>(rand.next(), 0);
ignore map.get(x);
};
};
public func batch_put(n : Nat) : async () {
for (_ in Iter.range(1, n)) {
let k = Option.get<Nat64>(rand.next(), 0);
ignore map.insert(k, k);
};
};
public func batch_remove(n : Nat) : async () {
let rand = Random.new(null, 1);
for (_ in Iter.range(1, n)) {
let x = Option.get<Nat64>(rand.next(), 0);
ignore map.remove(x);
};
};
};
3 changes: 3 additions & 0 deletions collections/perf.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ let triemap = wasm_profiling("motoko/.dfx/local/canisters/triemap/triemap.wasm",
let rbtree = wasm_profiling("motoko/.dfx/local/canisters/rbtree/rbtree.wasm", record { start_page = 16 });
let splay = wasm_profiling("motoko/.dfx/local/canisters/splay/splay.wasm", record { start_page = 16 });
let btree = wasm_profiling("motoko/.dfx/local/canisters/btreemap/btreemap.wasm", record { start_page = 16 });
let btree_stable = wasm_profiling("motoko/.dfx/local/canisters/btreemap_stable/btreemap_stable.wasm", record { start_page = 16 });
let zhenya = wasm_profiling("motoko/.dfx/local/canisters/zhenya_hashmap/zhenya_hashmap.wasm", record { start_page = 16 });
let heap = wasm_profiling("motoko/.dfx/local/canisters/heap/heap.wasm", record { start_page = 16 });
let buffer = wasm_profiling("motoko/.dfx/local/canisters/buffer/buffer.wasm", record { start_page = 16 });
Expand Down Expand Up @@ -87,6 +88,8 @@ perf(vector_rs, "vec_rs", init_size, batch_size);
let init_size = 50_000;
let batch_size = 50;
output(file, "\n## Stable structures\n\n| |binary_size|generate 50k|max mem|batch_get 50|batch_put 50|batch_remove 50|upgrade|\n|--:|--:|--:|--:|--:|--:|--:|--:|\n");
perf(btree, "btree", init_size, batch_size);
perf(btree_stable, "btree_stable", init_size, batch_size);
perf(btreemap_rs, "btreemap_rs", init_size, batch_size);
perf(btreemap_stable_rs, "btreemap_stable_rs", init_size, batch_size);
perf(heap_rs, "heap_rs", init_size, batch_size);
Expand Down
Loading