-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Combined version of LMDB mdb.master and mdb.master3 #278
Draft
Kerollmops
wants to merge
34
commits into
main
Choose a base branch
from
combined-lmdb-support
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 20, 2024 13:16
3fb591f
to
557ca5b
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
2 times, most recently
from
August 20, 2024 14:09
f16bd66
to
92104a4
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 20, 2024 14:11
92104a4
to
4fc5cf1
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 20, 2024 15:34
88cf060
to
0d3d2e7
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 21, 2024 10:10
bccccaf
to
641cd3d
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 21, 2024 10:19
641cd3d
to
9d6f1bf
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 21, 2024 10:35
96fc327
to
6cf66a4
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 21, 2024 10:36
6cf66a4
to
8efb2df
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 21, 2024 12:52
8e2336e
to
154dace
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 21, 2024 13:12
b824a69
to
975e730
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
August 21, 2024 16:51
0b257eb
to
ccd61c5
Compare
7 tasks
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
October 16, 2024 18:00
99aac37
to
b618cc5
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
November 17, 2024 17:53
b46fb4b
to
4570c9e
Compare
Kerollmops
force-pushed
the
combined-lmdb-support
branch
from
November 17, 2024 18:50
92dbe08
to
12fa866
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements #51, taking a large inspiration from the im-rs crate .
It uses a second heed3/Cargo.toml with the necessary dependencies. When you need to work or publish the heed3 crate, you need to
cp heed3/Cargo.toml heed/Cargo.toml
. The examples were moved out of the standard heed/examples/ folder to make them compile when working on both crates.There are three crates now...
heed
: the one you know. Based on LMDB 0.9 (themdb.master
branch).heed3
: Based on LMDB 1.0 (themdb.master3
branch), without support for encryption, will eventually support checksumming.heed3-encryption
: The same asheed3
but with support for encryption....but I want to only have two
heed
: the one you know. Based on LMDB 0.9 (themdb.master
branch).heed3
: Based on LMDB 1.0 (themdb.master3
branch), with support for encryption through theEncryptedDatabase
andEnvOpenOptions::open_encrypted
type and method and eventually support checksumming.Note that if we duplicate the code of the
Database
for theEncryptedDatabase
type, we no longer need the following proc-macro as we will copy/paste and change the method signature, i.e.,&RoTxn -> &mut RoTxn
, by hand.How it works?
By annotating all the heed methods that use a
&RoTxn
this way:It transforms the function signature to use a
&mut RoTxn
:This way, we ensure that users do not keep pointers for potentially invalid bytes from LMDB between two get/put operations. It's a limitation of LMDB when you use the encryption feature. The LMDB pages are decrypted on the fly in a buffer that cycles. As a result, only a restricted amount of values' pointers are valid until the next operations.
To Do
mdb.master3
branch.EnvOpenOptions/EnvEntry
?