-
Notifications
You must be signed in to change notification settings - Fork 151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Implement bloom_filter_agg #987
base: main
Are you sure you want to change the base?
Conversation
…w. Added spark_bit_array_tests.
fn state(&mut self) -> Result<Vec<ScalarValue>> { | ||
// TODO(Matt): There might be a more efficient way to do this by transmuting since calling | ||
// state() on an Accumulator is considered destructive. | ||
let state_sv = ScalarValue::Binary(Some(self.state_as_bytes())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One way to avoid the copy, which may be too ugly , would be to store bloom filter data as an Option<>
So instead of
pub struct SparkBloomFilter {
bits: SparkBitArray,
num_hash_functions: u32,
}
Something like
pub struct SparkBloomFilter {
bits: Option<SparkBitArray>
num_hash_functions: u32,
}
And then you could basically use Option::take
to take the value and leave a None
in its place
let Some(bits) = self.bits.take() else {
return Err(invalid state)
};
// do whatever you want now you have the owned `bits`
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #987 +/- ##
============================================
+ Coverage 34.03% 34.41% +0.38%
- Complexity 875 889 +14
============================================
Files 112 112
Lines 43289 43428 +139
Branches 9572 9627 +55
============================================
+ Hits 14734 14947 +213
+ Misses 25521 25437 -84
- Partials 3034 3044 +10
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Which issue does this PR close?
Closes #846.
Rationale for this change
What changes are included in this PR?
How are these changes tested?