-
Notifications
You must be signed in to change notification settings - Fork 353
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Gh-3295: Fix issue with Accumulo iterators when smart merging Federated POC #3296
Gh-3295: Fix issue with Accumulo iterators when smart merging Federated POC #3296
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #3296 +/- ##
==========================================
Coverage 67.77% 67.77%
Complexity 2595 2595
==========================================
Files 949 949
Lines 30237 30242 +5
Branches 3339 3341 +2
==========================================
+ Hits 20492 20497 +5
+ Misses 8279 8278 -1
- Partials 1466 1467 +1 ☔ View full report in Codecov by Sentry. |
Quality Gate passedIssues Measures |
There was an issue where the iterators from Accumulo could be exhausted or closed before the merging of elements had completed. There's not a great way around this other than to buffer the data into a set before using for merging. This does of course increase memory usage but might be faster overall.
(Also the previous solution was still technically buffering to memory with a
distict()
call).Related issue