Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse connection to Extract,Consume and execute substrait query plans #113

Merged
merged 17 commits into from
Nov 6, 2024

Conversation

pdet
Copy link
Contributor

@pdet pdet commented Sep 24, 2024

Up until now, we have been creating new connections to interact with substrait. This was necessary to circumvent the client context lock.

However, this was a brittle solution and especially problematic when dealing with temporary objects since these are not passed around new connections.

This PR allows us to reuse the same connection by utilizing a new flexible relation binder that allows us to consume and execute substrait plans from within the same connection; it also changes the from_substrait functions to use bind replace.

Aditionally, extracting plans will also go through a specialized code path, to allow us to use the same connection.

Since substrait also uses special options (e.g., some optimizations are disabled), these are set and reset on the original connection.

@pdet pdet merged commit bc9f4b3 into substrait-io:main Nov 6, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants