Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: separate out pandas and pyarrow usage/imports #10166

Open
gforsyth opened this issue Sep 18, 2024 · 2 comments
Open

refactor: separate out pandas and pyarrow usage/imports #10166

gforsyth opened this issue Sep 18, 2024 · 2 comments

Comments

@gforsyth
Copy link
Member

Today, you can install ibis-framework and generate expressions and compile to any of our backends without installing any of the backend-specific libraries.

However, if you want to actually execute that expression, and get data back, you need pandas and pyarrow installed.
We should separate out and isolate those imports, such that someone could attempt to use only one-or-the-other, or a separate method of pulling data back (like the pycapsule interface), without having those dependencies installed.

We would not be removing them as required imports, because that would make for a bad user-experience when people are getting started, but it would allow for users to explicitly install or uninstall dependencies they don't want, while maintaining other functionality.

xref #10120

@MarcoGorelli
Copy link

MarcoGorelli commented Sep 19, 2024

thanks @gforsyth for considering this!

would it be feasible to aim for something like:

  • pip install ibis-framework duckdb lets me run Ibis with DuckDB as engine, without requiring pandas/PyArrow
  • pip install ibis-framework[duckdb] brings in pandas+PyArrow too, so that t.to_pyarrow_table works by default and it's a good user experience for people getting started

?

@lostmygithubaccount
Copy link
Member

I believe that's the intention -- then in the first case you'll get import errors for things like t.to_pandas(), but most things should work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: backlog
Development

No branches or pull requests

3 participants