Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(databricks): isolate volume creation #10432

Merged
merged 1 commit into from
Nov 5, 2024

Conversation

cpcloud
Copy link
Member

@cpcloud cpcloud commented Nov 4, 2024

This PR moves the test volume in Databricks to be tied to the user and python version with which the volume is being created. Additionally, this PR does something similar for the memtable volume to avoid a similar problem with concurrent modification of memtable files.

@cpcloud cpcloud added this to the 10.0 milestone Nov 4, 2024
@cpcloud cpcloud added bug Incorrect behavior inside of ibis databricks The Databricks backend labels Nov 4, 2024
@github-actions github-actions bot added the tests Issues or PRs related to tests label Nov 4, 2024
@cpcloud
Copy link
Member Author

cpcloud commented Nov 4, 2024

Databricks is passing:

…/ibis on  databricks-test-flakiness is 📦 v9.5.0 via 🐍 v3.12.7 via ❄️  impure (ibis-3.12-env) took 2s
❯ pytest -m databricks -n auto --dist loadgroup --snapshot-update -q -x
bringing up nodes...
..................sssssssssssssssssssss.........ssssssssssssssssssssss.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss................x................... [  9%]
............x.xx.x....x.....x.................................x..............xx......................x...x........x................................xx.....xxxxx.x.xxxxxxxxxx.x...x.xx.x. [ 19%]
xx..........x..x...x...x...........x.xxx......xx...xx.....x.x.xx.....x.........x..........x..x.x.....x....................x.....x........x..x..x..x........xx.............x............. [ 28%]
....xxx...........................x.............x................x..........xs..........x............x..........................................s...................................s... [ 38%]
.x.x......xx........s.s.x...x.......s...x..................x...x.xxx.xxxxxxs..x.....x.x...x..........x......s....x...................x............................x.x................... [ 47%]
........x......x............................x.........x................xx......x....x.....x................x...x..x.............................................x....................... [ 57%]
................................................xx.xx.x.x..x.x....x.x..x...x..xxxx.xxxxxxx.....x.....x......................................x........x.....x...............x.........x.. [ 66%]
.............x.x..........xx.................x............x...........x....................x....x.......x...............................x............................................... [ 76%]
..............x............................x...........xx..........x....x...x.x.xx......x....x..x...x.x..............x...........xx...x..x.xx....x.........x.....X.x....X......x......xx [ 85%]
..x..x..........x..x...xx.x..x.........xx.x...xx.x..xx.........x.x.x..x...x............s..........................sx.................................................................... [ 95%]
.............................................................x....x....................                                                                                                  [100%]
1584 passed, 130 skipped, 211 xfailed, 2 xpassed in 248.09s (0:04:08)

@cpcloud cpcloud added the ci-run-cloud Add this label to trigger a run of BigQuery, Snowflake, and Databricks backends in CI label Nov 4, 2024
@ibis-docs-bot ibis-docs-bot bot removed the ci-run-cloud Add this label to trigger a run of BigQuery, Snowflake, and Databricks backends in CI label Nov 4, 2024
@cpcloud cpcloud merged commit 3cdccfb into ibis-project:main Nov 5, 2024
86 checks passed
@cpcloud cpcloud deleted the databricks-test-flakiness branch November 5, 2024 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior inside of ibis databricks The Databricks backend tests Issues or PRs related to tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant