-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cache "concrete" dists by Distribution instead of InstallRequirement #12863
base: main
Are you sure you want to change the base?
Conversation
5e48208
to
3c2aa93
Compare
cc @edmorley @pfmoore @sbidoul @pradyunsg: this should now be ready for review, as a pure refactoring change. |
Actually, sorry for the noise: I've convinced myself to split this out into two separate PRs. Will ping again with a smaller one to review first. |
3c2aa93
to
315ec74
Compare
Ok, this PR is ready for review, and #12871 has been created on top of it (in draft form). Please review this PR first, as it introduces the most important distinction of "concrete" vs metadata-only |
5a329fe
to
4e7d29f
Compare
Doc failures related to #12886 ? |
4e7d29f
to
ad93d54
Compare
Thanks so much for the ping! I just rebased but I'm not sure if that will fix it. |
docs tests passed! |
Thanks for the ping, @cosmicexplorer - I get a lot of notifications for your work, mostly commits that I'd rather ignore (because I can't keep up with them!) so a specific request is appreciated. I'll try to take a look at this over the next few days, but can I make a couple of requests?
|
In particular I would be appreciative of an order in which to read the PRs. If I get a chance to help review I am currently wasn't sure in which order to approach them. |
Thanks so much for the patient feedback, I had been thinking it would make sense to make a checklist of open work and not sure why I didn't do that earlier. It's also very easy to wait a little longer to push when changes are ready as well as avoid force pushes entirely. |
@pfmoore @notatallshaw: created #12921 just now to track this work. You'll see that this PR is first of 3 under "To avoid downloading dists for metadata-only commands". |
Split out of #12186.
Problem
The
RequirementPreparer
currently has a lot of mutable state it updates at various points of the resolve/prepare process. This difficulty has been exacerbated as we have added metadata-only resolve functionality via #12208 and #11111. We would like to solve #12603, but we don't have a uniform internal API for checking whether a distribution:--use-feature=fast-deps
),We currently only have a single boolean flag
InstallRequirement#needs_more_preparation
to determine whether we need to do any more work to get to phase (3).Solution
.is_concrete
property to ourDistribution
wrappers to describe whether the dist needs to be built into a wheel (or is already a built wheel).InstallRequirement#needs_more_preparation
, and introduce the two methods.cache_concrete_dist()
and.cache_virtual_metadata_only_dist()
to manage aDistribution
object attached to theInstallRequirement
object itself (this is the iteration of.dist_from_metadata
from use .metadata distribution info when possible #11512)..cache_concrete_dist()
from within theAbstractDistribution#get_metadata_distribution()
implementations to centralize the single place where we instantiate a dist.Result
There should be no changes to
pip
's external behavior, and a lot more documentation has been added to the methods which traverse the initialization phases of aDistribution
.In particular, we now have the following distinction:
AbstractDistribution
subclasses (still) always refer to installable distributions. This means they are not metadata-only.BaseDistribution
subclasses may be metadata-only. If so, they returnFalse
from.is_concrete
.