-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Poetry #402
Open
orsinium
wants to merge
9
commits into
pypa:main
Choose a base branch
from
orsinium-forks:poetry
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
cbc2c4f
add PoetrySource
orsinium 2d4ac0f
use PoetrySource in CLI
orsinium 7ba514c
skip coverage for invalid dependency
orsinium d38d1f1
drop fix support for poetry
orsinium 904d224
drop poetry integration tests
orsinium a43a371
Merge branch 'main' into poetry
orsinium f026149
update README
orsinium d2eda18
format the code
orsinium 657c21f
apply review nitpicks
orsinium File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
""" | ||
Collect dependencies from `poetry.lock` files. | ||
""" | ||
from __future__ import annotations | ||
|
||
import logging | ||
from dataclasses import dataclass | ||
from pathlib import Path | ||
from typing import Iterator | ||
|
||
import toml | ||
from packaging.version import InvalidVersion, Version | ||
|
||
from pip_audit._dependency_source import DependencyFixError, DependencySource | ||
from pip_audit._fix import ResolvedFixVersion | ||
from pip_audit._service import Dependency, ResolvedDependency, SkippedDependency | ||
|
||
logger = logging.getLogger(__name__) | ||
|
||
|
||
@dataclass(frozen=True) | ||
class PoetrySource(DependencySource): | ||
""" | ||
Dependency sourcing from `poetry.lock`. | ||
""" | ||
|
||
path: Path | ||
|
||
def collect(self) -> Iterator[Dependency]: | ||
""" | ||
Collect all of the dependencies discovered by this `PoetrySource`. | ||
""" | ||
with self.path.open("r") as stream: | ||
packages = toml.load(stream) | ||
for package in packages["package"]: | ||
name = package["name"] | ||
try: | ||
version = Version(package["version"]) | ||
except InvalidVersion: | ||
skip_reason = ( | ||
"Package has invalid version and could not be audited: " | ||
f"{name} ({package['version']})" | ||
) | ||
yield SkippedDependency(name=name, skip_reason=skip_reason) | ||
else: | ||
yield ResolvedDependency(name=name, version=version) | ||
|
||
def fix(self, fix_version: ResolvedFixVersion) -> None: | ||
""" | ||
Fixes a dependency version for this `PoetrySource`. | ||
|
||
Requires poetry to be installed in the same env. | ||
|
||
Note that poetry ignores the version we want to update to, | ||
and goes straight to the latest version allowed in metadata. | ||
""" | ||
raise DependencyFixError("fix is not supported for poetry yet") # pragma: no cover |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
from __future__ import annotations | ||
|
||
from pathlib import Path | ||
|
||
from packaging.version import Version | ||
|
||
from pip_audit._dependency_source import PoetrySource | ||
from pip_audit._service import ResolvedDependency, SkippedDependency | ||
|
||
TEST_DATA_PATH = Path(__file__).parent / "data" | ||
|
||
|
||
def test_collect(tmp_path: Path) -> None: | ||
lock_content = (TEST_DATA_PATH / "poetry.lock").read_text() | ||
lock_path = tmp_path / "poetry.lock" | ||
lock_path.write_text(lock_content) | ||
sourcer = PoetrySource(path=lock_path) | ||
actual = list(sourcer.collect()) | ||
expected = [ | ||
ResolvedDependency(name="jinja2", version=Version("2.11.3")), | ||
ResolvedDependency(name="markupsafe", version=Version("2.1.1")), | ||
] | ||
assert actual == expected | ||
|
||
|
||
def test_invalid_version(tmp_path: Path) -> None: | ||
lock_content = (TEST_DATA_PATH / "poetry.lock").read_text() | ||
lock_content = lock_content.replace("2.11.3", "oh-hi-mark") | ||
lock_path = tmp_path / "poetry.lock" | ||
lock_path.write_text(lock_content) | ||
sourcer = PoetrySource(path=lock_path) | ||
deps = list(sourcer.collect()) | ||
assert [dep.name for dep in deps] == ["jinja2", "markupsafe"] | ||
assert isinstance(deps[0], SkippedDependency) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another tiny nitpick: could you turn this into an
asset
ortest_data
fixture and put it inconftest.py
?Here's the pattern we use on a handful of other projects:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could. But why? This one is short, type-safe, and not coupled on conftest.py. I love good autocomplete.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My main reason is potential reuse: we have a lot of other tests (particularly the requirements file ones) that would benefit from being broken out into stand-alone assets, so having a centralized "asset" fixture would keep things unified.
Separately (and I admit I have no idea if this is "good" practice or not): my understanding is that
pytest
conceptually encourages each test module to contain nothing except test code, withconftest.py
being the idiomatic place to put any scaffolding.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
YAGNI ;)
Never heard of that. Test files can contain fixtures, and from my experience, they should when the fixture is used only in that file. Having all fixtures in
conftest.py
brings unnecessary coupling, makes it harder to track which fixtures aren't needed anymore, and that file being a typical example of "God class" (in this case, God module) can grow to some crazy numbers (last month, I decoupled 10k lines conftest.py). And even if we assume "keep all fixtures in conftest.py", it's not a fixture. It's literally a constant.Having a constant has direct benefits: it's shorter, type-safe, closer to the code (easier to unravel the test code), and makes the test file self-contained. If none of that persuades you, please, just say it, and I'll update the PR. I want the code I contribute to be nice and readable, but the final word is always on the direct maintainers.