Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement the bootc provision plugin #3161

Merged
merged 12 commits into from
Nov 21, 2024
Merged

Conversation

ckyrouac
Copy link
Contributor

@ckyrouac ckyrouac commented Aug 21, 2024

This creates a new provision plugin that is built on top of the existing TestCloud (virtual) plugin. It adds new parameters to pass a Containerfile or container image. The plugin will then build a container image (if necessary) then build a bootc disk image from the container image using bootc image builder. Currently, bootc requires podman to be run as root when building a disk image. This is typically handled by running a podman machine as root.

An additional parameter "add-deps" toggles building a derived container image with the tmt test requirements.

Resolves #3013

Pull Request Checklist

  • implement the feature
  • write the documentation
  • extend the test coverage
  • adjust plugin docstring
  • modify the json schema
  • include a release note

This is a work in progress. I'm opening this PR early to get feedback on the high level design. I will add tests, docs, etc. after we solidify the higher level design.

If you want to try running the code here is an example fmf plan:

provision:
  how: bootc
  containerimage: quay.io/centos-bootc/centos-bootc:stream9
  disk: 20
summary: Testing bootc plugin
execute:
  how: tmt
  script: |
    echo "ok"

@ckyrouac ckyrouac marked this pull request as draft August 21, 2024 20:16
@ckyrouac
Copy link
Contributor Author

@cgwalters fyi

Copy link
Contributor

@cgwalters cgwalters left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly a skim (I don't know the tmt codebase either) but looks sane to me!

tmt/steps/provision/bootc.py Show resolved Hide resolved
tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
@happz
Copy link
Collaborator

happz commented Aug 22, 2024

  • LGTM, I like the reuse of existing plugins, that's very nice.
  • Would it also make sense to support the container plugin? IIUIC, the image produced by podman can be run both with podman run, or, converted to qcow2, as a VM. I could imagine both ways being available to use, maybe through some runtime-plugin: container|virtual switch. Is it a bad idea?
  • I will have more low-level comments related to the actual implementation, but I won't bother you now till you receive the high-level review. Just ping when you're ready for the boring stuff :)

Copy link
Collaborator

@martinhoyer martinhoyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still trying to speedrun reading/learning all things bootc (and tmt provisioning plugins), but fwiw, looks cool to me.

@happz about the container plugin support - Fedora docs says:

for fully-fledged tests it is not recommended to run a bootable container via, for instance, podman-run. One reason among others is that the filesystem is writable when being executed as an OCI container while most of the filesystem is mounted read-only on a deployed bootc system. That means the running container behaves differently than a deployed system. Yet, if you desire to run some quick tests it is recommended to run the container in detached mode.

From what I understand, podman-bootc could be pretty cool to use, once available.

@happz
Copy link
Collaborator

happz commented Aug 22, 2024

I'm still trying to speedrun reading/learning all things bootc (and tmt provisioning plugins), but fwiw, looks cool to me.

@happz about the container plugin support - Fedora docs says:

for fully-fledged tests it is not recommended to run a bootable container via, for instance, podman-run. One reason among others is that the filesystem is writable when being executed as an OCI container while most of the filesystem is mounted read-only on a deployed bootc system. That means the running container behaves differently than a deployed system. Yet, if you desire to run some quick tests it is recommended to run the container in detached mode.

From what I understand, podman-bootc could be pretty cool to use, once available.

I agree, it's not a perfect 1:1 substitution, but, exactly: for quick tests or basic test development, it may give me results faster than VM. I for one work on binutils and C/C++ toolchain in general, and my area of focus is fairly simple - compile this, run objdump on that, grep for expected section names, this kind of stuff. Learning about a typo in my reproducer quickly is very valuable, and once I'm done, I can always use full VM. I do it today already: I develop tests with container, then I switch to beaker or 1minutetip to get a more real environment before committing the new test to git. So, podman-run, for my trivial component, would be very welcome, even with caveats like the one you mentioned :)

@martinhoyer
Copy link
Collaborator

I agree, it's not a perfect 1:1 substitution, but, exactly: for quick tests or basic test development, it may give me results faster than VM. I for one work on binutils and C/C++ toolchain in general, and my area of focus is fairly simple - compile this, run objdump on that, grep for expected section names, this kind of stuff. Learning about a typo in my reproducer quickly is very valuable, and once I'm done, I can always use full VM. I do it today already: I develop tests with container, then I switch to beaker or 1minutetip to get a more real environment before committing the new test to git. So, podman-run, for my trivial component, would be very welcome, even with caveats like the one you mentioned :)

Thank you for the insight in your development process. I hope I can see it in more detail one day.

@ckyrouac
Copy link
Contributor Author

Would it also make sense to support the container plugin? IIUIC, the image produced by podman can be run both with podman run, or, converted to qcow2, as a VM. I could imagine both ways being available to use, maybe through some runtime-plugin: container|virtual switch. Is it a bad idea?

I think the existing container plugin will handle this case without any additional code. The bootc image is just another container that can be run like a typical image.

Copy link
Collaborator

@lukaszachy lukaszachy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--add-deps doesn't install require/recommend yet, right?

Either way, we have chicken&egg problem in the case 'dist-git-source' is used as the list of packages is known after provision :/ But IMO we can ignore that for now to have something working.

@ckyrouac ckyrouac force-pushed the bootc-provision branch 2 times, most recently from e3058f9 to 44ff728 Compare September 10, 2024 14:00
@ckyrouac ckyrouac marked this pull request as ready for review September 10, 2024 14:00
@ckyrouac
Copy link
Contributor Author

@happz This is ready for a review. I added some docs, tests, and code to cleanup the container images.

tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
tmt/steps/provision/bootc.py Outdated Show resolved Hide resolved
@psss
Copy link
Collaborator

psss commented Nov 20, 2024

I've added a bunch of changes in f781e34, mainly to fix & simplify tests:

  • Make sure that all test cases are actually executed.
  • Run tests directly in the data directory.
  • Simplify plans using inheritance.
  • Plus some minor adjustments.

Last remaining problem is the avc check. I guess we should just disable it if we cannot xfail it?

@psss
Copy link
Collaborator

psss commented Nov 20, 2024

/packit build

@psss
Copy link
Collaborator

psss commented Nov 21, 2024

/packit build

@psss
Copy link
Collaborator

psss commented Nov 21, 2024

Ok the xfail mystery should be solved by 57762fb.

@psss
Copy link
Collaborator

psss commented Nov 21, 2024

/packit build

@psss
Copy link
Collaborator

psss commented Nov 21, 2024

/packit build

ckyrouac and others added 12 commits November 21, 2024 16:12
This creates a new provision plugin that is built on top of the existing
TestCloud (virtual) plugin. It adds new parameters to pass a
Containerfile or container image. The plugin will then build a container
image (if necessary) then build a bootc disk image from the container
image using bootc image builder. Currently, bootc requires podman to be
run as root when building a disk image. This is typically handled by
running a podman machine as root.

An additional parameter "add-deps" toggles building a derived container
image with the tmt test requirements.

Signed-off-by: Chris Kyrouac <[email protected]>
When the podman connection is rootless=True,
this will automatically start a new rootful podman-machine
to be used for the bootc disk creation.

Signed-off-by: Chris Kyrouac <[email protected]>
Ensures the /var/tmp/tmt directory exists before creating
the temp directories. This directory might not exist in
the CI environment. It usually exists when running locally.

Signed-off-by: Chris Kyrouac <[email protected]>
Make sure that all test cases are actually executed.
Run tests directly in the `data` directory.
Simplify plans using inheritance.
Plus some minor adjustments.
Let's go the other way round: Enable the check by default, allow
individual tests to set it according to their needs by a simple
definition and disable it globally when not initiated by packit.
This is needed to actually allow disabling the flag from the
command ine as it is `True` by default.
@psss
Copy link
Collaborator

psss commented Nov 21, 2024

/packit build

@psss psss merged commit e8edeeb into teemtee:main Nov 21, 2024
21 of 22 checks passed
@psss psss self-assigned this Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci | full test Pull request is ready for the full test execution priority | must high priority, must be included in the next release step | provision Stuff related to the provision step
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a podman-bootc provisioner
8 participants