Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare images for arm64 compatibility #30

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

krlmlr
Copy link
Contributor

@krlmlr krlmlr commented Jun 3, 2023

I'm struggling to run the images (or R, for that matter) on my M2 in a virtual machine using Rosetta. I'm experimenting with arm64 images in https://github.com/cynkra/r-debug. These Dockerfiles have worked for me on arm64, chances are that they will continue to work on amd64.

Are you open to building multi-arch images in this repo, and perhaps to hosting them (also) on the GitHub container registry? What does CI/CD for these images currently look like?

@wch
Copy link
Owner

wch commented Jun 3, 2023

Yes, I think it's a great idea to build multi-arch images. I don't know anything about the GitHub container registry, but if the images can be hosted there instead of (or in addition to) Docker Hub, that would be good.

The CI is a daily build, which happens on a computer in my office. I did this because the builds took too long on the Docker build servers and timed out. It's possible that they could be built in GitHub Actions as a multi-step build, but I haven't tried.

The build machine is Intel, so it won't be able to build ARM images.

If you want to make these things work, I can give you access to this repo.

@krlmlr
Copy link
Contributor Author

krlmlr commented Jun 4, 2023

I have a successful build of r-devel (not r-debug) on arm64 on GHA: https://github.com/cynkra/r-debug/actions/runs/5167865060 . This build runs for almost six hours, the limit for GHA. (QEMU is slow, it would be faster if we had arm64 workers. Soon, or perhaps self-hosted.)

For now, to make this work for the entire suite of images, I think we need multi-step builds or a build matrix.

How do you feel about having multiple smaller images, one for each flavor?

@krlmlr
Copy link
Contributor Author

krlmlr commented Jun 4, 2023

Update: I managed to run the original x86_64 version of your image on my M2. Blog post following, unless this is old news and I was missing something.

Having arm64 images will help in the medium to long term (much faster, perhaps detecting slightly other types of errors), but lower priority for me now. I can help with moving the build to GHA to ease the burden on your office machine and to have CI/CD.

@krlmlr
Copy link
Contributor Author

krlmlr commented Jun 4, 2023

GHA for the original image now running at https://github.com/cynkra/r-debug/actions/runs/5170275732/jobs/9313080747 . Let's see if it finishes in under 6 hours.

Would MAKEFLAGS=-j2 help, or is it already specified in the build instructions?

@wch
Copy link
Owner

wch commented Jun 6, 2023

How did the build go?

IIRC, it already uses -j4.

@krlmlr
Copy link
Contributor Author

krlmlr commented Jun 7, 2023

The arm64 build for r-devel finished in just under 6 hours, we may want to split that image further (or wait for arm64 builders to become available). Emulation is slow.

The entire amd64 build takes 90 minutes on GHA. See https://github.com/cynkra/r-debug/tree/latest . GHCR packages are created automatically (using the GitHub token), for Docker you'd need to add secrets and tweak the workflow YAML.

Successful build: https://github.com/cynkra/r-debug/actions/runs/5173276938/jobs/9318400873

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants