GitHub Actions Building In Depth

Making a build with GitHub Action

The cura-installer workflow will create an AppImage, dmg and nsis installer from the specified Conan package of Cura.

It consists of roughly the following steps:

1. Run conan install cura ... -g PythonRunEnv ... this will create a virtual Python environment from where it will install the cura conan package from our Artifactory and all of her dependencies.
2. Activate the newly created Python environment and run pyinstaller. Which will create a standalone directory structure, containing the used Python modules and shared libraries, it will create an executable UltiMaker-Cura from the cura_app.py and place it along the collected files.
3. Those collected files are then packaged according to the scripts in the package folder, which are installed along-side step 1 and are operating system specific. They're also signed for MacOS and Linux; Windows requires a physical key.
4. The installer is then stored as a GitHub Action build Artifact to be downloaded.

TLDR;

Follow the steps below to prepare and create a Cura release.

Setting up release branch

1. Create release branch <major>.<minor> in: Cura, Uranium, CuraEngine, fdm_materials, cura-binary-data and other relevant repo's
2. Update conandata.yml in the above-mentioned release branches:
   1. <major>.<minor>.<patch>-beta.1 (top entry). Dependencies which are worked on during the release cycle (see list in item 1), should have the following conan id: <name>/(latest)@ultimaker/stable, dependencies which are pinned should specify the exact version, instead of (latest). E.q.: cpython/3.4.10 or pyarcus/5.2.0.
   2. <major>.<minor>.<patch>-beta.2 (second entry). Copy of <major>.<minor>.<patch>-beta.1. This will specify the dependencies once the beta.1 has been released.
   3. <major>.<minor>.<patch> (third entry). The pinned dependencies of the actual release. All the <name>/(latest)@utlimaker/stable should be replaced with: <name>/<major>.<minor>.<patch> (This includes the above-mentioned repo's).
3. ... Add features and bug-fixes to the release branch ...

Creating the beta.1 release

4. Create a <major>.<minor>.<patch>-beta.1 release Conan package for each repo.
   1. Update the <major>.<minor>.<patch>-beta.1 entry in the conandata.yml for the above-mentioned release branches. Work your way upstream; Edit and commit the changes to the Cura repo last. All the specified dependencies should now be pinned to the actual used version: <name>/(latest)@utlimaker/stable should be replaced with: <name>/<major>.<minor>.<patch>-beta.1. Which is the Conan package identification created for the downstream dependencies by tagging it.
   2. Commit these changes and tag with: <major>.<minor>.<patch>-beta.1
   3. Push the commit and wait for the GitHub Action to build the Conan package for this tag.
   4. ... repeat for the project upstream up until you reach Cura.
5. Create installers for beta.1 by start the cura-installer workflow. See figure 1.
   1. The workflow from the main branch should be used in field (1).
   2. The Conan package cura/<major>.<minor>.<patch>-beta.1@_/_ should be used in field (2).
6. Download and smoketest the installers. The Windows installer should be signed with the physical key.
7. Create a GitHub release page from the Cura <major>.<minor>.<patch>-beta.1 tag.
   1. Check the prerelease box
   2. Add changelog in the comment field
   3. Upload all the signed artifacts
   4. Publish the release

Getting ready for the actual release

8. ... Add bug-fixes to the release branch ...

Creating the release

9. Create a release for each repo, starting from the leaves in the dependency tree.
   1. Check the previous created <major>.<minor>.<patch> entry in the conandata.yml for the above-mentioned release branches. Make sure the used dependencies are pinned to the actual used version: <name>/(latest)@utlimaker/stable should be replaced with: <name>/<major>.<minor>.<patch>. Which is the Conan package identification created for the downstream dependencies by tagging it.
   2. Commit these changes and tag with: <major>.<minor>.<patch>.
   3. Push the commit and wait for the GitHub Action to build the Conan package for this tag.
   4. ... repeat for the project upstream up until you reach Cura.
10. Create the installers by starting the cura-installer workflow. See figure 1.
11. The workflow from the main branch should be used in field (1).
12. The Conan package cura/<major>.<minor>.<patch>@_/_ should be used in field (2).
13. Download and smoketest the installers. The Windows installer should be signed with the physical key.
14. Create a GitHub release page from the Cura <major>.<minor>.<patch> tag.
    1. Add changelog in the comment field
    2. Upload all the signed artifacts
    3. Publish the release

Merging release branch <major>.<minor> to main

13. Ensure that <major>.<minor_+_1>-alpha is the top-entry in the conandata.yml. Either by removing the <major>.<minor>* entries or by moving them on entry down. This is needed because a local conan install . command will look at the first entry in the conandata.yml, while a conan install . <name>/<version>@<user>/<channel> will look for that specific entry (following semver) and used that

GitHub Action UI

Figure: 1

Package versioning scheme

We use semver to determine the version of our Conan packages. On a push event to one of our repositories, such as Cura, the conan-recipe-version workflow will look at that repo and create a conan package id based on the branch, the latest commit and the event that triggered the workflow, eq: commit, pull request, tag etc.

The following rules apply:

• It looks at the latest tag with the semver formatting in the git history and count the number of commits since that tag, adding this as build metadata +<no_of_commits_since_last_tag> This goes for al non-tagged pushed commits.
• Semver formatted tagged commits will result an exact version as tagged.
• Every pushed commit will also create a latest alias point towards the last made conan package.
• On the main branch:
  • It bumps up the minor version
  • It will have a -alpha pre-release identifier
• On a feature/bug-fix CURA-1234_bla_bla branches:
  • It bumps up the minor version
  • It will have a -alpha pre-release identifier
• On a release <major>.<minor> branches
  • It will use the <major>.<minor>.<patch>-beta.1 if we didn't release the first beta candidate yet
  • It wil bump up the prerelease_id once a beta tag is created, like so <major>.<minor>.<patch>-beta.2
  • Once a release tag has been created <major>.<minor>.<patch>
    • It will bump-up the <patch> number and use the -beta.1 prerelease tag.

Package user/channel naming scheme

Conan user and channel differ per branch, this ensures that the (latest) alias is branch specific.

The following rules apply:

• Pushes on branch main are considered testing e.q.: <name>/<major>.<minor>.<patch>-alpha+<no_commits_since_last_tag>@utlimaker/testing or <name>/latest@utlimaker/testing for the latest main package
• Pushes on release branch <major>.<minor> are considered stable e.q.: <name>/<major>.<minor>.<patch>-beta.<prerelease_no>+<no_commits_since_last_tag>@utlimaker/stable or <name>/latest@utlimaker/stable for the latest release package
• Pushes on a FR/bug-fix branch CURA-1234 will use the cura_1234 channel e.q.: <name>/<major>.<minor>.<patch>-alpha+<no_commits_since_last_tag>_cura_1234@utlimaker/cura_1234 or <name>/latest@utlimaker/cura_1234 for the latest package
• Pull requests will use the pr_<github_pr_number>

Example version, channel and naming scheme

The square commits are pushed commits which trigger creation of a Conan package

gitGraph
  commit id: "5.2.0" tag: "5.2.0" type: HIGHLIGHT
  commit
  commit id: "5.2.0-alpha+2@testing" type: HIGHLIGHT
  branch CURA-1234_feature_a
  checkout main
  commit
  commit id: "5.3.0-alpha+4@testing" type: HIGHLIGHT
  checkout CURA-1234_feature_a
  commit
  commit id: "5.3.0-alpha+4@cura_1234" type: HIGHLIGHT
  checkout main
  commit
  branch release/5.3
  commit id: "5.3.0-beta.1+6@stable" type: HIGHLIGHT
  commit
  commit id: "5.3.0-beta.1" tag: "5.3.0-beta.1" type: HIGHLIGHT
  checkout main
  commit id: "5.3.0-alpha+6@testing" type: HIGHLIGHT
  commit
  checkout release/5.3
  commit
  commit id: "5.3.0-beta.2+2@stable" type: HIGHLIGHT
  commit id: "5.3.0" tag: "5.3.0" type: HIGHLIGHT
  checkout main
  merge release/5.3
  commit id: "5.4.0-alpha+1@testing" type: HIGHLIGHT
  checkout release/5.3
  commit id: "5.3.1-beta.1+1@stable" type: HIGHLIGHT

Loading

Sssst: secrets

The workflow relies on a couple of GitHub (repo) secrets (the below given examples are bogus ofcourse):

• CONAN_USER: GitHub service user account on ultimaker.jfrog.io e.q.: [email protected]
• CONAN_PASS: Github service user account token for ultimaker.jfrog.io e.q.: KJGHIHJKLUHGUIYGUIOKLJ
• CODESIGN_IDENTITY: Apple developer codesign identity
• MAC_NOTARIZE_USER: Apple developer notarizing user
• MAC_NOTARIZE_PASS: Apple developer notarizing user token
• MACOS_CERT_P12: Apple developer P12 certificate decoded
• MACOS_CERT_PASS: Apple developer P12 cert pass
• MACOS_CERT_USER: Apple developer certificate user
• MACOS_CERT_PASSPHRASE: Passphrase to decode the Apple developer P12 certifacte
• GPG_PRIVATE_KEY: Linux GPG private key