Skip to content

1. Setup

Jump to bottom
Alex Villalba edited this page Oct 1, 2024 · 6 revisions

SSH

In order to build the unity-explorer project you will need to ensure you have SSH setup with correct access rights to the private repository.

To do this

  1. Generating a new SSH key: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#generati[…]w-ssh-key

  2. Adding a new SSH key to your account: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account#adding-a-new-s[…]to-your-account

  3. To be able to correctly import the UPM packages that need SSH make sure to follow this short guide: https://docs.unity3d.com/Manual/upm-config-ssh-git-win.html

Notes:

  • You may need to update known_hosts file in your ssh keys directory: https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/
  • If the repo cloning is done with a terminal, you may need to open it as an administrator to avoid a fetch-pack: unexpected disconnect while reading sideband packet error
  • If during the Unity project loading Unity throws an error window mid-loading with a fetch-pack: unexpected disconnect while reading sideband packet error, try using the "retry" button, it has proved to work...

GPG

GPG keys are used to sign commits and verify that it was sent from your computer with your user. To create your GPG key, follow the instructions in this website: https://docs.github.com/en/authentication/managing-commit-signature-verification/generating-a-new-gpg-key Do not forget to do everything in the GitBash console.

Troubleshooting

  1. If you get this error:

gpg: skipped "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX": No secret key
gpg: signing failed: No secret key
error: gpg failed to sign the data
fatal: failed to write commit object

Add this to the .gitconfig file (at C:/Users/<user>/)

[gpg]
program = "C:\Program Files\Git\usr\bin\gpg.exe"

  1. If you get this error when sending a commit:

GPG does not find the agent after restarting the PC:
gpg: error running '/usr/lib/gnupg/keyboxd': probably not installed
gpg: failed to start keyboxd '/usr/lib/gnupg/keyboxd': Configuration error
gpg: can't connect to the keyboxd: Configuration error
gpg: error opening key DB: No Keybox daemon running
gpg: skipped "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX": Input/output error
gpg: signing failed: Input/output error
error: gpg failed to sign the data
fatal: failed to write commit object

Open the GitBash console and execute:

gpg --list-secret-keys --keyid-format SHORT

Verification

To verify that GPG signing is working, send a commit attached to a GitHub task (putting its task #number in the description) and check how the Verified tag appears next to the hash of the commit.

imagen

LODs

By default LOD's are currently disabled in Explorer Alpha because they are too big to be packaged with the build. They can however be downloaded and integrated manually for testing.

  1. LODs can be downloaded here (latest): https://drive.google.com/file/d/1YoYpi-FhXL3IxNTOptazWquvv4VRCLbL/view?usp=drive_link

  2. To get LOD's working you just need to extract the above into the following folder: StreamingAssets/AssetBundles/lods

Build automation & CI

The Unity project is built automatically on certian triggers by a combination of GitHub workflows, python wrappers and the Unity Cloud Build system.

Workflows & triggers

There are two main workflows that handle all major builds:

  • build-unitycloud
  • build-release-weekly

The main workflow is build-unitycloud, which will be automatically triggered by commits and PRs to main, or by a manual workflow dispatch call. By default, PRs marked as draft will not trigger the build. If this is something you want, you need to add a label force-build to this PR.

The build-release-weekly workflow mostly wraps & handles build-unitycloud for release, and currently it's triggered manually with a workflow dispatch call. The release build would create a new target based on the version, which means that this does not blocks main branch and merging to main can continue

GitHub workflow

The workflow file (and any actions used by it) is mostly a wrapper of the Python handler which communicates with the Unity DevOps Build API.

It takes care of triggering builds, handling parameters, getting dynamic information from the repository (like tags and secrets) and executing the Python handler script. It also waits for the cloud build to finish, uploading all logs and artefacts from the cloud build to the workflow run.

Python handler

Located in scripts/cloudbuild, the Python handler is mostly contained inside build.py with a utils.py extension for extra functionality.

All requirements are listed in requirements.txt and are automatically installed by the GitHub Workflow.

The script expects specific environment variables to be set and accepts some arguments. These are all currently set by the workflow:

  • API_KEY: Unity Cloud API Key (secret)
  • ORG_ID: Unity Cloud Organization ID (sensitive)
  • PROJECT_ID: Unity Cloud Project ID (sensitive)
  • POLL_TIME: Time to wait in seconds before checking the API for any build status updates (while building)
  • TARGET: Template build config to use for builds
  • BRANCH_NAME: Name of the branch that triggered this build
  • CLEAN_BUILD: Triggers a clean build that forces a reimport of library folder, and not using any caching
  • CACHE_STRATEGY: Defines what strategy we want to use for caching (available options none, library, workspace, inherit). This affects the speed of next build. By default it's set to workspace
  • COMMIT_SHA: The SHA value of the commit that this build is triggered on
  • BUILD_OPTIONS: Any Unity BuildOptions to define for the build
  • PARAM_<NAME>: Any ENV variables starting with PARAM_ will be passed to Unity without the prefix to be used with Editor.CloudBuild.Parameters[]

Arguments include:

  • --resume: If set, tries to find the build executed in the same runner and track it, instead of creating a new one
  • --cancel: If set, tries to cancel a build executed in the same runner
  • --delete: If set, tries to delete a build target on Unity Cloud side, it's executed by pr-closure.yml and triggered when PR is closed.

In generic terms, the Python handler goes through the following steps in order. If any of them fail, the entire script fails. Some steps may error out without failing immediately, but mark the build as unhealthy, which eventually fails:

  1. Execute resume or cancel logic if requested
  2. Checks if the build config (target) exists on Unity Cloud
  • If new target, clone the given build config (target) for new build
  • If target exists, reuse the same build target (this optimize the caching)
  1. Set build parameters
  2. Start the build
  3. Poll the API for build status
  4. Download artifacts
  5. Download logs
  6. Cleanup

The Python handler is built to run inside a fully configured GitHub Workflow. It could also run locally, but it's not an expected usage.

Unity Cloud

The Unity Cloud Build, or Unity Cloud DevOps is the environment where builds execute. It's used directly by the Python handler and is not meant to be used manually.

This environment has some constraints (mostly on Unity's side), which include:

Concurrency

Only one build config can run at the same time. Therefore a "template" config build (named @T_<TARGET_NAME>) must be created for each target.

Each build currently clones this template and renames it, using the format TARGET_NAME-BRANCH. If build is already trigged for the same target, and new commit is applied, it will cancel previous build automatically.

Storage

All build artefacts are stored in the GitHub workflow run after the build is finished. This allows deleting old builds from the cloud to save space. The entire build must be deleted, as there's no way to delete just the artefacts.

Cache

The cache usage and level can be controlled by the template config build or by CACHE_STATEGY param. To update the cache or change settings of template, the template itself must be run manually on the Unity Cloud Build UI.

Rebuilding

If a build fails, the auto-generated config build is not removed from the cloud, which allows any re-runs from GitHub to use the exact same cache and settings as before. If you need to run a clean build you can trigger the build with CLEAN_BUILD param

Clone this wiki locally