New issue when creating check runs.

[Zimmi48]

The creation of some check runs have failed in the last hours / days (?). An example failed query (that I could reproduce) is:

mutation newCheckRun($name: String!, $repoId: ID!, $headSha: String!, $status: RequestableCheckStatusState!, $title: String!, $text: String, $summary: String!, $url: String!, $conclusion: CheckConclusionState, $externalId: String) {
  createCheckRun(input: {status: $status, name: $name, repositoryId: $repoId, headSha: $headSha, conclusion: $conclusion, detailsUrl: $url, output: {title: $title, text: $text, summary: $summary}, externalId: $externalId}) {
    clientMutationId
  }
}

with variables:

{
  "name":"GitLab CI job doc:stdlib:dune (pull request)",
  "repoId":"MDEwOlJlcG9zaXRvcnkxMzc3MTU5",
  "headSha":"8fac78830df30a62e4ba46c567680f0ca5050ca2",
  "status":"COMPLETED",
  "title":"Test has failed on GitLab CI",
  "text":"```\ngroup root\n\n<><> Package variables ('opam config list PKG' to show) <><><><><><><><><><><><>\nPKG:name # Name of the package\nPKG:version # Version of the package\nPKG:depends # Resolved direct dependencies of the package\nPKG:installed # Whether the package is installed\nPKG:enable # Takes the value \"enable\" or \"disable\" depending on whether the package is installed\nPKG:pinned # Whether the package is pinned\nPKG:bin # Binary directory for this package\nPKG:sbin # System binary directory for this package\nPKG:lib # Library directory for this package\nPKG:man # Man directory for this package\nPKG:doc # Doc directory for this package\nPKG:share # Share directory for this package\nPKG:etc # Etc directory for this package\nPKG:build # Directory where the package was built\nPKG:hash # Hash of the package archive\nPKG:dev # True if this is a development package\nPKG:build-id # A hash identifying the precise package version with all its dependencies\n$ dev/tools/check-cachekey.sh\n$ dev/ci/gitlab-section.sh end before_script\n\n$ tar xfj _build.tar.bz2\n$ make -f Makefile.dune \"$DUNE_TARGET\"\ndune build @stdlib-html\n bash doc/stdlib/index-list.html (exit 1)\n(cd _build/default && /bin/bash -e -u -o pipefail -c 'doc/stdlib/make-library-index doc/stdlib/index-list.html doc/stdlib/hidden-files')\nBuilding file index-list.prehtml... Error: extra files:\ntheories/Numbers/Cyclic/Int63/Int63.v\nMakefile.dune:126: recipe for target 'stdlib-html' failed\nmake: *** [stdlib-html] Error 1\nUploading artifacts for failed job\nUploading artifacts...\n_build/log: found 1 matching files and directories \nWARNING: _build/default/doc/stdlib/html: no matching files \nUploading artifacts as \"archive\" to coordinator... 201 Created id=2144390236 responseStatus=201 Created token=oqipeVPy\nCleaning up project directory and file based variables\nERROR: Job failed: exit code 1\n\n```","summary":"This job has failed. If you need to, you can restart it directly in the GitHub interface using the \"Re-run\" button.\n\nThis job ran on the Docker image `registry.gitlab.com/coq/coq:bionic_coq-V2022-02-10-48d860fb4f`, depended on the build job `build:edge+flambda:dune:dev` with OCaml `4.13.0+flambda`.\n\nWe show below the last 40 lines of the trace from GitLab (the complete trace is available [here](https://gitlab.com/coq/coq/-/jobs/2144390236)).",
  "url":"https://gitlab.com/coq/coq/-/jobs/2144390236",
  "conclusion":"FAILURE",
  "externalId":"projects/6138686/jobs/2144390236"
}

leading to the following error:

{
  "errors": [
    {
      "path": [
        "mutation newCheckRun",
        "createCheckRun",
        "input",
        "headSha"
      ],
      "extensions": {
        "code": "variableMismatch",
        "variableName": "headSha",
        "typeName": "String!",
        "argumentName": "headSha",
        "errorMessage": "Type mismatch"
      },
      "locations": [
        {
          "line": 7,
          "column": 79
        }
      ],
      "message": "Type mismatch on variable $headSha and argument headSha (String! / GitObjectID!)"
    },
    {
      "path": [
        "mutation newCheckRun",
        "createCheckRun",
        "input",
        "detailsUrl"
      ],
      "extensions": {
        "code": "variableMismatch",
        "variableName": "url",
        "typeName": "String!",
        "argumentName": "detailsUrl",
        "errorMessage": "Type mismatch"
      },
      "locations": [
        {
          "line": 7,
          "column": 123
        }
      ],
      "message": "Type mismatch on variable $url and argument detailsUrl (String! / URI)"
    }
  ]
}

[Zimmi48]

The error message is pretty clear: we need to fix the request in the following way:

-  mutation newCheckRun($name: String!, $repoId: ID!, $headSha: String!,
+  mutation newCheckRun($name: String!, $repoId: ID!, $headSha: GitObjectID!,
   $status: RequestableCheckStatusState!, $title: String!, $text: String, $summary: String!,
-  $url: String!, $conclusion: CheckConclusionState, $externalId: String) {
+  $url: URI!, $conclusion: CheckConclusionState, $externalId: String) {

Unfortunately, this means using a custom scalar, and it seems like this has not been implemented yet in graphql-ppx (teamwalnut/graphql-ppx#272).

At the moment, the only workaround I can imagine is to intercept and patch the request before sending it (around this point):

bot/bot-components/GitHub_mutations.ml

Line 134 in 15a4b50

makeVariables ~name ~repoId:repo_id ~headSha:head_sha ~status ~title ?text

[Zimmi48]

The tested hackish workaround (d9c1d87) seems to work.

[Alizter]

I still don't understand the cause, has the GitHub API changed?

[Zimmi48]

From what I can tell, GitHub's API used to tolerate that we use the String type when it was expecting custom scalars instead. I couldn't find anything in the changelog that would indicate that GitHub's API has become stricter with respect to this, but maybe they are considering this a bug fix not worth documentation?

[Zimmi48]

The explanation from GitHub is that they have upgraded their graphql-ruby dependency from 1.11.10 to 1.12.24, and the new version started correctly enforcing the types.