Feature request: support scoop json format as proto plugin definition format

[W1M0R]

While looking into the scoop json format, it felt very similar to the proto toml plugin format. Then I thought it could be very helpful to try and write a converter that can take a scoop json file as input and output a proto toml file. In that way, the proto package ecosystem (on Windows) can get a quick boost.

But perhaps an even quicker win, is to support the scoop json format natively in proto. Since the scoop packages are also distributed on GitHub, the usage will also be very similar.

Consider the following Scoop manifest for just:

{
    "version": "1.31.0",
    "description": "A command runner written in rust",
    "homepage": "https://just.systems",
    "license": "CC0-1.0",
    "suggest": {
        "fzf": "fzf"
    },
    "architecture": {
        "64bit": {
            "url": "https://github.com/casey/just/releases/download/1.31.0/just-1.31.0-x86_64-pc-windows-msvc.zip",
            "hash": "9e926f50bcfe730d48d798dca06a6e1529834bfafde3fb467ea024588e5e6fae"
        }
    },
    "bin": "just.exe",
    "checkver": {
        "github": "https://github.com/casey/just"
    },
    "autoupdate": {
        "architecture": {
            "64bit": {
                "url": "https://github.com/casey/just/releases/download/$version/just-$version-x86_64-pc-windows-msvc.zip"
            }
        }
    }
}

And a proto TOML plugin for just:

name = "just"
type = "cli"

[resolve]
git-url = "https://github.com/casey/just"

[platform.linux]
download-file = "just-{version}-{arch}-unknown-linux-musl.tar.gz"

[platform.macos]
download-file = "just-{version}-{arch}-apple-darwin.tar.gz"

[platform.windows]
download-file = "just-{version}-{arch}-pc-windows-msvc.zip"

[install]
download-url = "https://github.com/casey/just/releases/download/{version}/{download_file}"

From these two definitions, it feels like there is enough information in the scoop manifest to derive the proto toml.

A .prototools file using the Scoop manifest could look like this:

just = "1.30.0"

[plugins]
# Use just from proto def:
#just = "https://raw.githubusercontent.com/Phault/proto-toml-plugins/main/just/plugin.toml"
# Use just from scoop manifest:
just = "https://raw.githubusercontent.com/ScoopInstaller/Main/master/bucket/just.json"

[plugins.just]
type = "scoop"

A first attempt at this would only install Windows packages, but it may be possible to infer enough information (or use a github repo lib to gather more information about release assets) to also calculate some information for the other platforms.

Here is an example of a more complex Scoop manifest: https://github.com/ScoopInstaller/Versions/blob/master/bucket/dotnet7-sdk.json

[W1M0R]

Thinking about it a bit more, something like this should probably not live in core proto. A helper utility that does the conversion to toml and just commits the toml files to a repo is probably a better fit. No need to complicate proto, it's already doing a good job at the problem it is solving.

[W1M0R]

The proto TOML files are so easy to write that there might not be that much value in creating a generator, but if someone wants to do it, it might also be helpful to "import" packages from the Aqua Registry in a similar way. Like proto, and unlike scoop, aqua also has true cross-platform support.

Aqua has a pkg.yaml that looks like this for just:

packages:
  - name: casey/[email protected]
  - name: casey/just
    version: 1.25.1
  - name: casey/just
    version: 1.25.0
  - name: casey/just
    version: 1.1.0
  - name: casey/just
    version: 0.10.6
  - name: casey/just
    version: 0.10.5
  - name: casey/just
    version: 0.10.4
  - name: casey/just
    version: v0.9.3
  - name: casey/just
    version: v0.4.4
  - name: casey/just
    version: v0.3.3
  - name: casey/just
    version: v0.2.32
  - name: casey/just
    version: v0.2.28
  - name: casey/just
    version: v0.2.25

Aqua has a registry.yaml that looks like this for just:

packages:
  - type: github_release
    repo_owner: casey
    repo_name: just
    description: Just a command runner
    version_constraint: "false"
    version_overrides:
      - version_constraint: semver("<= 0.2.25")
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        rosetta2: true
        files:
          - name: just
            src: "{{.AssetWithoutExt}}/just"
        replacements:
          amd64: x86_64
          darwin: apple-darwin
          linux: unknown-linux-musl
        supported_envs:
          - linux/amd64
          - darwin
      - version_constraint: Version == "v0.2.28"
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: zip
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          windows: pc-windows-msvc
        supported_envs:
          - windows/amd64
      - version_constraint: semver("<= 0.2.32")
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        rosetta2: true
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          darwin: apple-darwin
          windows: pc-windows-msvc
        overrides:
          - goos: linux
            goarch: amd64
            replacements:
              linux: unknown-linux-musl
          - goos: linux
            goarch: arm64
            replacements:
              arm64: aarch64
              linux: unknown-linux-gnu
          - goos: windows
            format: zip
      - version_constraint: semver("<= 0.3.3")
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        rosetta2: true
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          darwin: apple-darwin
          linux: unknown-linux-musl
          windows: pc-windows-msvc
        overrides:
          - goos: windows
            format: zip
        supported_envs:
          - darwin
          - windows
          - amd64
      - version_constraint: semver("<= 0.4.4")
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        rosetta2: true
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          darwin: apple-darwin
          linux: unknown-linux-musl
          windows: pc-windows-gnu
        overrides:
          - goos: windows
            format: zip
        supported_envs:
          - darwin
          - windows
          - amd64
      - version_constraint: semver("<= 0.9.3")
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        rosetta2: true
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          darwin: apple-darwin
          linux: unknown-linux-musl
          windows: pc-windows-msvc
        overrides:
          - goos: windows
            format: zip
        supported_envs:
          - darwin
          - windows
          - amd64
      - version_constraint: semver("<= 0.10.4")
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        rosetta2: true
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          darwin: apple-darwin
          windows: pc-windows-msvc
        overrides:
          - goos: linux
            goarch: amd64
            replacements:
              linux: unknown-linux-musl
          - goos: linux
            goarch: arm64
            replacements:
              arm64: aarch64
              linux: unknown-linux-gnu
          - goos: windows
            format: zip
      - version_constraint: Version == "0.10.5"
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        rosetta2: true
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          darwin: apple-darwin
          linux: unknown-linux-musl
          windows: pc-windows-msvc
        overrides:
          - goos: linux
            replacements:
              arm64: aarch64
          - goos: windows
            format: zip
      - version_constraint: Version == "0.10.6"
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        replacements:
          arm64: aarch64
          linux: unknown-linux-musl
        supported_envs:
          - linux/arm64
      - version_constraint: semver("<= 1.1.0")
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        rosetta2: true
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          darwin: apple-darwin
          linux: unknown-linux-musl
          windows: pc-windows-msvc
        overrides:
          - goos: linux
            replacements:
              arm64: aarch64
          - goos: windows
            format: zip
      - version_constraint: semver("<= 1.25.0")
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          arm64: aarch64
          darwin: apple-darwin
          linux: unknown-linux-musl
          windows: pc-windows-msvc
        overrides:
          - goos: windows
            format: zip
            replacements:
              arm64: arm64
      - version_constraint: Version == "1.25.1"
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        replacements:
          amd64: x86_64
          linux: unknown-linux-musl
        supported_envs:
          - linux/amd64
      - version_constraint: "true"
        asset: just-{{.Version}}-{{.Arch}}-{{.OS}}.{{.Format}}
        format: tar.gz
        windows_arm_emulation: true
        replacements:
          amd64: x86_64
          arm64: aarch64
          darwin: apple-darwin
          linux: unknown-linux-musl
          windows: pc-windows-msvc
        overrides:
          - goos: windows
            format: zip
            replacements:
              arm64: arm64

The one benefit of piggy-backing on other project's package registry, is the searchability of packages (both scoop and aqua offer very helpful search functions), although, if I understand correctly, there is a desire to improve package discoverability with proto plugin search <QUERY>.

[W1M0R]

For reference, this is the spec for proto on the Aqua Registry.

[W1M0R]

@milesj Can I close this feature request as not planned?

[milesj]

I haven't had time to look into this yet, but off the top of my head, it can maybe be solved in a similar fashion to how we plan to support asdf? moonrepo/tools#9

[W1M0R]

@milesj Yes, that approach does sound very promising! In short, a WASM plugin ("scoop") that knows how to process scoop manifests. The default would then use the "main" bucket of scoop, but the wasm plugin might be configured to allow other buckets. A package might be available on more than one bucket (say "main" and "edge"), so users might be able to say I want scoop package xyz from bucket edge, but the rest from main, etc.

A similar approach could be done for aqua packages, etc.

[milesj]

Yeah exactly. In theory this should work.