Can't use Alternative S3 URLs with Staging/Production Option

[ronilan]

Overview

It is not possible to use alternative s3 urls (path style) and s3-compatible backends (e.g. min.io) in conjunction with the production_host and staging_host option.

Status

• Originally host was defined as a string property of binary. The required AWS settings were automatically detected and extracted from the string.
• Pull request allow prod & stage s3 targets #533 defined production_host and staging_host as string properties of binary and the logic that governs which is used when. The required AWS settings were also automatically detected and extracted from the strings.
• Pull request Allow alternative s3 urls and s3-compatible backends (e.g. minio).  #576 added bucket region and s3ForcePathStyle as string properties of the binary object. The trio of keys, along with host value, enables to explicitly set access to a bucket, bypassing automatic detection and extraction.

Problem

bucket region and s3ForcePathStyle are set as properties of binary when in fact they are (together with the endpoint) properties of the host target. Since there can be multiple host targets (staging/production) but there can only be a single explicitly set bucket name, the alternate host option doesn't work in conjunction with the production_host and staging_host option.

Proposed Solution

Moving forward host will be defined as an object with optional properties.

Example 1: (S3 Virtual Host URL)

{
  "binary": {
    "host": {
      "endpoint": "https://npg-mock-bucket.s3.us-east-1.amazonaws.com",
    }
  }
}

Example 2: (Path Style alternative host)

{
  "binary": {
    "host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-production",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    }
  }
}

Example 3: (Utilizing Staging/Production)

{
  "binary": {
    "host": {
      "endpoint": "https://my-production-bucket.s3.us-east-1.amazonaws.com",
    },
    "staging_host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-staging",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    }
  }
}

{
  "binary": {
    "host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-production",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    },
    "staging_host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-staging",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    }
  }
}

Note: the proposed object keys are the same as the ones passed to the AWS sdk.

Backwards compatibility

Existing string definitions and existing keys will be supported.

Example 1: (S3 Virtual Host URL)

{
  "binary": {
    "host": "https://npg-mock-bucket.s3.us-east-1.amazonaws.com",
  }
}

Example 2: (Alternative S3 URL)

{
  "binary": {
    "host": "https://play.min.io",
    "bucket": "node-pre-gyp-staging",
    "region": "us-east-1",
    "s3ForcePathStyle": true
  }
}

Example 3: (Utilizing Staging/Production for S3 Virtual Host URL)

{
  "binary": {
    "staging_host": "https://npg-mock-bucket-staging.s3.us-east-1.amazonaws.com",
    "production_host": "https://npg-mock-bucket-staging.s3.us-east-1.amazonaws.com"
  }
}