Add baseUrl option, rename prefixUrl, and allow leading slashes in input

[sholladay]

This PR aims to improve the flexibility of input URLs and options for resolving them prior to the request.

• prefixUrl is renamed to startPath
  • As before, when this option is used, it is prepended to input before any other processing takes place
  • startPath can still be a full URL if needed, however the name is meant to signal its primary use case is a simple base path without a domain, e.g. /api
• A new baseUrl option is added
  • When this option is used, input (including any startPath) is resolved against the baseUrl according to standard URL resolution rules
  • baseUrl will respect the HTML <base> tag if present, and is similar to it, except that it only applies to Ky
• input can now be any valid URL, with or without a leading slash
  • Slashes are automatically normalized when using startPath
  • Slashes are kept as-is and respected in accordance with standard URL resolution rules when using baseUrl

ky('books', { startPath : '/api' });  // GET /api/books
ky('books', { baseUrl : '/api' });  // GET /books
ky('books', { baseUrl : '/api/' });  // GET /api/books
ky('./books', { baseUrl : '/api/' });  // GET /api/books
ky('/books', { baseUrl : '/api/' });  // GET /books
ky('http://bar.com', { startPath : 'http://foo.com' });  // GET http://foo.com/http://bar.com
ky('http://bar.com', { baseUrl : 'http://foo.com' });  // GET http://bar.com
ky('books', { baseUrl : 'http://foo.com', startPath : '/api' });  // GET http://foo.com/api/books

Possible names for the string that is prepended to input

• startPath (as is currently implemented here)
• pathPrefix
• prefixPath
• rootPath

TODO:

• Finalize name for startPath option
• Documentation
• TypeScript types

[faradaytrs]

i see that it's going to be a breaking change, maybe it's worth about discusssing about the next major? i think it's good to remove default timeout and retry settings, because many people come from fetch and axios and it's not obvious ky have these defaults.

[sholladay]

I would be okay with removing the default timeout, but not the default retries.

[alexgleason]

baseUrl looks like what I want. I only care about the URL's origin, and also need the ability to pass full URLs. If it works like this, it's perfect:

ky.get('/api/v1/instance', { baseUrl : 'https://ditto.pub' }); // GET https://ditto.pub/api/v1/instance
ky.get('https://ditto.pub/api/v1/timelines/home?max_id=1234', { baseUrl : 'https://ditto.pub' }); // GET https://ditto.pub/api/v1/timelines/home?max_id=1234

In this case, I don't care about a path in the baseUrl. I assume it works like this:

ky.get('/api/v1/instance', { baseUrl : 'https://ditto.pub/@alex' }); // GET https://ditto.pub/api/v1/instance

But I will never pass a baseUrl with a path anyway.

[sholladay]

@alexgleason correct, each of those examples behaves as you described.

I would expect most people to structure their requests like this:

const api = ky.extend({ baseUrl : 'https://ditto.pub/api/v1/' });
api.get('instance'); // GET https://ditto.pub/api/v1/instance
api.get('timelines/home?max_id=1234', { baseUrl : 'https://ditto.pub/api/v1' }); // GET https://ditto.pub/api/v1/timelines/home?max_id=1234
api.get('https://example.com'); // GET https://example.com
api.get('/api/v2/secret'); // GET https://ditto.pub/api/v2/secret

But you could certainly put the api/v1 part in the input instead of at the end of baseUrl, that works, too.

[alexgleason]

@sholladay I think the way to overcome code complexity and confusion while solving all use-cases is to just support a resolver function like mentioned here: #291 (comment)

But I think it should actually look like this:

const api = ky.create({
  // Accept any `Input` and return any `Input`. Flexible and pure.
  resolver(input: Input): Input {
    // It's not actually complicated to do this yourself either.
    const url = input instanceof Request ? input.url : input.toString();
    return new Request(new URL(url, 'https://ditto.pub'), input);
  }
});

We can keep the prefixUrl for backwards-compatibility, but internally it just creates a resolver, so you can't have both.

const api = ky.create({
  prefixUrl: 'https://ditto.pub', // WARNING: `prefixUrl` is deprecated.
  resolver(input: Input): Input { // ERROR: `prefixUrl` and `resolver` cannot be specified at the same time.
    // ...
  }
});

For improved DX, Ky can include resolvers for a prefix and baseUrl.

import ky from 'ky';

const api = ky.create({
  resolver: ky.prefix('https://ditto.pub/api/v1')
});

const api = ky.create({
  resolver: ky.baseUrl('https://ditto.pub')
});

[sholladay]

@alexgleason the way I am planning to solve that is by allowing users to return a URL instance in a beforeRequest hook. Do you think that is sufficient? It avoids needing to add another option.

[alexgleason]

@sholladay That could work as long as we have the input there too. In fact that's the only reason it doesn't work already. You don't even need to let it return a URL, you just need to add the input as a parameter.

[alexgleason]

Can it even get to the point of beforeRequest without already having a valid absolute URL though? I don't think it can construct the needed Request object unless you're already specifying a prefixUrl.

It's fine as a hook, but I think it might still have to be a separate hook. beforeInput or something.

[sholladay]

While it doesn't provide input, you can use request.url to see where the request is going to go. That might not suit every use case, though.

[alexgleason]

@sholladay Currently we can already do that with beforeRequest, but when the request looks like Request { url: "https://ditto.pub/https://ditto.pub/api/v1/timelines/home?max_id=1234" } it's not very helpful. I need the Input so I can join it with a baseUrl. The input might be full URL, or just a path like /api/v1/instance.

[sholladay]

What's the use case that causes you to need a base URL that is unknown at the time of creating the Ky instance? In other words, why is it dynamic?

[alexgleason]

@sholladay Supporting multiple accounts on Mastodon.

[alexgleason]

I ended up just writing a custom bare minimum fetch wrapper and accepting that I will have to do await (await ()).json() all over my application code. https://gitlab.com/soapbox-pub/soapbox/-/blob/main/src/api/MastodonClient.ts?ref_type=heads

[sholladay]

@bertho-zero you didn't explain what you prefer about this PR and why.

Your regex is shorter but the current code is more readable and should have slightly better performance. Both are fine, though, and I don't have strong feelings about it.

[LinusU]

Leading slashes in input are disallowed when using this option to enforce consistency and avoid confusion [...]

From what I can tell from the changed code, this is no longer the case?

I personally prefer it that way though and think that we should keep that behaviour

[sholladay]

Indeed. As mentioned in the TODOs section, I haven't gotten around to updating the docs yet, other than a simple find & replace. But thank you for the reminder. 🙂

[bertho-zero]

I like #561 because I don't understand why we should throw an error if the input starts with a slash, I read the topic but still don't find this constraint justified.

I don't like #606 because I can have a prefix like https://domain.com/api/v2 and I don't want to have to parse the prefix to put the host in baseUrl and the path in startPath.

[bertho-zero]

I think if _input starts with a protocol (or is a valid URL?) then we should ignore startPath in order to avoid an URL like http://foo.com/http://bar.com

[sholladay]

That kind of "double URL" construct is useful for reverse proxies. It should be supported.

Also, you would be surprised how hard URL parsing is. Especially for a library like Ky that supports non-browsers.

[sholladay]

I don't like #606 because I can have a prefix like https://domain.com/api/v2 and I don't want to have to parse the prefix to put the host in baseUrl and the path in startPath.

You don't need to split it up like that, you can just put the whole thing in startPath if you always want that host to be used. I realize that may not be entirely obvious. I am open to suggestions for how to make it more obvious. We could just keep the name prefixUrl , then it would be more obvious to me, but that's just too similar to baseUrl, so it creates a new kind of confusion.