feat: http caching

[flakey5]

Implements bare-bones opt-in http caching as per rfc9111. Bare-bones in this case means what's required by the spec and a few extra bits, with more coming in future prs.

Opening as a draft since there's still some more work to be done (mostly tests, but a bit more functionality-wise)

No request cache directives are supported at this time, this will come later.

Response caching directives supported:

• public
• private
• s-maxage
• max-age
• Expires header
• no-cache
• no-store
• stale-while-revalidate

This relates to...

Closes #3231
Closes #2760
Closes #2256
Closes #1146

Changes

Features

• Opt-in http caching
• In-memory default cache store

Bug Fixes

n/a

Breaking Changes and Deprecations

n/a

Status

• I have read and agreed to the Developer's Certificate of Origin
• Tested
• Benchmarked (optional)
• Documented
• Review ready
• In review
• Merge ready

[ronag]

Would it maybe be better to use jsdoc types rather than separate type definition files? @mcollina

[mcollina]

Would it maybe be better to use jsdoc types rather than separate type definition files? @mcollina

I think using type definitions are in line with the rest of the codebase, and therefore the correct implementation.

If we want to migrate to a jsdoc-generated world, let's discuss in another issue!

[ronag]

@flakey5 please make sure to add me and @IsakT as co-authors (assuming you took some of our code in nxt-undici).

[Uzlopak]

headers could be null or undefined

[Uzlopak]

@ronag
Should we use fastTimers?

[flakey5]

Hmm so with sharing an in-flight response to requests to the same resource, how do we want to handle if the body ends up exceeding the maxEntrySize?

Currently when this happens, we close the writable. However, the readables waiting for body chunks are just left hanging.

I think we could either

1. emit an error in the readable that makes it way to handler.onError, aborting the request
2. let the requests hanging on this just pass through to the origin

The first would be the easiest to implement but the second is what I intuitively think would happen. The second option comes with some implementation difficulties though.

When we create the read stream in the interceptor, we would need to listen for some passToOrigin event, this part is simple enough. The issue is we would also need to not send the body until the response successfully finishes. This doesn't work great with createReadStream returning a Readable, since we need to store the data for each request separately until the response finishes.

The implementation would look something like

const dataChunks = []
stream.on('data', dataChunks.push)

let passedToOrigin = false
stream.on('passToOrigin', () => {
  passedToOrigin = true
  dispatch(opts, handler)
})

stream.on('end', () => {
  if (!passedToOrigin) {
    // actually send the body chunks
  }
})

Adding to it, this doesn't apply to HEAD requests since there isn't a response body for those.

Something that might work is allowing createReadStream to return just a generic EventEmitter that would give us the body chunks so we're not keeping a separate copy for each request, i.e.

stream.on('end', chunks => {/* ... */})
stream.on('passedToOrigin', () => {/* dispatch */})

If we want to, we could go with option 1 for simplicity sakes and then later down the road add support for option 2. Wdyt?

[ronag]

TBH I think we should skip in-flight de-duplication in this first phase. @mcollina wdyt?

I would propose:

Step 1: in-memory cache
Step 2: sqlite cache
Step 3: in-flight de-deduplication

[mcollina]

TBH I think we should skip in-flight de-duplication in this first phase. @mcollina wdyt?

I think this might be handled here already. Why do you want to remove it?

[flakey5]

I think this might be handled here already. Why do you want to remove it?

Re #3562 (comment)

[flakey5]

I would propose:

Step 1: in-memory cache Step 2: sqlite cache Step 3: in-flight de-deduplication

To add to this, I'll also be opening another pr that adds support for cache control directives specified by the request (wip branch https://github.com/flakey5/undici/tree/flakey5/20240924/cli-cache-control)

[flakey5]

Pushed a commit removing the request de-dupe code re #3562 (comment) and #3562 (comment). Can revert it if we do want to have it, but, I agree that it should be added afterwards in a pr dedicated to it. That way we can address the issues there without tying the rest of this up.

[mcollina]

lgtm

[ronag]

I have opinions... Will try to make time over weeekend.