rfc: protocol handling enhancement

[nmggithub]

This is very much a work-in-progress, but I am opening it up so we have a centralized place to discuss these things that's not my PR.

To briefly give my personal answers to the questions posed:

• Is protocol.handle meant, by us, to be an middleware-like interception API or a provider API for the https scheme? Not sure, I didn't make it.
• Should protocol.handle be split into two different API's (e.g. protocol.handle and protocol.intercept)? Splitting might be better, if we want to support more middleware-like features.
• In the case that a more middleware-like interception API is desired:
  • Should we allow for the intercepted request to be modified before a potential fallback? Would be nice in concept, hard to implement (especially if the in-flight request is used for the fallback).
  • Should we allow for the returned response to be modified before passing it along? Could be easier to implement than request modification, but poses questions about side-effects.
  • Should the "fallback to Chromium" option use the in-flight request or dispatch a new one? A more-aligned fallback should use the in-flight request.
  • Should setting the Set-Cookie header in a returned response set the cookies? Would be nice in concept, hard to implement (when and where would the API commit to applying the cookies?).
  • Should layered handlers (middleware-like) be supported? Simple enough in concept, but side-effects may make things difficult to implement.
• What should happen to the webRequest handlers API? Perhaps deprecated if their behavior can be superseded by any new API?

[nmggithub]

After talking in the API WG meeting I'll clarify a few things here for the record:

The main goal of my questions in this RFC is to determine API intent.

My original PR added a way for users using protocol.handle to decline to handle a request, instead passing off the in-flight request to continue being handled by the Chromium. Passing requests back to Chromium seems to be a desired ability, going back to at least this issue from 2018 (before protocol.handle even existed). It also seems that some users attempt to use net.fetch with bypassCustomProtocolHandlers (which we document) to achieve this behavior, but it does not work as they expect.

Question 1

My first question is pointed.

Is protocol.handle meant, by us, to be an middleware-like interception API or a provider API for the https scheme?

If it's meant to be a provider API, that doesn't leave room for passing back to Chromium other than dispatching another request with the same properties and then returning the response from that (the net.fetch with bypassCustomProtocolHandlers way). If users are expected and required to return their own responses, there is no real way to defer a request; only bypassCustomProtocolHandlers (which, at best, as worded in the docs (emphasis mine), "allows forwarding an intercepted request to the built-in handler").

Question 2

My second question follows.

Should protocol.handle be split into two different API's (e.g. protocol.handle and protocol.intercept)?

If we decide that we don't want deferral semantics (choose not to return a response) inside protocol.handle, but we do want those semantics, we would likely have to split protocol.handle in two. Or, even if we don't want deferral semantics, perhaps there's enough reason to split up the two different use cases for protocol.handle.

Question 3

In the case that a more middleware-like interception API is desired:

These mostly touch on either use cases I see where users expect protocol.handle to work in a certain way and it doesn't (see using a Set-Cookie header), disagreements on approach (see pre-flight request modification and post-reception response modification), or fallback implementation (see use of in-flight request vs dispatching a new one). The specifics of these can be deferred until we need to answer them.

Question 4

What should happen to the webRequest handlers API?

I purposely avoided pre-flight request modification and post-reception response modification from the deferral path I built in my original PR for improving protocol.handle. The former, though, may likely be desired behavior. Requests can already be modified pre-flight with our webRequest handlers, but having that behavior separated from the protocol.handle API may not be desirable. If it makes sense (and overlaps enough), we should also improve the interplay between protocol.handle and webRequest handlers.

[nmggithub]

To push things forward, a big question I want answered: Do we want to provide a way for users, inside a handler, to decline to return a response at all and let Chromium handle it?

I'd vote "yes." And if the answer is "yes", then I'd also argue that the bypassCustomProtocolHandlers method is insufficient. I'd also argue that (re-)dispatching of any kind is less preferable of a solution than passing back the in-flight request. Using the in-flight request ensures as many semantics and as much of the browser as possible are kept intact, with the downside that it might make modification-then-deferral of requests difficult (if not infeasible).

I could potentially be swayed and convinced that (re-)dispatching a requests ends up being functionally identical. However, I've spoken to @deepak1556 and @MarshallOfSound about how the Chromium network stack works, and there doesn't seem to be a consensus on if that is true. We may also want to answer that question. Regardless, I am still an advocate for using the in-flight request, as it makes the most sense to me to achieve what users are expecting the API to behave like.