Initializing context/content specific fetch defaults

[igrigorik]

When initiating a fetch request for a resource, the user agent initializes the fetch with content/context specific settings. For example, when fetching an <img>, the UA applies the appropriate image-src CSP checks; may advertise different set of request headers (e.g. advertise support for some image formats via Accept header, and may emit additional "hint" headers); may set different transport options, such as request priority, dependencies, etc.

How do we enable the same functionality with fetch() API? Without the above, in the worst case, an image resource fetch initiated via fetch() may violate CSP, will return an incorrect/suboptimal asset due to missing headers, and may be fetched with incorrect priority. Of course, images are just one example; same logic applies to all other content types. Related discussions:

• I want to make an image request: "I want to make an image request" w3c/ServiceWorker#279 (comment)
• Initializing fetch settings via "as": (WIP) Remove loadpolicy attribute w3c/preload#17 (comment)
• Initializing fetch settings via markup attribute: https://www.w3.org/Bugs/Public/show_bug.cgi?id=26533

---

Intuitively, the difference between using fetch() vs. say, and <img>, is that the latter knows its context and is thus able to initialize all the right checks + properties. Perhaps we can address this whole issue by providing "treat this fetch as if it was an 'image'" setting or signal? Handwaving...

fetch(url, {as: 'image', headers: {...}, otherOpt: {}, ...})

• Use as (or whatever fits best) as an explicit context signal that initializes appropriate defaults?
• headers and other fetch options are then applied on top of initialized defaults as overrides
• As a last step, UA initializes any missing / not specified headers? E.g. User-Agent, etc.

The mental model here is very similar to what @domenic proposed in #37 (comment):

fetchSettings = mergeMultimaps(asDefaults, optSpecifiedSettings, uaDefaultSettings);

Assuming above sounds plausible, we could then also enable this in other parts of the platform:

• <img src="photo.jpg" fetch-settings="{...}"> -- implicit: as == image, and fetch-settings can be used to override other UA + fetch defaults.
• <link rel=preload as="image" href="photo.jpg" fetch-settings="{...}"> -- as is declared explicitly, and options passed in through same mechanism.

WDYT, does it sound crazy?

[annevk]

To be clear, <img> and fetch() already share logic. They both invoke fetch. That algorithm takes care of HSTS, CSP, Mixed Content, Referrer Policy, Service Workers, etc. E.g. fetch() ends up as connect-src in CSP.

The problematic case with allowing fetch() to set the CSP policy to something other than connect-src is that you could then use fetch() to bypass CSP. The only way to prevent that would be to taint the response somehow. So that if you do fetch(url, {context: "image"}) the Response object can only be used by <img> and background-image and such.

About initializing defaults. We could do this in fetch, but are engines actually setting these defaults based on an enum in the network layer or are they set in the DOM layer based on the environment? I can see how we could take over handling Accept in fetch, but client hints seems trickier?

(As for header management. The way it works is that feature-specific headers are set by those invoking fetch. So currently e.g. the specification for <img> would have to set Accept and any client hint headers. And fetch adds headers in the network layer for CORS, cookies, etc. and these are therefore not exposed to service workers.)

[igrigorik]

The problematic case with allowing fetch() to set the CSP policy to something other than connect-src is that you could then use fetch() to bypass CSP. The only way to prevent that would be to taint the response somehow. So that if you do fetch(url, {context: "image"}) the Response object can only be used by and background-image and such.

That makes sense. In the case of rel=preload you'd get same enforcement when you try to use the preloaded response within <img>, etc.. just as we noted here: w3c/preload#17 (comment).

About initializing defaults. We could do this in fetch, but are engines actually setting these defaults based on an enum in the network layer or are they set in the DOM layer based on the environment? I can see how we could take over handling Accept in fetch, but client hints seems trickier?

Our goal here is to provide context to the engine where its currently missing, such that the engine can make more informed decisions. For example, when the engine sees an <img> resource, it will initialize some common defaults (e.g. Accept headers), and it may also use some environment/runtime variables to adjust priority and other settings: lower priority for below the fold images, advertise some hints if those values are known (e.g. resource width), and so on.

By contrast, if you try to fetch() an image resource today, the engine is completely blind and can't do any of the above. So, exposing "as", which can communicate that this is an "image" resource is already a big step forward. The engine may not have as rich of a context to determine all the other plausible optimizations, but our goal here is not to provide feature parity... When you're using fetch() you're opting into "manual control", which is why custom headers and other properties are important -- e.g. I want to override the UA defaults and use fetch to advertise own hints and other fetch settings.

In short, I think "as" is sufficient as a bootstrapping mechanism to help the UA set some defaults, it doesn't need to provide the exact same behavior.

[annevk]

You have to get more specific than "engine" and "some defaults". We actually have to define that in more detail than just a hint since it is all rather observable and user agents will be required to support the same functionality.

[igrigorik]

Sure, let's make it concrete. The developer provides the destination context via "as" attribute, e.g..

fetch('/gallery/photo.jpg', {as: 'image'}).then(...)

or

<link rel=preload href=/gallery/photo.jpg as=image>

The user agent needs to...

1. Enforce CSP based on specified context? /cc @mikewest
2. Set appropriate HTTP request headers based on context - e.g. Accept, etc.
3. Set appropriate request priority: for HTTP/1.x this is for internal prioritization; for HTTP/2 this prioritization information is communicated to the server.

I know that in Blink we advertise different headers, and assign different priorities, based on type of resource being fetched. AFAIK, same applies to FF, WebKit, and IE.

@pmeenan @mcmanus @toddreifsteck any thoughts or comments on this one?

[annevk]

What I'd like to do is that if we're certain that e.g. Accept is always set based on a request context, we specify that logic as part of Fetch. That way it is clear when Accept is set, whether service workers can observe it, and what kind of values it can have. (That would mean that <img> no longer needs to talk about Accept but can depend on the request context.)

And then go through that for each desired feature so that it's clear what the realm of a request context (e.g. "image") is and what's in the realm of a feature (e.g. <img>).

In the case of fetch(url, {as: "image"}) we'd need to restrict the Response so that it can only be used by <img> and background-image (otherwise CSP is broken). At this point, that would only make such an API useful in service workers (and even that is a bit of a stretch as service workers have their own CSP policy), but perhaps in the future <img> et al can be fed a Response directly.

[igrigorik]

What I'd like to do is that if we're certain that e.g. Accept is always set based on a request context, we specify that logic as part of Fetch. That way it is clear when Accept is set, whether service workers can observe it, and what kind of values it can have. (That would mean that no longer needs to talk about Accept but can depend on the request context.)

That makes sense. What's the best way to tackle this? Go through each context and compile a set of context-specific headers and values into a table, or some such? Would it make sense to draft some fetch-spec boilerplate that we can fill in as we go? Might help clarify what we need, etc.

In the case of fetch(url, {as: "image"}) we'd need to restrict the Response so that it can only be used by and background-image (otherwise CSP is broken). At this point, that would only make such an API useful in service workers (and even that is a bit of a stretch as service workers have their own CSP policy), but perhaps in the future et al can be fed a Response directly.

Not sure I follow the "useful in SW only" - how so? For <img> case, I'm picture: fetch() the data, then img.src = window.URL.createObjectURL(responseBlob)... not sure what the right/required security plumbing to make that work though.

[annevk]

So Accept and Accept-Language are set by pretty much all contexts except for "fetch". EventSource sets an header but it needs to set that itself. (Also, I'm not sure it makes sense to allow fetch(url, {as:"eventsource"}). Perhaps we should limit the allowed values initially.) I could come up with language for Accept and Accept-Language I think.

Re: "useful for SW only". There is no Blob for opaque responses. So we'd need some new API that creates a URL from a Response to make that work in such a way. I was even thinking that it might be nice if that API could take a promise, so you could have:

img.src = createResponseURL(fetch(url, {as:"image"}))

(The other reason why I think we want it for a Response rather than a body of sorts is that then we keep the response headers available.)

[annevk]

An alternative approach would be to reuse the media element srcObject API on other objects and make one of the accepted objects a Response. That does seem a little cleaner as using a URL effectively requires another roundtrip through Fetch (and gets a bit ickier with GC). @zcorpan, is srcObject being considered for <img>?

[zcorpan]

Yes: https://www.w3.org/Bugs/Public/show_bug.cgi?id=23502 (marked as "Needs Impl Interest")

[yoavweiss]

Enforce CSP based on specified context? /cc @mikewest

Having as set context makes sense, but wouldn't that prevent authors from specifying various priorities to the fetched resource? e.g. If I have a background image that's extremely important and I want it to download in the same priority as CSS, I won't have a way to communicate that to the browser. IIRC that use case is the main reason the attribute is called as rather than context.

If we go that route then maybe, for the case of <link rel=preload>, we need to add some priority hints on top of the context.

[annevk]

I don't understand why you think you would not be able to control the priority. That's a largely orthogonal feature we have not sorted out yet.

[yoavweiss]

I agree that it's orthogonal to context, but the original intention of as was to indicate download priority to the browser. The way this discussion is going, we'll probably need to add another, separate hints to enable the "download a low priority context in high priority" use case. (which is fine, but we'd need to address that)

[annevk]

If the intent was fetch priority, as would be way too generic a name and not that useful (priority is much more than a context). I recommend opening up a distinct issue for indicating priority.