Cropping non-self-capture tracks

[eladalon1983]

Assume getDisplayMedia() is called and the user chooses to capture another tab. The spec currently disallows cropping in such scenarios. But why?

With Capture Handle Identity, it is possible for the capturing application to discover that it is capturing a tab with a known application. For example, maybe meet.google.com is capturing docs.google.com. Such same-site applications will be able to communicate with each other using a BroadcastChannel[*], and the capturee can postMessage() a CropTarget to the capturer. Usecases are obvious (see example above). Why should we disallow this?

--
[*] Using the mailman pattern. If the top-level documents are same-site, this communication method will not be blocked by Storage Partitioning.

[eladalon1983]

As mentioned during the interim yesterday (2022-06-23), it's incumbent on those who wish to maintain this limitation to explain why it's necessary. Barring a good explanation, we have resolved[1] to remove this limitation in the interim after the next one.

One argument mentioned by @jan-ivar[2] during that meeting as well as in the past, was his concern that a captured application might falsely advertise a CropTarget consisting of 0x0 pixels, and the capturer might mindlessly apply this CropTarget, to the detriment of the user.

I do not find that concern compelling:

• Why would the capturer apply an unknown CropTarget from an unknown source?
• Why can't it solve its own issue by NOT applying that CropTarget?
• Is the Web in general better off without this useful feature, just because very rare applications might use it unwisely?
• How does the capturee transmit the CropTarget to the capturer?
  • Using BroadcastChannel or similar? So they're same origin and have trust.
  • Using Capture Handle and stringification or similar? Then it can check the origin and ignore CropTargets from unknown/untrusted sources.

I am looking forward to hearing either counter-counter-arguments or alternative rationales for maintaining the limitation. Barring those, I hope that we can all agree that removing the limitation would be useful; we don't necessarily have to wait two interim meetings for that.

CC interested parties as reflected by the active participants of the meeting: @dontcallmedom, @aboba, @alvestrand, @jan-ivar, @youennf, @steely-glint.

--
[1] I shall point to the minutes once they're posted.
[2] The usual caveat - if my attempt at representing was your concern fell short, please set the record straight by posting a better description of it. I will edit my own comment to better reflect your stated position, most likely by simply linking to it.

[DannyMoerkerke]

I'm building a screen recording PWA that uses .getDisplayMedia() to select a tab, window, or screen to record.
It would be nice if any source could be cropped.

[alvestrand]

@DannyMoerkerke how would you like to define what to crop to in the window or screen case?
I think it would be good to take that discussion off into a separate bug.

[DannyMoerkerke]

Is that different from the tab case?
Just started playing with it so my understanding is still limited.

In the app I preview the stream returned from .getDisplayMedia() using a <video> tag and I added a resizable cropArea <div> to the common parent element, just like the demo.

[alvestrand]

In the tab case, we crop to an element of the source page. Read the spec for the considerations (security among them) that led to this design rather than "crop to these coordinates".
Windows and screens have no HTML elements on them.

[beaufortfrancois]

Hello @DannyMoerkerke, as said in https://developer.chrome.com/docs/web-platform/region-capture/#future, we hope to bring improvements to the current state of Region Capture:

• Region Capture will support all Element types.
• Region Capture will support captures of other tabs.

[jan-ivar]

Barring a good explanation, we have resolved[1] to remove this limitation in the interim after the next one.

I disagree with this summary, as no such promises were made. Let's stick to discussing the issue itself here, keep conversation inclusive, and limit comments to new information on the issue topic. Feel free to email chairs with questions about process.

My action was to clarify my concerns, which I'll do next.

[jan-ivar]

Mozilla judges the impact of API changes not just on API users, but on end-users as well. The more sites become involved in what was traditionally a 100% user-controlled power tool (getDisplayMedia), the more we have to consider the tradeoff of loss of user-control as a side-effect of site involvement. Current discussion assumes all site involvement is helpful. What if it isn't? We'd like to make sure user agents can offer options to keep the user in control should sites make decisions that aren't helpful.

My concern is about unintentionally enabling self-censoring:

• With this new API, a service could decide to offer cropping to any site that asks. Today it can't.
  • This doesn't seem farfetched given that we see "*" passed as origin to postMessage and allow= all the time. Expecting sites to show restraint in the interest of end-users doesn't work for all sites.
• This could be exploited by captured sites to defeat capture (self-censor), which may neither be the capturing site's intent or in the end-user's interest.

Allowing sites to self-censor may have merit or not, and merits explicit discussion, which we haven't had. Hopefully this isn't controversial. I'd like to separate comprehension of this argument from whether we agree what weight to put on it.

This has been an ongoing concern, but I'll explain why I think this particular feature crosses the Rubicon.

• I concede that a service today could decide to let captured sites know they're being captured. It seems reasonable to assume such a service would understand the consequences of that: it lets captured sites self-censor by modifying their DOM. But those captured sites wouldn't know when capture ends, and would be unable to uncensor again, a usability issue for this strategy, and thus e.g. they'd likely not take the extreme step of trying to subvert a user refreshing the page to workaround this censoring.

• I concede that a service today might attempt to let captured sites know when capture ends, though I'm not sure how well it would work in practice if the user closes a tab or a process is killed. In any case, the service's intent in this case also seems clear.

• In contrast, censoring with a cropTarget seems a lot simpler (if censoring is one's goal) since it only affects the capture, not the DOM. It seems concerning to me that services cannot offer cropping to sites without simultaneously risking this power being used to self-censor. This seems new with this proposal, given that this is the first tool that solves it satisfactorily IMHO (based on the shortcomings I described above).

Mozilla isn't necessarily opposed to self-censoring, provided users can override it. But step one is acknowledging this risk. Step two for us is, to find ways to mitigate this concern, which for us to support this feature, would require some language around this, e.g.

User agents MAY offer users the option to defeat cropping through this feature outside of self capture.

[eladalon1983]

Feel free to email chairs with questions about process.

Where your chairman hat is right now? This proposal was supported by (1) yours truly, (2) @steely-glint, (3) @alvestrand and (4) @dontcallmedom. It does not look like it would be necessary to help us build consensus, so I assume you're taking the chairman hat off?

[eladalon1983]

Mozilla isn't necessarily opposed to self-censoring, provided users can override it.

The app-based solution of not applying CropTargets from unknown sources, or offering app-based overrides to stop cropping, seems perfectly sufficient. I don't see why the user agent should step in and help. Capturing apps can do crazy things like invert pixels, reshuffle them, etc. We don't offer user-overrides for those, nor should we.

Nevertheless, if adding that user agents MAY offer overrides builds consensus - let's do it!

Quoting your proposed text again:

User agents MAY offer users the option to defeat cropping through this feature outside of self capture.

I don't think it's useful, but I have no desire to block Mozilla from offering that option to its users. Let's add that text.

[alvestrand]

Jan-Ivar, the repeated use of the term "censoring" confuses me. This is usually a negatively loaded term, used to describe undesired behavior. I can easily see how this feature could allow non-useful things to happen, but API design is not about preventing stupid things.

The sentence "censoring with a cropTarget" sounds far less dangerous if described as "proposing a crop with a cropTarget".

I do not understand what the behavior is that is perceived as a risk here - who is at risk, and what is the impact?

[steely-glint]

User agents MAY offer users the option to defeat cropping through this feature outside of self capture.

It seems to me we already have this - at both ends. The captured webapp can choose not to create (or transfer) a cropTarget. The capturing webapp can choose not to apply any cropTarget it gets.

'self-censorship' isn't possible - both ends have to collaborate to get the 'censor' effect.

[jan-ivar]

Why is it impossible for both ends to collaborate to get the 'censor' effect? The end-user is the one browsing the web in a browser, and there's precedent for "undesired behaviors" not aligning between them and sites. E.g. popups and autoplay.

Maybe an example would help:

1. A video conferencing site uses Capture Handle Identity to implement a communication channel and documents how to use it on the open web for any site to use (or a select few partners, or for a fee, doesn't matter).
2. This communication channel accepts any CropTarget.
3. All bank/credit card/shopping sites, say, quickly use this to defeat capture of any page with account or transaction information (makes perfect sense to them).
4. A user is in a one-to-one video call with their accountant/financial advisor/spouse whatever, attempting to go over expenses, and screen-sharing isn't working or they're told they're not allowed to share those pages.

I purposely picked an example with some merit on the side of censoring being desirable to protect users from themselves and scammers. But we also know restraints can get in the way of user choice, which is why most browsers offer choices around popups and autoplay. That's what we're asking for here.

[jan-ivar]

I appreciate discussions involving a co-chair can be confusing. FWIW I never invoke my chair role without stating it.

My rationale for this is we have 3 co-chairs, and weekly co-chair meetings, so chairs speaking as chairs on their own should be rare (and with ⅓ weight anyway). I'll say "For the chairs" if I'm speaking for all chairs, or "with my chair-hat on" if I'm not.

[eladalon1983]

Jan-Ivar, this message of yours clarifies the scenario that worries you. But I believe all participants were clear on that matter. The issue of unclarity that I believe Harald, Tim and I share, is that the app can expose app-based user-facing controls to stop cropping. (Bonus: App-based user-overrides are equally available on all platforms, and look the same for all users of the app.)

So I don't think users would benefit from these browser-based overrides. But I am curious to see what UX Mozilla will implement for them. I hereby volunteer to participate in the beta.

Action item: Shall I prepare the PR for removing the self-capture limitation, and you (@jan-ivar) the PR for adding the MAY line?

[jan-ivar]

App-based user-facing controls to stop cropping sound great! Do we know if Google Meet is planning to have them?

Not all sites will have sufficient incentives to provide such controls however — imagine app-based controls for popups and autoplay — That's where user agents can help. We don't need help on UX though thanks, nor does that need to block the spec.

I'd prefer reviewing a single PR adding both changes at once.

[eladalon1983]

App-based user-facing controls to stop cropping sound great! Do we know if Google Meet is planning to have them?

I am not aware of a Meet plan to automatically receive and apply CropTargets, so why would user-facing controls to stop cropping be needed? That's the only case where this concern would be applicable, after all.

Not all sites will have sufficient incentives to provide such controls however

Sites are incentivized to keep their users happy by implementing functionality that users deem necessary. Otherwise, users migrate to competing products - as well they should.

imagine app-based controls for popups and autoplay

I don't think these scenarios are similar to auto-cropping. And let's not forget that auto-cropping is highly-hypothetical. But that doesn't matter, as we've agreed to add MAY and that's all that matters.

I'd prefer reviewing a single PR adding both changes at once.

What if we merge your PR, which adds this MAY text, ahead of mine? Would that make you more comfortable? To explain my own reluctance to add it myself - I think he who believes this MAY is necessary should (1) come up with the right formulation and (2) sign their own name to the PR.

[jan-ivar]

I'm happy with the formulation I provided in #63 (comment). We're both editors tasked with merging what the WG decides, so I'd expect a single PR to close this issue to do the trick. I'm not worried about attribution. Thanks

[eladalon1983]

Jan-Ivar, please do your own work and sign your own name to it. This MAY is your conviction. I have politely explained why I find it completely unnecessary, but that I am happy to add it as a compromise. Kindly produce and send your own PR. Thank you.

You're also welcome to suggest-change on my PR when it arrives, adding your MAY that way.

[alvestrand]

Discussion of #63 in June 23 interim: https://www.w3.org/2022/06/23-webrtc-minutes#t06