Add WebDriver as Required spec to "Other web specifications"

[mavgit]

I believe we need to add the W3C Candidate Recommendation WebDriver as a required spec. WebDriver supports remote testing of browsers. It's particularly important for testing consumer products. I believe we will need it for our testing program.

WebDriver is not currently tracked at caniuse. (Vote here to add WebDriver to caniuse.)

However, all four target web clients have support docs for how to use WebDriver:

• Chromium: ChromeDriver
• Edge: Microsoft WebDriver
• Gecko: GeckoDriver
• WebKit: safaridriver

The added line to section 2.9 Other web specifications would be:

• WebDriver [webdriver]

[jpiesing]

Other places have discussed making webdriver required and rejected it as manufacturers felt they would not be able to meet some content provider's compliance & robustness / security requirements if it was enabled by default or enabled by some kind of password that could leak. This is about security requirements by app providers typically not using DRM.

[mavgit]

I agree that security is important. Since this is now shipping in consumer products like smart phones, does anyone know what the security practice is for WebDriver in shipping products?

This is about security requirements by app providers typically not using DRM.

I don't understand this. Can you expand? Is the concern about using WebDriver for content theft?

[jpiesing]

I don't understand this. Can you expand? Is the concern about using WebDriver for content theft?

Device manufacturers have to sign agreements with content providers in order to get content and these agreements typically include language on security. Device manufacturers felt that their devices would not comply with the terms of those agreements if they shipped with webdriver enabled or trivially enable-able.

Device makers can only guess exactly what motivates the language in those agreements but several felt that shipping with webdriver enabled would not comply or at least there was a significant risk that it would not comply.

[mavgit]

Device makers:

1. How are devices currently shipping with WebDriver addressing the security of that API?

2. Is there a way to have the benefit of WebDriver for WAVE testing without compromising device or content security?

[jpiesing]

Device makers:

How are devices currently shipping with WebDriver addressing the security of that API?

Is there a way to have the benefit of WebDriver for WAVE testing without compromising device or content security?

@mavgit I think you should be asking content providers - would they be happy with their services running on devices where WebDriver was enabled by default or where it could be enabled with something trivial that could easily leak into the wild?

[mavgit]

I would love to hear from content providers or anyone who can make this issue concrete.

Given virtually all content providers currently deliver content to PC browsers that all support WebDriver, is there really an issue?

1. Can anyone confirm that browser support for WebDriver blocks any web media apps and describe the issue?
2. Is the issue addressed by implementation of the recommendations in the WebDriver Security Considerations section?
   https://www.w3.org/TR/webdriver/#security-considerations
3. Is the issue addressed by the additional limitations implemented by Safari/WebKit. See "Safari-exclusive Safeguards"
   https://webkit.org/blog/6900/webdriver-support-in-safari-10/

If no one can make this issue concrete, all we can do is call for consensus on adding WebDriver.

[davemevans]

On 28/06/17 telco, @mavgit requested Content Providers to contact him privately, and took an action to proactively contact them for input.

[mavgit]

My further understanding is that there is concern that WebDriver can enable easy source code access to commercial web media apps targeting Smart TVs.

To be clear, I am proposing WebDriver for only one use case: enabling groups like W3C, ATSC, HbbTV and WAVE to run platform tests on Smart TVs and other consumer devices. The only source code to be displayed for this use case is the test source code.

I am not trying to enable WebDriver for the use case of debugging the commercial, consumer-targetted web media apps that the consumer may download, for instance from an app store.

Given this limited use case, I hope that devices that want to support the test use case, but prohibit the commercial source code access use case, can do so with limitations on the enablement of WebDriver, similar to those referenced in my previous comment #71 (comment)

[jpiesing]

@mavgit As long as WebDriver can't be used on commercial web media apps then I think making it "Required" in our spec would be fine. Of course how to express that requirement would be interesting. A lot of the references rely on the user which doesn't work in all devices.

[mavgit]

As long as WebDriver can't be used on commercial web media apps then I think making it "Required" in our spec would be fine. Of course how to express that requirement would be interesting.

I agree that it would be "interesting" (i.e. difficult or impossible) to express this requirement (let alone create tests to verify this requirement) since the specific security requirements are currently part of private agreements with certain content providers and the specific security requirements may vary between providers and also vary over time and also vary by device type (e.g. PC vs. smart TV).

Fortunately, there seem to be multiple technical solutions available to device makers to support WebDriver while also addressing these requirements, as far as I understand.

I would encourage anyone who thinks there need to be additional security requirements on WebDriver to submit the use cases as WebDriver issues.

Therefore, I Call for Consensus to add a requirement for WebDriver to section 2.9 Other web specifications:

WebDriver [webdriver]

[jpiesing]

Sorry @mavgit but just adding a reference to WebDriver takes us right back to the beginning of this discussion. I think it needs some additional text explaining that it needs to be possible to enable it for running tests but there's no intention/requirement for it to be possible to enable it for debugging media web apps. It could then have a note referring to the security considerations in the web driver spec and saying that for devices that cannot themselves host a debugger, the suggestion that "To prevent arbitrary machines on the network from connecting and creating sessions, it is suggested that only connections from loopback devices are allowed by default. " could be relaxed to limit connections to those coming from hosts on the same network as the device.

[mavgit]

I think it needs some additional text explaining that it needs to be possible to enable it for running tests but there's no intention/requirement for it to be possible to enable it for debugging media web apps.

This seems OK to me. Perhaps the latter part could be something like: "The ability to debug media web apps is not required and is up to the manufacturer."

It could then have a note referring to the security considerations in the web driver spec and saying that for devices that cannot themselves host a debugger, the suggestion that "To prevent arbitrary machines on the network from connecting and creating sessions, it is suggested that only connections from loopback devices are allowed by default. " could be relaxed to limit connections to those coming from hosts on the same network as the device.

I don't support our spec references changing the referenced specs - especially in a security section. I'd rather work with the spec authors, as we did with Fullscreen API, where our issue has traction.

Here's a suggested way forward to resolve this issue in parallel to publishing a review version of our spec:

1. You open an issue on the Web Driver repo explaining the security issue wrt testing consumer devices.

2. We add WebDriver to required specs and put in a note that states:

• your first point above about usage intended for testing
• that the inclusion of WebDriver is tentative due to resolving the issue above and include a link to the WebDriver issue.

This way we can get feedback on the whole spec, including our interest in WebDriver and might drive more people to comment on our WebDriver issue.

[jpiesing]

Looking at this in more detail, there is a difference between what is being debugged and where the debugger is running. The two are separate but may be confused in some of the discussions earlier. The security considerations section of the web driver spec is about where the debugger is running and does not address what is being debugged. A webdriver implementation that meets the security considerations section of the webdriver spec is (IMHO) completely pointless in the case of a device that cannot host a debugger itself. It would also prevent WebDriver from being used to run WAVE test cases from any kind of test harness or test automation. The Safari security guidelines also do not address what is being debugged.

If existing browser code bases don't support restricting what can be debugged then I would be nervous about including this in our spec.

[mavgit]

there is a difference between what is being debugged and where the debugger is running

This is indeed a helpful distinction.

It seems you have two issues:

1. A webdriver implementation that meets the security considerations section of the webdriver spec is (IMHO) completely pointless in the case of a device that cannot host a debugger itself. It would also prevent WebDriver from being used to run WAVE test cases from any kind of test harness or test automation.

This seems like an issue with applying WebDriver to a consumer product:

2. If existing browser code bases don't support restricting what can be debugged then I would be nervous about including this in our spec.

This seems like a question/issue as to whether WebDriver can limit what can be debugged.

Could you file issues on both of these on the WebDriver repo, so we can either find a solution or get WebDriver fixed?

As to publishing the review version of the spec, can we include a note that adding WebDriver as a required spec is contingent on resolving these two issues?

[jpiesing]

there is a difference between what is being debugged and where the debugger is running

This is indeed a helpful distinction.

It seems you have two issues:

    A webdriver implementation that meets the security considerations section of the webdriver spec is (IMHO) completely pointless in the case of a device that cannot host a debugger itself. It would also prevent WebDriver from being used to run WAVE test cases from any kind of test harness or test automation.

This seems like an issue with applying WebDriver to a consumer product:

Yes.

    If existing browser code bases don't support restricting what can be debugged then I would be nervous about including this in our spec.

This seems like a question/issue as to whether WebDriver can limit what can be debugged.

Is this an issue of the WebDriver spec or of specific implementations? I suspect it's the latter.

Could you file issues on both of these on the WebDriver repo, so we can either find a solution or get WebDriver fixed?

As to publishing the review version of the spec, can we include a note that adding WebDriver as a required spec is contingent on resolving these two issues?

Yes.

[mavgit]

Call for Consensus: include a note that adding WebDriver as a required spec is contingent on resolving these two issues.

Needs pull request.

[davemevans]

#88 changed the title of the document to snapshot, and #84 moved away from "requiring" towards documenting the current state of the web platform as a whole.

As a result, it seems to me that, adding a specification to Section 2 doesn't "require" it but merely documents it as being well supported, documented exceptions aside.

It isn't clear to me how we can add this to Section 2 with the caveat above (#71 (comment)).

(EDIT: This is a specific issue, but I have also raised the more general question in #93)

[mavgit]

In my previous comment #71 (comment) , I meant that we'd list WebDriver in the usual way, but add an editor's note that this feature is at risk contingent on resolving some issues. We should also include a reference to w3c/webdriver#977 into the editor's note.

[davemevans]

Closed via fbdbccc.