Brave should download Chrome extensions without going through go-updater

[levicki]

I see a lot of threads on Reddit and elsewhere of people complaining about "Download interrupted" error when trying to install Chrome extensions in Brave and this is happening on new profiles and with shields and all other extensions disabled.

I have faced the same issue and I have found what the problem is.

Namely, I have blocked go-updater.brave.com and go-variations.brave.com domains on my network.

By doing that I have discovered that Brave does not download the extensions directly from Chrome Web Store as it should, but through go-updater.brave.com via some kind of redirect.

An example of an extension download attempt:

https://go-updater.brave.com/extensions?response=redirect&os=win&arch=x64&os_arch=x86_64&nacl_arch=x86-64&prod=chromiumcrx&prodchannel=&prodversion=116.1.57.47&lang=en-US&acceptformat=crx3,puff&x=id=bbeaicapbccfllodepmimpkgecanonai&installsource=ondemand&uc

I find this behavior really shady and unnecessary.

Can we please get some clarification why not use a direct download? Also, why no fallback to direct download in case of error?

Finally, can the error message be improved to say host name resolution failed or domain not found instead of generic network error?

[fmarier]

Can we please get some clarification why not use a direct download?

We built our go-updater service because there are three different cases for downloading crx files in Brave:

1. Chrome Store extensions installed by the user: we redirect to extensionupdater.brave.com which then redirects the client to Google's servers directly.
2. Chrome components which are needed in Brave too like the list of revoked TLS certificates: we redirect to componentupdater.brave.com which then proxies these components so that our user's IP address is not revealed to Google.
3. Brave components like our adblock lists: these ones (you can see the full list at https://go-updater.brave.com/extensions/test) are delivered by us because they don't exist on Google's servers.

Namely, I have blocked go-updater.brave.com and go-variations.brave.com domains on my network.

If you want to block go-updater.brave.com, then you can self-host our backend code and user the --component-updater URL to redirect your browser there. Otherwise your browser is going to be missing lots of essential components.

[fmarier]

https://github.com/brave/brave-browser/wiki/Proxy-redirected-URLs has the guidelines we follow when choosing whether or not to proxy a given third-party service.

[levicki]

@fmarier Thanks for the clarification.

However, right there in your guidelines it says that Chrome Store extensions should not be proxied:

So if I am clicking the Add to Brave button on Chrome Store page (which is a user-initiated request), why is that request still proxied?

My expectation as a user are:

1. If Shields are down or if I disable auto-redirects or if I disable any sort of Safe Browsing, the requests I make by clicking on links should go directly to the server hosting the content without the browser interfering.

2. If certain traffic is implicitly proxied, then that traffic should still be visible in dev tools in the Network tab.

Currently neither of those expectations are met.

If you want to block go-updater.brave.com...

I did it as an experiment, I do not intend to keep blocking it. However, I would really like if certain stuff wasn't proxied (like installing extensions) and that any proxying going on is both disclosed upfront and visible in dev tools so we can at least see where the request is going and why is it failing.

I assume that your intent for proxying extension installs is to protect people from bad extensions by blocklisting (my cynical self is telling me there must be more to it such as collecting usage data on extensions and getting ideas what to integrate into the browser next), but it should still be possible to opt-out of that.

[fmarier]

However, right there in your guidelines it says that Chrome Store extensions should not be proxied.

If you read past the headline, which arguably could be phrased better, you will see this:

do not typically need to be proxied

It's not that we think it's wrong to proxy these internal requests, it's about not breaking the implicit promise we made to our users: by default Brave won't talk to non-Brave servers. That promise does not include user customizations such as installing extensions.

That's why the extensions themselves (i.e. the crx files) are not actually proxied. We redirect the browser to the Google server in that case. In contrast, the essential components such as the TLS revocation list is fully proxied (the user's browser never hits Google servers for that).

If Shields are down or if I disable auto-redirects or if I disable any sort of Safe Browsing, the requests I make by clicking on links should go directly to the server hosting the content without the browser interfering.

That's true of web requests. If you put google.com in the URL bar, it will go straight there, not proxied. If you find an example of a website request that doesn't go direct, let us know because that would be a bug.

Internal requests for downloading parts of the browser (e.g. adblock lists, Safe Browsing lists) are different, see the implicit promise above. Extensions are also internal requests because they are downloaded and installed by the browser's update system, not the website.

If certain traffic is implicitly proxied, then that traffic should still be visible in dev tools in the Network tab.

By default, the devtools will show you network traffic for the current website only. If you want to see all traffic (including from "system" components), I'd recommend using a tool like Wireshark or mitmproxy. The latter is what I regularly use to ensure that we are making the expected requests.

(my cynical self is telling me there must be more to it such as collecting usage data on extensions and getting ideas what to integrate into the browser next)

If we were collecting that information, it would likely be visible in https://github.com/brave/go-update, and we would need to disclose it in https://brave.com/privacy/browser/.

To save you time, I can tell you that we are in fact not collecting that information. In fact, we have customized the client code which updates extensions so that they get updated one at a time in order to avoid creating a kind of fingerprint based on a user's potentially unique combination of installed extensions. That endpoint was never designed for collecting usage data.

If we wanted to collect a list of popular extensions, we would do it via P3A and you would be able to see it in the list of questions that this component is asking clients. This is a service that is designed for gathering these kinds of statistics at scale. It does it in a transparent and respectful way using privacy-preserving techniques and providing an opt-out in brave://settings/privacy.

[levicki]

If you read past the headline, which arguably could be phrased better, you will see this

Of course I did read past the heading, but if the heading in the guideline says "should NOT" (emphasis on not is the guideline writer's, not mine), then listing request types after "do not typically need to be proxied" exception under such a heading doesn't respect said heading at all, and thus doesn't belong under that heading. You can't have it both ways.

In contrast, the essential components such as the TLS revocation list is fully proxied (the user's browser never hits Google servers for that).

With this I fully agree, Apple is doing similar things in Safari.

Extensions are also internal requests because they are downloaded and installed by the browser's update system, not the website.

With this I disagree. I don't initialte manually TLS revocation list update, nor I initiate manually adblock list updates, nor extension updates for that matter, but I do click the button on Chrome Store.

By default, the devtools will show you network traffic for the current website only.

Chrome Store is also a website but I don't see an XHR request in devtools to download the extension when I click the "Add to Brave" button. In my opinion, any user initiated action should be visible there even if the actual request is the result of Javascript code handling the click() event. Not showing the proxied request makes it impossible to diagnose the download error.

I'd recommend using a tool like Wireshark or mitmproxy.

As a developer myself I can certainly do that provided that I am expecting a proxied request (which in this case I didn't hence this request for clarification). But what chance a regular user has to troubleshoot connection or DNS issues with hidden requests?

To be clear, I am using Brave because of the privacy it offers compared to competition and I appreciate your explanation about your privacy policy.

However, here's a (admittedly very) hypothetical situation -- go-updater gets hacked by criminals and requests get redirected to serve malware extensions, or Brave gets served a FISA court order (along with a gag order), and is asked to redirect requests for popular extension to some 3-letter agency server. The scenario might sound far-fetched, but the troubling part in both cases is that the end users will have no clue where the final request for that CRX file has landed.

Once again, thanks for your time and for clarifying.

[fmarier]

Re-reading through this issue, I think there are two concrete actions we can take here:

1. Update the wiki page to remove the ambiguity.
2. Now that we no longer have of our list of bad extensions to block, extension downloads no longer have any reason to go through our server-side dispatcher. Instead, we could go direct to the Chrome store for these requests and never hit a Brave endpoint.

I will go ahead and update the wiki, let's use this issue to track that second point.

[levicki]

@fmarier Thanks for reconsidering, I agree on both points and I am looking forward to the updates.