Add AdGuard's "$removeparam" as an alias for "$queryprune"

[DandelionSprout]

Prerequisites

• I verified that this is not a filter issue
  • Filter issues MUST be reported at filter issue tracker
• This is not a support issue or a question
  • Support issues and questions are handled at /r/uBlockOrigin
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
  • Your issue may already be reported.
• I tried to reproduce the issue when...
  • uBlock Origin is the only extension
  • uBlock Origin with default lists/settings
  • using a new, unmodified browser profile
• I am running the latest version of uBlock Origin
• I checked the documentation to understand that the issue I report is not a normal behavior

Description

I've been working on https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt for some weeks now, which I did while presuming that only AdGuard had a feature to remove $/& parameters from URLs. But during some syntax checking of regional lists, I discovered that $queryprune seems to have been added to uBO.

However, those two $ qualifiers seem to me to do the exact same thing, with only 2 current exceptions:

1. AdGuard removes the parameters at the network level instead of in the URL bar
2. $removeparam does not seem to need | in its values.

I therefore propose that they should be made into aliases of each other. This would save me from having to maintain and sync two different versions of that list, one for uBO and one for AdGuard.

A specific URL where the issue occurs

https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt

Steps to Reproduce

1. Turn on uBO's logger
2. Set the tab to "All"
3. Attempt to add or sync https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt
4. See that the logger considers all its entries to be an Invalid network filter in https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt:.

Expected behavior:

$removeparam is treated as being $queryprune

Actual behavior:

$removeparam does not work in uBO.

Your environment

• uBlock Origin version: 1.31.1b2
• Browser Name and version: Chrome 87.0.4280.66 x64
• Operating System and version: Windows 10 October 2020 Update

[gorhill]

AdGuard removes the parameters at the network level instead of in the URL bar

This is incorrect. Where did you get the idea that uBO removes it only "in the URL bar"?

[DandelionSprout]

I took a guess based on how https://chrome.google.com/webstore/detail/redirector/ocgpenflpmgnfapjedencafcfakcekcd primarily does it, as I presumed that was how browser extensions preferred doing it.

I apologise for my mistake.

[gorhill]

You took a guess based of how uBO works on a completely unrelated extension somewhere in the Chrome store, while not coming to the same guess-based conclusion at how AdGuard work? How did you discovered that uBO now supported queryprune= without going through the related thread which has plenty of information about it?

In any case, the main difference between AdGuard's removeparam= and uBO's queryprune= is that in uBO the queryprune= values are always literal regex, except that the special anchoring characters ^ and $ have to be replaced with | so as to avoid parsing ambiguity when using $ since it is also used as pattern-option separator.

I could align uBO's behavior to removeparam= one by internally treating plain alphanum value as |...=, and when the filter does not have a pattern (i.e. equivalent to match-all *), to use the alphanum value as the pattern.

[DandelionSprout]

while not coming to the same guess-based conclusion at how AdGuard work?

AdguardTeam/CoreLibs#1372 (comment)

How did you discovered that uBO now supported queryprune= without going through the related thread which has plenty of information about it?

On a whim I decided to use the uBO logger to look through invalid entries in regional lists. For this specific test session I used Vivaldi, whose uBO version apparently hadn't been updated for some weeks and was still on 1.30.6, thus the logger claimed that $queryprune in Frellwit's Swedish Filter was an invalid network entry. And that's how I learned about it.

https://kb.adguard.com/en/general/how-to-create-your-own-ad-filters#removeparam-modifier

gorhill/uBlock@bde3164

Fine tuning queryprune=

The following changes have been implemented:

The special value * (i.e. queryprune=*) means "remove all query
parameters".

If the queryprune= value is made only of alphanumeric characters
(including _), the value will be internally converted to regex equivalent
^value=. This ensures a better future compatibility with AdGuard's
removeparam=.

If the queryprune= value starts with !, the test will be inverted. This
can be used to remove all query parameters EXCEPT those who match the
specified value.

[gorhill]

I am reopening because the requirement to use | as alias for regex ^/$ has been nagging me.

Since queryprune= is not yet officially documented, it's ok to keep settling on the final syntax. I've given more thoughts and I will go along with being fully compatible with AdGuard's removeparam= syntax, i.e. the value will no longer be implicitly treated as a regex and will require bracing with / to explicitly treat the value as a regex. i.e. queryprune=/.../. I will modify the parser to deal with the potential $ ambiguity.

One other ambiguity which can still arise is the use of comma , or literal dollar sign $ inside the regex, and in such case filter authors will have to use the \x.. form of the character, i.e. \x2C for comma and \x24 for $.

The only thing I want which is not the way removeparam= is currently implemented, is that when a regex value is specified, then the match will be tested against name=value instead of the name of the query parameter, so this way this allows to filter according to the value of a parameter. When the value is not regex based, then it will be treated just as AdGuard specifies it, i.e. it's a straightforward comparison with the parameter name.

---

Something else, consider the following filter:

/ad-$queryprune=/^ss$/

There is an ambiguity in there, this can be either:

• the plain pattern /ad- with the queryprune= option
• the regex-based pattern /ad-$queryprune=/^ss$/ with no option

To resolve that sort of ambiguity, I added the noop filter option _: this does nothing and its purpose is to disambiguate such instance:

/ad-$queryprune=/^ss$/,_

Then it becomes the first case, i.e. the plain pattern /ad- with the queryprune= option

---

cc @ameshkov