Bug : bookmarks of sites requiring authentication are showing authentication windows when opening extension

[syldub69]

When I open Bookmark Search Plus 2 in the sidebar, it loads during a long time (I have 3300 bookmarks), and it shows many authentication windows like this one:

The corresponding bookmark is: http://sql.free.fr/phpMyAdmin/
This behavior is really boring, because each time I open the extension, not only is is very long to open, but it has also to show 15 ou 20 authentication windows (I click on "Cancel" for each one).
It seems each bookmarks is trying to be opened by the extension upon loading.
Could you please disable this checking?

[aaFn]

Hello @syldub69 , thank you for this.
So there are two topics in this issue:

1. Long loading = indeed, depending on the number of bookmarks you have and the core speed of your system, also if you are on SSD or disk where the FF bookmark DB is installed .. etc .., this can get long. Fyi, my nearly 1900 bookmarks load in 0.7s on an i7 4500 MHz, and @vitaaliy has nearly 80000 bookmarks, and this takes 80s. I made a VM with a little less than 80000 bookmarks, and it takes 825s to load just in FF API !! (I can't do anything about it :-( ) .. see On elaboration of Bookmark-search-plus-2 #5
   As you will read there, I am currently stuck because of FF bugs, see On elaboration of Bookmark-search-plus-2 #5 again and Optimising loading #2 . I am still thinking about ways to get around this, but this is not going to happen so soon then, I am sorry for that.

2. Password prompts = that is because the add-on is fetching the favicons in background, to get the nice images at front of the bookmarks. This is needed because FF doesn't make favicons available in the bookmarks API :-( https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/bookmarks/BookmarkTreeNode
   Some of the sites are asking for password when fetching their bookmarked contents, and I can't control that or I didn't find a way so far, FF is then itself prompting for the password, not the add-on.
   For now, this is a necessary evil, but this is not at each load, this is a temporary period. If you look carefully inside the tree, the bookmarks are initially created with a temporary icon (a blue "T" at bottom right), and then when the add-on has passed over it in background, it is replaced by either the true favicon, or by a "nofavicon" image.
   Then the add-on saves the image so that on next load it is not fetched again, and will not come back on it ever, unless you right click and click on the "Refresh favicon" action .. for that specific bookmark only.

On 2):

• I will see if I can find a way to not have those popups again, and will let you know.
• Another way would be to add an option for people to disable favicon fetching .. not nice.
• The third option is to go to Google www.google.com/s2/favicons?domain_url=xxx, but as I wrote on the add-on description page, I do not want to do that so far for privacy issues ... now I guess I could leave the choice to the user by another option, warning him/her about the privacy exposure.

There are bugs open on FF to let them give us the favicons from the FF DB through the API, which will avoid me to re-fetch and re-store them in the add-on .. but these bugs are very old, I am not optimistic it will come soon .. :-(
Anyway, rest assured that as soon as they make that available, I will use the API and discard that piece of code.

So let me know how you would want to go about that, given the above options, or what you wuld think of.

Or if you are ready to bear with the popups until the last favicon is obtained by the add-on .. just let it run for some time, and when the last bookmark is done (the most bottom one ..), this boring thing will not happen again.

[vitaaliy]

I have just stumbled upon this info: https://blog.mozilla.org/addons/2018/03/05/updates-add-review-policies/ - maybe it could be of interest to you, aaFn.

[aaFn]

Hello @vitaaliy , yes when submitting the add-on, we are asked to read those. I believe BSP2 fully complies with those policies, or I tried to :-)
Let me know if you see I missed something.

[aaFn]

Hello @syldub69 , I believe I found a way to avoid the prompts.

However, in some cases it can lower the chances to get the favicon.
Indeed, authentication can be based on many methods. One of them is user/pwd (the prompts you are getting), another is based on cookies, ... etc ..

The solution I found so far to avoid the prompts is to disable authentication in the request. That means any form of authentication .. I didn't find a way to be selective. So in case cookies stored in the browser are used to authenticate the request to get the favicon, they won't be sent and the request will fail, resulting in no favicon.

However, I believe this is much less of an inconvenient than getting the prompts, as you highlight.
Therefore I am setting by default the mode to "omit" to disable authentication, and will add an option to enable the mode allowing to to use cookies (= "include"), warning about the side effect of getting also user/pwd prompts.

Let me know if that is better suiting your taste.
That should be in 2.0.8.

[syldub69]

Hi,
First of all, thanks for your very detailed answers!
You were right, my sidebar is now showing really quicker, as the images are cached.
And now it does not show authentication windows anymore.

I think the solution you propose is good: being able to disable authentication in the request. I don't mind if I don't have favicon for these few bookmarks. And if I want it later, I could "force" the favicon retrieval using the manual "Refresh favicon" action (I mean ideally, if you could disable authentication in the request when the bookmarks are automatically processed, but enable authentication in the request when clicking on "Refresh favicon")

Maybe the global options could be named like this: "Enable favicon retrieval for bookmarks having an url needing authentication".
Or something like that :o)

[aaFn]

Ok, done deal.

New option is:

[ ] Enable cookies to increase background favicon retrieval success rate. Drawback: prompts for user/pwd on some URLs

And yes, easy to have it always with authentication when clicking "Refresh favicon", the mechanism I coded was planned for such things, so this is in. Good idea, thanks.

In 2.0.8 (coming soon).

[aaFn]

Closed by 2.0.8