Feature request: "Mark all read" for filtered view

[gumbo2k]

First of all: Thank you for providing such a great tool. 🙇‍♂ It is a pleasure to work with mailpit!

I love the "Delete all" function for filtered view. It helps to partially emulate having multiple mailboxes by using auto tagging.

There is however a feature that I am missing. When changing from "full" inbox view

to "filtered" view (e.g. by clicking on a tag)

the "Mark all read" button disappears.

I haven't checked the sources yet, but would it be hard to make the "Mark all read" feature available for the filtered view?

[axllent]

Thanks @gumbo2k, I'm glad to hear Mailpit is working well for you 👍

That's a great question! As you've observed, Mailpit currently provides the "mark all as read" button only in the Inbox view, and only when there are any unread messages present (else the button is disabled). To give you some context, viewing a tag functions similarly to performing a search, meaning the same logic applies to both. There are two main reasons for this limitation:

1. Performance: Mailpit only retrieves the total number of unread messages through the API. Calculating the number of unread messages for each search or tag would be very CPU-intensive and slow.
2. Live-loaded messages: Mailpit received live notifications as new messages arrive in the Inbox, however this capability does not extend to tags or searches, which necessitate a refresh to update the view. Imagine there are 100 messages per second coming into Mailpit - it's just too much variable data to account for in the web UI if we were also trying to track every tag etc.

So looking at potential solutions:

1. We could have a permanent "Mark all read" (enabled) button in a filtered view, however I strongly feel this would be very misleading because the UI would not know if the search actually contained any unread messages at all, and thus would lead to inconsistency between how the inbox view and the filtered view work, and confusion as to why there appears to be unread messages in the filtered view (ie: I have 137k messages in my Mailpit). You could just keep clicking "mark all as read" and not see any change but assume you still have unread messages because Mailpit hasn't disabled the button.
2. In order to return the number of unread messages matching a search (at the time of search), an additional heavy SQL query would need to be run in addition to the the existing queries. This would (approximately) double the execution time for the search, so result in a bad user experience. Filtering by just tags is relatively quick compared to a fulltext search, however as I pointed out earlier, the filtering/search system is directly linked (for instance it is also possible to filter by multiple tags at once, or a tag plus fulltext search etc).

would it be hard to make the "Mark all read" feature available for the filtered view?

Basically yes, I don't think it's possible to implement this without either introducing a significant SQL/CPU overhead to every query (slowing down Mailpit), or introducing inconsistencies (and confusion) between how the Inbox button works vs: the filtered view.

On that note, I'm not sure if you are aware but it is possible to select multiple messages at the same time (using Shift-click or Ctrl-click) where you can process all selected messages at the same time? This may already be a working solution for you, unless of course you have multiple pages of results.

[gumbo2k]

Hi Ralph,
thanks for the quick and in-depth reply. 👍

..., Mailpit currently provides the "mark all as read" button only in the Inbox view, and only when there are any unread messages present (else the button is disabled).

Oh, I wasn't aware of that second condition. I thought that button was there (and enabled), regardless of the state of the inbox.

Performance: Mailpit only retrieves the total number of unread messages through the API. Calculating the number of unread messages for each search or tag would be very CPU-intensive and slow.

Then the question becomes a different one... would it be hard to extend the api, so that a search would reply not only the total number of matching messages, but also the number of unread ones.

Who exactly performs that search? The mailpit server, or the storage backend (inside the server or remote)?

If the search happens in the server, then I think it wouldn't be that much more load on the server, if it checked one more binary field and updated one more counter for the search result.

If the search happens on the SQL layer? Then the solution is possibly even easier.
Assuming there is a SELECT COUNT(*) ... statement running when a search is done (the gui shows the number of matching messages in the top right for pagination purposes), that COUNT(*) ... could be augmented with an additional COUNT (CASE:

SELECT COUNT(*) as result_total, COUNT(CASE WHEN state='unread' THEN 1 END) as result_unread FROM ... WHERE <filter> ...

Disclaimer: I have not checked if sqlite / rqlite supports that kind of conditional COUNT expression and I don't know how much of a performance penalty it introduces, if it works. I doubt however, that it would double the load the way two separate searches would.

Live-loaded messages: Mailpit received live notifications as new messages arrive in the Inbox, however this capability does not extend to tags or searches, which necessitate a refresh to update the view. Imagine there are 100 messages per second coming into Mailpit - it's just too much variable data to account for in the web UI if we were also trying to track every tag etc.

True. I wouldn't ever expect to see an unread counter on each tag in the in inbox view. I am already very pleased by new tags showing up in the inbox view without any reload. 😀

So looking at potential solutions:

1. We could have a permanent "Mark all read" (enabled) button in a filtered view, however I strongly feel this would be very misleading because the UI would not know if the search actually contained any unread messages at all, and thus would lead to inconsistency between how the inbox view and the filtered view work, and confusion as to why there appears to be unread messages in the filtered view (ie: I have 137k messages in my Mailpit). You could just keep clicking "mark all as read" and not see any change but assume you still have unread messages because Mailpit hasn't disabled the button.

True. It would get worse, if we added an unconditional "Mark all read" button in the search view.
We could enable the "Mark all read" button in the search result view, based on the state at the time the search was run, if we had that information from the extended COUNT(CASE statement. That would be consistent with the inbox view, wouldn't it?

Currently the search view doesn't update when new messages arrive. So this is an inconsistency that already exists.

Regardless of that, maybe there is a compromise between consistency and performance/load here: a delayed conditional reload of searches:

The client UI already gets the information that there were changes on the server side, through the websocket connection, but not, if those changes affect the current search view, correct? Maybe the user experience here could be improved by adding a switch next to the search field for a delayed-reload feature. If this feature is enabled, then a change notification from the server schedules a delayed reload of the search on the client. There would only ever be at most one scheduled reload. If no new messages arrive, no reload is scheduled. If 100 messages arrive in short sequence, the first one schedules a search-reload in 5 seconds. The other 99 don't do anything because a reload is already scheduled. After the search-reload is done the client return to idle mode.

2. In order to return the number of unread messages matching a search (at the time of search), an additional heavy SQL query would need to be run in addition to the the existing queries. This would (approximately) double the execution time for the search, so result in a bad user experience. Filtering by just tags is relatively quick compared to a fulltext search, however as I pointed out earlier, the filtering/search system is directly linked (for instance it is also possible to filter by multiple tags at once, or a tag plus fulltext search etc).

would it be hard to make the "Mark all read" feature available for the filtered view?

Basically yes, I don't think it's possible to implement this without either introducing a significant SQL/CPU overhead to every query (slowing down Mailpit), or introducing inconsistencies (and confusion) between how the Inbox button works vs: the filtered view.

I agree. If it isn't possible to cheaply add that unread-counter, then the feature is not worth the problems it may cause.

Independent of that, a switch for delayed-reload might be a workaround for some users to allow updating of the search view,

On that note, I'm not sure if you are aware but it is possible to select multiple messages at the same time (using Shift-click or Ctrl-click) where you can process all selected messages at the same time? This may already be a working solution for you, unless of course you have multiple pages of results.

Yeap. I am aware of that feature and I like it a lot 😁 I do have however have cases where a lot of pages might come up.

[axllent]

Leave this with me. If I can add an unread counter for a filtered view (without significant overhead) then I should have what I need to add a "mark all read" button. It's not the same problem as live loading messages though, which I don't think is feasible. No promises at this stage, but I'll definitely look into it.

[axllent]

@gumbo2k I can confirm there is very minimal overhead needed to calculate the unread total unread messages within a search, so I have added this feature to the web UI. I haven't released it yet, however the new functionality is included in the latest axllent/mailpit:edge Docker image. Are you by any chance using Docker, and if so, could you please test the edge build and let me know how this works for you? Thanks.

[gumbo2k]

Great! I have deployed the edge version in my staging environment and it works like a charm! Thank you! 🥳

[axllent]

Awesome, thank you for the feedback. It'll probably still take a few days for a new release, but this will be included either way. I'll leave this open until the release 👍

[axllent]

This feature has now been released in v1.24.1 👍