Highlight sources with unseen source submissions

[eloquence]

User Story

As a journalist, I want to know at a glance which sources have unseen submissions, so that I can quickly identify new information that may require attention.

Justification

Seen/unseen status is a fundamental paradigm in messaging applications. The feature exists in the pre-workstation user experience (as a global status of messages and documents, internally referenced as is_downloaded, and shown in the Journalist Interface UI as bolded/unbolded). User research has validated the need for similar functionality in the SecureDrop Client.

Mockup

(Design specification to come)

Terminology:

Source list: the panel on the left in the client view, which shows a list of sources by codename (e.g., "benign artichoke")

Conversation view: the panel on the right, which shows the messages, documents and replies for the currently selected source

Submissions: Messages or files sent by a SecureDrop source

Unseen submissions: Messages or files that have not been displayed in the conversation view of the SecureDrop Client, or downloaded via the Journalist Interface, by any user

Acceptance Criteria

Basic seen/unseen status and synchronization

Given that there are sources with unseen submissions on the server
When the client retrieves these submissions as part of a sync
And those sources are not actively selected
Then these sources should be displayed as unseen in the source list

Given that there is a source with unseen submissions on the client
And that all submissions from this source are marked as seen on the server
When the client synchronizes with the server
Then the source should be displayed as seen in the source list

Explanation: The information on the server is canonical; the same user may be connecting from more than one workstation.

Selecting a source with unseen submissions

Given that there is a source with unseen submissions on the client
When I select that source in the source list
Then the source should immediately be displayed as seen in the source list
And the client should record all the submissions from that source as seen by the logged in user

Explanation: Recording a change in the status implies a server-side API request followed by a client-side database update.

Error case

Given that there is a source with unseen submissions on the client
When I shut down the server while the client is running
And I select that source
Then the source should immediately be displayed as seen in the source list
And the client should keep retrying pending server operations indefinitely
And a generic network error should be displayed to the user until network access is restored

Explanation: We optimistically update the UI after the user selects a source, and keep retrying indefinitely if there's a network problem.

Offline usage

Given that there are sources with unseen submissions in the local client database
When I use the client in offline mode
Then all sources should be displayed in a read-only state (TBD) in the list source list
And seen/unseen status should not be modified as a result of my actions

Deleted journalist

Given that there is a source whose submissions have all been seen by a single journalist
When I delete that journalist user through the Admin Interface
Then the source should still be displayed as seen in the source list on the client after the next sync

Replies

Given that I have selected a source with unseen submissions and changed its status to seen
When I send a reply to that source
Then the source should continue to be displayed as seen in the source list

Explanation: Replies do not impact the seen/unseen status in the source list

Given that there is a source with no unseen submissions on the client
When another journalist sends a reply to that source
Then the source should continue to be displayed as seen in the source list after a sync

Explanation: As above, replies do not impact the seen/unseen status in the source list

Journalist Interface

Given that there is a source with unseen submissions on the server
When I initiate a download of these submissions via the web-based Journalist Interface
Then the source should be displayed as seen in the source list on the client after the next sync

Explanation: Downloading via the Journalist Interface counts as "seeing" a message or file.

Updated per user research review & team discussions - 2020-08-26 - @eloquence

[ninavizz]

I've died and gone to Gherkin heaven!

[eloquence]

Just a note that we've tentatively agreed that we'll aim for tracking this on a per-user basis from the start, instead of using the existing global read/unread flag (this information would be stored separately and independently on the server). Will rewrite the issue accordingly & file a corresponding server-side issue.

Update: As part of this change, I've also changed the language to new/seen/unseen, to set it apart from the language we use in the web UI, and to foreshadow additional "Seen by" functionality.

[redshiftzero]

This issue should address @creviera's note on #240 regarding the following situation:

[Unseen Submission]
Depending on how large the journalist's reply is, the source submission that was sent before the reply might not appear in the conversation box when the journalist refreshes. It'll appear in the middle of the conversation somewhere and that could be outside the scope of the conversation box.

[eloquence]

@redshiftzero @creviera @ninavizz

This issue has been reworded significantly over the last couple of days. Allie, as you make progress on your sprint commitments, I suggest we schedule a chat to go over this.

This issue is becoming quite complex; as we get a handle around the server-side changes that are required, we may discover a logical way to split it.

Regarding the reply issue Allie flagged in her review of #240, if I understand correctly, the issue is that replying will potentially scroll previously received messages/documents out of view. If so, I think that's fine and intended behavior, consistent with other messaging applications. I believe it's important, however, that we scroll new activity into view when you select the source to reply to, and that's reflected in the acceptance criteria above.

This "scroll new activity into view" behavior would only occur the first time; if you then un-select and re-select the source, it would scroll to the bottom. That's consistent, e.g., with the behavior in Slack or other applications designed to make new activity easily identifiable.

We've not yet fully user-tested the scrolling behavior and the "seen" indicator for files, so those are good areas to consider for the next round of testing as well.

[sssoleileraaa]

@eloquence - yes, the issue I was referring to can be compared to responding to a Slack message inline by starting a thread - sometimes it can be challenging to know where in the conversation a new message was added so they have an All Threads tab to see any changes that you might have missed.

In the case of SecureDrop, this is less likely to happen since message sync should happen every 5 minutes (and there isn't a threads feature). But it could happen when a journalist sends a bunch of replies or a long reply that pushes a lot of conversation history into backscroll around the same time that a source is submitting message(s) or a document(s).

Scrolling to the first occurrence of new activity the first time a source is selected helps indicate to the journalist that they should read from that point on, but it would also be useful (post beta) to have some sort of way to indicate which new submissions came in from the server that are now outside the scroll view for the scenario where it's not the first time a source is selected.

As you mentioned, the "seen" indicator for files could potentially address this issue. And we might want to consider adding unseen messages to the indicator.

[eloquence]

But it could happen when a journalist sends a bunch of replies or a long reply that pushes a lot of conversation history into backscroll around the same time that a source is submitting message(s) or a document(s).

Intuitively, my sense is that users would commonly notice without an additional notification (most messengers "de-escalate" notification levels for conversations that are in focus) due to the keyboard<->screen eye movement that is common during message composition for people who are not touch typists, and the screen focus that is typical for people who are. (Down the road if we can develop for accessibility, we'd want to make sure that a real-time voice readout occurs for blind users.)

I fully agree it's worth carefully thinking about the different options of no notification vs. ephemeral notification vs. persistent notification for this case.

This also raises the important general point of how the client should behave when a message or document arrives from a source while the source is selected. Assuming we can determine whether the client window is actively selected or not, I believe the behavior should likely be different -- i.e. we need to do more to get the user's attention if the client window is not selected but the source is. Will flesh out a bit along those lines.

(Update: Updated the spec a bit more to account for "currently selected" status. Whether the client window is selected or not adds significant complexity and I'm okay keeping that out of scope for now, assuming that the client is primarily used as a foreground application or not at all. But we should probably revisit that in an iteration.)

[ninavizz]

So: I want to be very, very careful with patterns we pick-up from other apps... and to inject a cautionary segue here, as I think it's very relevant to this discussion.

GMail and Slack are two app suites that target a broad enterprise audience. Unlike Microsoft, those teams are not led by MBA/Sales, but by UX-favoring product teams. So: while Microsoft's UX research efforts are very well funded and also target their broad enterprise base... Microsoft product teams have a ways to go, still, with embracing a "user first" approach in bridging the gap between research and what gets shipped. Microsoft products ship to meet the desire of their direct customers (IT executives, not the actual users) whereas Slack and Google "get" and embrace who actual users (and what their needs) are.

Messaging apps that are limited to the early-adopter crowd, hackers, techies, and security/privacy enthusiasts, are often not influenced by much user research. Signal has had an open "UX Lead" role on their Jobs board for almost 2 years now—so I doubt its product is influenced much from what is actually usable, vs what the developers chose to do. Discourse, I don't know much about... but the fact that it's a FOSS product whose team is mostly white dudes, and is a product I've only ever seen used for techie forums, has me doubting much research informs its shaping.

Slack, on the other hand, was created as very much the "early adopter crowd" app, but is now far more heavily influenced by usability research in what goes to market. Their micro-interactions are all very carefully studied, and I trust what both them and GMail put into their products, when in doubt, purely because of how I understand their research and product team dynamics (and investments in user research) to be. FYI.