Articles showing dummy date of 12/31/1969 in "Last Update" column

[tonycpsu]

Describe the bug
Since the most recent update, some of my articles are showing dummy date of 12/31/1969 in "Last Update" column.

To Reproduce
Not sure what caused it, but it seemed to appear around when I upgraded to Version 3.9.3 :fd467d52: (8375)

I use Inoreader as my feed backend, if that is relevant.

Screenshots

Please complete the following information:

• Vienna version: Version 3.9.3 :fd467d52: (8375)
• OS version: 14.1.1 (23B81)

Additional information:

[tonycpsu]

Downgrading to 3.9.2 seems to fix this, FWIW.

[TAKeanice]

Thank you, we thought we had made sure that no default date of 0 (milliseconds after 1.1.1970 00:00 UTC) can appear anywhere. Can you tell us the feed source as well? Then we can make sure that we refine the feature to avoid that behavior.
Can you add the column for creation date and see what that shows in cases when updated date is 0?

[tonycpsu]

Feed source is Inoreader -- it seems to affect all of my feeds, not a single RSS source.

Here's what it looks like with 3.9.3 release with published date visible.

[TAKeanice]

Another anomaly here: Date published is later than last update
This is the Github feed for recent commits to vienna´s master branch (an atom feed)

[TAKeanice]

I was not able to find the source of such anomalies. According to the current code, if the feed source does not contain the last update, it is set to the publication date if that exists, or to the date when the article was received. For the publication date, if the feed does not contain it, the updated date (if it exists) or the current date is used, whatever is earlier.

@barijaona, @josh64x2 or @Eitot, maybe you can find a reason for the observed anomalies.

I am not sure about my anomaly, it could have been caused by using a development version of Vienna. @tonycpsu did you use any beta version?

[joostdekeijzer]

Not sure if it's related but using Version 3.9.3 :fd467d52: (8375) I get the date "01-01-1970, 01:00" for both published as updated dates. The source of the feed seems correct?

I think this only happens after a refresh when an article was already there and is updated?

[TAKeanice]

Not sure if it's related

It certainly is.

I get the date "01-01-1970, 01:00" for both published as updated dates. The source of the feed seems correct?

Looks like the source is correct, have to look up whether "pubDate" is taken into account though. The date is the same as in the other cases, "0" but in your time zone.

I think this only happens after a refresh when an article was already there and is updated?

That's a good hint and in my case, I had the same gut feeling but couldn't find the code causing that. I will try another time this weekend.

[TAKeanice]

I just realized that the RefreshManager is also manipulating the article dates, which may be causing effects that I have not considered so far.

[joostdekeijzer]

Hi TAKeanice,

Just to inform you: with Version 3.9.4 :498937af: (8382) I'm still getting reset dates...

From some WordPress blogs I'm following and eg. the example below from which I know it's an updated article.

[TAKeanice]

Hi @joostdekeijzer , thank you for adding that example. I suspect the RefreshManager, which I didn't fix in the last update, is the culprit. I hope to find some time this weekend to figure it out and create a fix.

[barijaona]

I just subscribed to https://exiftool.org/rss.xml and I do not reproduce the issue…

[joostdekeijzer]

@barijaona

I just subscribed to https://exiftool.org/rss.xml and I do not reproduce the issue…

The issue occurs when an article in the feed is updated by that site. That does not happen very often (and ExifTool is not a very busy feed anyway).

Some other feed I get the 1970 issue with just now (after refresh of article) is https://feeds.feedburner.com/9To5Mac-MacAllDay .

[catxeger]

Vienna 3.9.4 :49893af: (8382) - I'm seeing this issue regularly with what looks like it may be updated articles as noted above. The dates are sometimes 1969-12-31, and sometimes 1970 (don't have an example to hand). In earlier versions, the provided publication date seemed reasonably appropriate.

[brianallenlevine]

If it helps, I've also been encountering "12/31/69, 6:00 PM" dates since a Vienna update or two ago. They're recur regularly in certain feeds, including these two:

https://tidbits.com/feed/
https://blog.documentfoundation.org/feed/

I'm currently using Vienna Version 3.9.4 :498937af: (8382) on a current macOS Sonoma 14.7.1 (23H222).

I'll include a couple screen shots.

(If the screenshots don't appear properly after this comment is posted, I'll try to add them again in a different fashion.)

[TAKeanice]

I have done the change I was talking about a couple of weeks ago, it's just waiting to be released. I'd suggest retesting afterwards.

605dc64

[brianallenlevine]

Will do. Thanks, @TAKeanice!

[joostdekeijzer]

Sorry to have to report that with Version 3.9.5 :7784c797: (8414) I'm still experiencing the issue 😢

[brianallenlevine]

Same here! I upgraded to this same version this morning, but I'm still seeing some "12/31/69, 6:00 PM" dates where I was seeing them in the earlier version.

…

--Brian -- Brian A. Levine ***@***.*** https://www.brianallenlevine.name/

On Dec 11, 2024, at 9:31 AM, joost de keijzer ***@***.***> wrote: Sorry to have to report that with Version 3.9.5 :7784c797: (8414) I'm still experiencing the issue 😢 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[TAKeanice]

Can you @brianallenlevine and @joostdekeijzer tell me whether you're seeing this with the old articles that you already received before the upgrade or new ones that were loaded afterwards?

The last update didn't include a script for correction again, and I'm afraid some of the dates of older articles are not salvageable. There's no code any more that I know that would assign strange dates to new articles. That's why it's important for me to know whether the articles with wrong dates are old or new ones.

[brianallenlevine]

@TAKeanice, I just looked again at the tidbits and documentfoundation links that I posted screenshots of earlier. I still saw the "12/31/69, 6:00 PM" dates in both feeds, in at least some of the same old articles. After this, I unsubscrived and resubscribed both feeds. In the newly resubscribed feeds, I saw fewer articles, with only valid dates.

I do have additional feeds with these invalid dates. Is there perhaps a simpler way of accomplishing an unsubscribe/resubscribe without manually unsubscribing and resubscribing each feed with bad dates? I've looked, but I'm not seeing anything.

Maybe export all my subscriptions, unsubscribe them all, and then re-import would do the trick? I don't mind losing unread/read status and such, if I can get rid of the bad dates.

Thanks!

--Brian

[joostdekeijzer]

@TAKeanice I'm quite sure it's still happening on new articles. But I did not remove/resubscribe a feed.

See WordPress Core rss feed (Agenda for dec-11 article, published dec-10)

[brianallenlevine]

I'm again seeing some "12/31/69, 6:00PM" dates for "Last Update" and "Date Published" in the TidBITS feed I unsubscribed and resubscribed earlier.

…

-- Brian A. Levine ***@***.*** https://www.brianallenlevine.name/

On Dec 13, 2024, at 2:04 AM, joost de keijzer ***@***.***> wrote: @TAKeanice I'm quite sure it's still happening on new articles. But I did not remove/resubscribe a feed. See WordPress Core rss feed (Agenda for dec-11 article, published dec-10) image.png (view on web) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[TAKeanice]

Thank you @brianallenlevine . Means one more round of investigation, pulling out some more gray hair and hopefully finding the place that still manipulates the dates. Can you post a screenshot?

[brianallenlevine]

Here you go, @TAKeanice. Thanks so much for all your effort on this!

[joostdekeijzer]

It's hard to test, but I get the impression that this issue has a bigger change of happening on startup when you have "refresh on startup" enabled or when you press the "Refresh all your subscriptions" button.

[catxeger]

I'm still seeing the issue with Version 3.9.5 :7784c797: (8414), haven't hit 'refresh all your subscriptions', and it appears to be happening on newly retrieved articles (after 'mark all read' in a given feed).

Oddly, and visible in the screenshot, one of the articles has the wrong date -- 'Trudeau gave a speech to the Liberals' holiday party — but Freeland stole the show' is showing as '2024-06-23, 08:35', but a check of the website shows 'CBC News · Posted: Dec 18, 2024 3:26 PM EST | Last Updated: 8 hours ago' -- off by nearly 6 months!

[artcs]

I'm seeing this issue also on Version 3.9.5 :7784c797: (8414). AFAIKT it happens if an article which existed before is updated with a new published and updated date. Source has correct dates.

[barijaona]

I think I found the cause:

in

vienna-rss/Vienna/Sources/Database/Database.m

Line 1586 in 35bc17e

-(BOOL)updateArticle:(Article *)existingArticle ofFolder:(NSInteger)folderID withArticle:(Article *)articleUpdate

existingArticle may be an incomplete version originating from -minimalCacheForFolder:, and may therefore have neither lastUpdate nor publicationDate.

[TAKeanice]

existingArticle may be an incomplete version originating from -minimalCacheForFolder:

Oh man... I would never have searched in that direction... Let's make sure existingArticle is complete then!

[barijaona]

Minimalist approach regarding articles' caching makes completely sense when you may have plenty of feeds to update simultaneously, and you have to handle them in memory rather than on disk.

We just need to reproduce the date logic which exists in -addArticle:toFolder: to calculate the new last update date.

It would make sense to keep the existing publication date in database.

[TAKeanice]

I don't think that loading an article from the DB when an update is saved to the DB anyway is too much of an effort.

[barijaona]

It is a waste when you may have to handle tens of thousands of articles at a time during a "Refresh All" and you have the possibility to avoid this.

Look thoroughly at commit 9b4c126

[TAKeanice]

I'd rather not alter the last update date if we just don't know the original one because we got the ground truth from some cache. It may override semantically relevant information carried by that date. Assume a feed updates that date only for content changes, not for typo corrections - we would update the date anyway.

Maybe we can write a smarter update query that combines the query and the update into one, then no additional time is wasted by a second query.

[barijaona]

I don't really understand your concern.
If the feed provides a last update date, we just use it without questioning its semantics.
It's only when the feed does not provide an update date that we get back to exactly the same calculation that we used on initial publication.

[TAKeanice]

I am thinking about feeds that have a last updated date initially, but an update doesn't have it any more. From what I saw in the code, that's possible for atom feeds for example.

[barijaona]

Another edge case : Wired's RSS feed where they change the title of an article, push it with a new publication date, while the guid reveals that it is in fact an update of a preexisting article.

https://www.wired.com/feed/rss

[barijaona]

See the logic around https://github.com/barijaona/vienna-rss/blob/08572675ef5f38d69050efd8c72c82f4d3f331db/Vienna/Sources/Database/Database.m#L1619-L1656