more docs on rss

Author: tschellenbach

RSS.md

@@ -8,6 +8,13 @@ Post uniqueness in an RSS feed can be determined by 4 different methods:
 - The url of the first enclosure (common amongst podcasts)
 - A hash of the title, link, description and enclosures
 
+Note that the guid shouldn't change. The link, url and hash can change when the post is updated though.
+So using the first approach is preferable.
+
+While not available in the RSS feed you could also consider the
+
+- Canonical URL on the page
+
 ## RSS feed Uniqueness ##
 
 Determining the uniqueness for an RSS feed is harder.
@@ -29,3 +36,17 @@ Many sites have dropped support for this tag though. We could add special cases
 - Wordpress blogs
 
 As it's easy to determine the feed location
+
+## How Winds handles uniqueness ##
+
+For every feed Winds will evaluate which one of these fields are unique:
+
+['guid', 'link', 'enclosure[0].url', 'hash']
+
+Note that the hash is computed before any enrichment is done on the feed content.
+After that it stores the unique value in `article.fingerprint` in the format `guid:123` or `hash:123` etc.
+After selecting the best algorithm it will use a batch select and update to update the feed articles.
+
+The uniqueness of the last 20 articles is used to compute a hash for the RSS feed.
+We use this information to occasionally merge RSS feeds.
+After merging the alternative URLs are stored to prevent people from submitting the same feed under a different url.