Trim Down Your RSS Reading List By Removing Duplicate Feeds

If you have been following RSS feeds for a long period of time, there may be duplicate feeds present in your news aggregator that have different URLs but point to the same content on the web.

May 16, 2008

RSS FeedIf you have been following RSS feeds for a long period of time, there may be duplicate feeds present in your news aggregator that have different URLs but point to the same content on the web.

This can happen due to various reasons like:

1. Website xyz.com originally had a feed at xyz.com/feed but later moved to xyz.com/rss without redirecting the previous one. You may be subscribed to both feeds.

2. Your news reader is not smart enough to recognize duplicate feeds and doesn’t throw a warning when you try adding a feed that already exists in your subscription list.

How to Detect and Remove Duplicate RSS Feeds ? (and Reduce RSS Stress)

Whatever be the reason, it is relatively easy to remove duplicate RSS feeds from your subscriptions.

The first step is to export your entire feed subscription list as an OPML file. They are plain text files and you can view the contents in any text editor including Notepad.

The next step is to sort the content of this file. You can use the sort command in DOS / Unix or just import this OPML file into Google Docs / Microsoft Excel and use the built-in Sort feature.

duplicate rss feeds

If your reading list is small, simple scroll through the list to detect any duplicate subscriptions – if two or more feeds share the same values in text, title and htmlURL fields, that means they are duplicates.  Make a note of such feeds and unsubscribe from them in your RSS Reader.

If you reading list has hundreds of RSS feeds, you may like to explore Excel Macros or Advanced filters that can detect duplicate records in spreadsheets.

Related: How to Embed RSS Feeds into HTML Web Pages
https://www.labnol.org/software/organize/trim-rss-subscriptions-detect-remove-duplicate-feeds/3316/
www.labnol.org
crawl=CC-MAIN-2017-17
74615
application/http; msgtype=response
sha1:26M22HVEX25DE5EBQAGLHDXKZE3UUF7T
<urn:uuid:d54cd690-ec29-4a80-b2a6-892ed704de0d>
2017-05-01T02:27:14Z
66.33.199.148
sha1:CGO2IXXGI7DLAC3WQPO6EOG2BXZY2OKK
<urn:uuid:36870b59-3488-468f-9914-77d978270e03>
https://www.labnol.org/software/organize/trim-rss-subscriptions-detect-remove-duplicate-feeds/3316/
length
response
<urn:uuid:2376ea9a-b4cb-4c00-8265-0b3712007613>
robots: classic
hostname: ip-10-145-167-34.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2017-17
operator: CommonCrawl Admin
description: Wide crawl of the web for April 2017
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf