Archiving social media

Social media is instant and ever changing, users post, others comment and content gets pushed further and further down the feed, a challenge that must be overcome for users to fully exploit the potential of any medium. Each social media site archives in its own way: Facebook has all posts going back but the complicated privacy settings means that only certain users can see certain content. Twitter’s two settings (public and private) and the intimidate nature of the network make it prime for RSS archiving.

Twitter is characterized by a constant stream of information, and each new updates pushes each previous update further down the list making it harder to find. Constant updating, combined with Twitter not enabling search for posts more than a few days old requires RSS or another form of indexing to make the site a viable tool for reporting and researching.

RSS for Twitter users

Until a few months ago, Twitter natively supported RSS for public accounts. For users this facilitated following without joining Twitter but also allowed Twitter users to see a user’s history in an archived, searchable manner beyond a few days.

Following a user in RSS is still possible but more complicated. RSS feeds for users created by Twitter all follow a single formula:

http://twitter.com/statuses/user_timeline/USER-ID-NUMBER.rss where the “USER-ID-NUMBER” is a unique identifier assigned by twitter to each user.

Creating an RSS feed for a twitter user requires finding the ID and inserting it into the formula. One site, ID from User is a simple online application that only finds the user ID number. After entering “georgetowncct” for the Communication, Culture, and Technology program’s twitter account the program ID from User provides the resulting user ID of “57067733.”

The second step in this process is inserting the ID into the formula and then adding the finished URL to a reader which is done by copy and pasting the finished URL (http://twitter.com/statuses/user_timeline/57067733.rss) into the address bar of any browser.

Why RSS

For a reporter who knows in advance who to follow RSS provides an irreplaceable tool since the result is a searchable timeline of all tweets. Reporters can follow a user without making that information public and can then use the information in the feed to see a user’s complete Twitter history including deleted content. It also allows reporters to keep a full, searchable history of their own Twitter activities.

2,014 Responses to "Archiving social media"

  • Steve Buttry says:
  • asKrvuwhck says:
  • Sid Mumm says:
  • Linux VPS says:
  • asKruneyzy says:
  • Deejay says:
http://smr.joshuaialtman.com/?p=252
smr.joshuaialtman.com
crawl=CC-MAIN-2019-13
1048920
application/http; msgtype=response
sha1:PXZCXVDQXLT3AR6UOEW5PD5UNKRJJ7HP
<urn:uuid:d3ba7cee-7284-4fb3-838c-5cbc042ee94a>
2019-03-26T18:49:03Z
23.229.191.142
application/xhtml+xml
sha1:5X24XFJKVUW6DTZUS4ZZ6F2H53I77IRR
<urn:uuid:5598791c-dd26-4ab3-9395-acd6ddc4866b>
http://smr.joshuaialtman.com/?p=252
response
<urn:uuid:3921211b-f012-4d7b-b8c3-090f2f8db5db>
isPartOf: CC-MAIN-2019-13
publisher: Common Crawl
description: Wide crawl of the web for March 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-145-45-50.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 0.11-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/