Aggregating news from community contributors or groups

There is ever growing interest in ways for organizations to aggregate news based on the input of multiple people, then select and push that news feed through their site. The good news is, there are some very slick methods for doing just this. As an example, Jeff C. on ProgressiveExchange, posted what he needed to which I responded and am cross-posting here.

I have been working on a system whereby members of DAP can find an article, either on the internet or in a Google News Alert, which many of us receive on a daily basis, decide it is something that needs to be shared with others, click a button, and have the article loaded to a blog-like web page…  One last thing:  In the future, everything that is posted to this page will then be automatically fed to a page in ************.org, so the page must have the ability to have an RSS feed.

From previous posts, readers know we’ve had a lot of success with Yahoo!Pipes for aggregating content from across the Web using keywords/phrases in a fully automated or review-based process. A few of our clients are also using del.icio.us accounts – one for each tagger, then aggregating those individual RSS feeds through either a master account or through Yahoo!Pipes. The final step it sounds like you want is to pull that feed onto your site. There’s potential rights issues will pulling the content into a CMS itself, but there are workarounds to display on your site.

Here’s a snapshot of how we’ve made this work:ither have one master, or give out unique Del.icio.us accounts to everyone who is gathering the data. These Del.icio.us accounts should be unique to this aggregator or at least tags should contain a unique keyword (so that personal likes don’t wind up in their feed). The entire Delicious account or just those tags with unique keyword can be turned into an RSS feed.

  1. If there are multiple del.icio.us accounts, aggregate the feed using Yahoo Pipes, which could easily be turned into an RSS feed. Take a look at http://www.iue-cwa.org/ in the Recommended Reading, this is culled from delicious tags of IUE-CWA staff across multiple accounts.
  2. Depending on the technology available to you, you could import this content onto your site. If PHP is available to you, you could use MagPie (which we used for Conservation International’s Great Sea Turtle Race to aggregate race updates). That’s the best possible method. If you don’t have PHP, you could use any number of Javascript libraries (We used Spry for IUE-CWA to pull Pipes content into their homepage, take a look at http://www.iue-cwa.org/ homepage – the labor & industry news is a Yahoo!Pipe feed).

Another thing to look into is feeding the point source RSS content into a NetVibes community. We’re just looking into this and it seems to have some promise.

If you’ve got other ideas or approaches, share them in comments!

2 Responses to “Aggregating news from community contributors or groups”

  1. Olga Howard Says:

    This is a great article Michael. I just did some research on this as well and found that some don’t work as well as others. On my site’s homepage, olgahow.com (bottom, right), I’m using Splush. It’s pulling from three sources and presenting them to me by date rather than by source. I really like it.

  2. Tim Says:

    I was just subscribing to some more podcasts from NPR yesterday and they now have added a custom feed builder as well. You can add specific shows as well as keywords that you want be used to build your feed, give it a name, and copy the unique URL to use in your feedreader or to publish on your site.

    NPR Podcast Directory: http://rubyurl.com/5U9e

    You can also use their API Generator to build more sophisticated queries with sorting and more. Here’s the help info on how to do that: http://rubyurl.com/MpR1 It looks slightly more complex, but it’s certainly nothing our readers couldn’t handle.

    Once you’re done, there’s no reason you couldn’t take that custom feed and drop it into Splush or Yahoo Pipes to add to your other feeds for use as you discussed.
http://www.beaconfire.com/blog/2009/02/aggregating-news-from-community-contributors-or-groups/
www.beaconfire.com
crawl=CC-MAIN-2015-27
20116
application/http; msgtype=response
sha1:LNXASKSGJOSJ2YGGMIF72ZDRR6X6XKG6
<urn:uuid:9b71949c-080d-43e1-8653-f92d3b75012e>
2015-07-02T05:17:21Z
173.255.232.200
sha1:Z2GVZDYCFVFYABBDBJBQRP5MIB7NFUWO
<urn:uuid:57ac6cdc-1b91-4393-bcd5-a0e6d81f453c>
http://www.beaconfire.com/blog/2009/02/aggregating-news-from-community-contributors-or-groups/
response
<urn:uuid:2b3322c0-76b4-4438-aa0e-b03e3dd97957>
robots: classic
hostname: ip-10-179-60-89.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2015-27
operator: CommonCrawl Admin
description: Wide crawl of the web for June 2015
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf