Feature
Post

Category
Code


Mashing Up Feeds Using Yahoo Pipes

Yahoo Pipes may just be in beta, but it’s already a great tool for web publishers that wants to add a bit of more flair and life to their websites. It’s in no way hard to mash up RSS feeds using Yahoo Pipes, and I’ll walk you through the most basic way of using the service in this tutorial.

But first of all, why would you want to mash up several RSS feeds into one? Let’s do a list of reasons, shall we?!

  • You’ve got a bunch of sites you like and would like to monitor all in one. Mash them up!
  • You publish a handful of sites and wants your users to subscribe to them all. Mash them up and offer the feed, through Feedburner if you like!
  • You’d like to display relevant stories in your sidebar. Nab the feeds and mash them into one!

That latter one is exactly what we’ll be doing here on the next version of Devlounge, and something I’m adding to the new Wisdump design as I’m writing this.

Yahoo Pipes

Let’s Mash Those Feeds

It’s really easy. Go to http://pipes.yahoo.com, click Create a pipe, and sign in with your Yahoo ID (get one if you don’t have one). You get a big grid, a debugger, and stuff to the left for inclusion using drag and drop. Yikes!

Yahoo Pipes - the big grid

So let’s get started then. We want to mashup these feeds:

  • Devlounge’s RSS feed (http://feeds.feedburner.com/Devlounge)
  • The Blog Herald’s RSS feed (http://feeds.feedburner.com/blogherald)
  • Performancing’s RSS feed (http://feeds.feedburner.com/performancing)
  • Blogger Jobs’ RSS feed (http://feeds.feedburner.com/blogger-jobs)

We could add more, but that’ll do for now. I’ve collected the feeds as you can see.

Now, let’s add them. Grab the Fetch Feed badge under Sources in the left menu, and drag it top the big grid. A box gets added, and we also get a second box called Pipe Output. That’s where we want the feeds to land.

Elements in Yahoo Pipes

Let’s add the feeds. Paste the feed URL in the input field within the Fetch Feed box. Then click the + sign thrice beside the URL to get three more input fields. Add the RSS feed URL:s in these.

All these feeds are powered by Feedburner, as you can tell by looking at their URL:s, and the little icon to the left of them.

Yahoo Pipes - four feeds added

Good.

Now let’s sort them so that the most recent one is on top. Click Operators to the left, and drag the Sort field to the grid. For this we want to sort by item.pubDate (when the items in the mashed up feeds where published) so we’ll pick that from the first dropdown in the Sort box, and we want it in descending orders so we’ll select that in the second one.

Let’s make something off this! Click the circle in the bottom of the Fetch Feed box, and drag it to the top circle in the Sort box. Voilá, they’re connected! Now do the same from the bottom circle in the Sort box, to the Pipe Output box.

Attaching pipes

There we are. If you want to test it you can click the Refresh link in the debugger below the grid.

Now click Save in the top right, and name your pipe. When it has finished saving, click Run Pipe at the top of the screen. A new window opens, where you can do stuff with your pipe.

Looks good? Then click Publish to make it accessible. Now, what we want is the pipe’s RSS feed, so click the More options link to the right and Get as RSS.

There you go, one mashed RSS feed containing the latest from four different feeds!

Adding The Feed To Wisdump

So let’s add the feed to Wisdump’s sidebar then, shall we? The pipe’s feed is this: http://pipes.yahoo.com/pipes/pipe.run?_id=bntjQkH73BGSmXmSjknRlg&_render=rss

So how do we add it? With a RSS parsing plugin? We could, but since Wisdump is powered by WordPress, why not just use WordPress’ built-in feed parser? Let’s do that.

This little code snippet is well known, several blogs have reported variants of it. It’s snatched from the RSS feeds in WordPress’ admin.

<?php
require_once (ABSPATH . WPINC . '/rss-functions.php');
// insert the feed URL here
$rss = @fetch_rss('http://pipes.yahoo.com/pipes/pipe.run?_id=bntjQkH73BGSmXmSjknRlg&_render=rss');
if ( isset($rss->items) && 0 != count($rss->items) ) {
?>
<?php
// set the number of items from the feed to display (10)
$rss->items = array_slice($rss->items, 0, 10);
foreach ($rss->items as $item ) {
?>
<li>
<a href='<?php echo wp_filter_kses($item['link']); ?>'>
<?php echo wp_specialchars($item['title']); ?>
</a>
</li>
<?php } ?>
<?php } ?>

This will spit out a list where each item in the pipe’s feed is a li list item, so you’ll probably want to insert an ul around it.

The key is:

// insert the feed URL here
$rss = @fetch_rss('http://pipes.yahoo.com/pipes/pipe.run?_id=bntjQkH73BGSmXmSjknRlg&_render=rss');

…where you insert the feed’s URL, and:

<?php
// set the number of items from the feed to display (10)
$rss->items = array_slice($rss->items, 0, 10);
foreach ($rss->items as $item ) {
?>

The number of items you want to display. I went with 10 here.

Here’s the code used in Wisdump’s sidebar:

<ul class="network">
	<h2>News From The Network</h2>
	<?php
	require_once (ABSPATH . WPINC . '/rss-functions.php');
	// insert the feed URL here
	$rss = @fetch_rss('http://pipes.yahoo.com/pipes/pipe.run?_id=bntjQkH73BGSmXmSjknRlg&_render=rss');
	if ( isset($rss->items) && 0 != count($rss->items) ) {
	?>
	<?php
	// set the number of items from the feed to display (10)
	$rss->items = array_slice($rss->items, 0, 10);
	foreach ($rss->items as $item ) {
	?>
	<li>
	<a href='<?php echo wp_filter_kses($item['link']); ?>'>
	<?php echo wp_specialchars($item['title']); ?>
	</a>
	</li>
	<?php } ?>
	<?php } ?>
</ul>

You can see it in action over at Wisdump (down a bit in the sidebar), which incidentally has got a new design, so check that out too.

Pipes in action over at Wisdump

Good luck with your feed mashing!


  1. By Dan N. Moldovan posted on March 26, 2008 at 11:52 am
    Want an avatar? Get a gravatar! • You can link to this comment

    Nice Article.
    It seems that you like Yahoo Pipes. I managed to implement almost 30 Yahoo Pipes in my site MacrosReader (http://reader.macrostandard.com/). Just take a look and run a few of it! Thank you.

  2. By Scott Clark posted on March 26, 2008 at 1:07 pm
    Want an avatar? Get a gravatar! • You can link to this comment

    Excellent. Note that Geckotribe’s stuff is capable of this too, and with a lot more flexibility. Carp and Grouper, along with their new WordPress plugin can give you tons of control.

  3. By TDH posted on March 26, 2008 at 5:11 pm
    Want an avatar? Get a gravatar! • You can link to this comment

    Thanks.

    There are several services that could do this. I do like Yahoo Pipes though, it’s stable enough. :)

  4. By raj posted on March 27, 2008 at 10:33 am
    Want an avatar? Get a gravatar! • You can link to this comment

    Thord, I’ve written no fewer than 20 articles on Pipes, including video screencasts, and none of them looked as clear as yours. Good work, especially with that bit of code explanation.

  5. By TDH posted on March 28, 2008 at 4:41 am
    Want an avatar? Get a gravatar! • You can link to this comment

    Thanks Raj.

    I urge everyone interested in tweaking WordPress to check out Raj’s series on http://performancing.com

  6. By Sathish AR posted on March 31, 2008 at 4:19 am
    Want an avatar? Get a gravatar! • You can link to this comment

    Love this Post. Great tips! Surely will try this

  7. By Ken posted on April 17, 2008 at 11:58 am
    Want an avatar? Get a gravatar! • You can link to this comment

    Are you all having luck using the Yahoo Pipes RSS feed with fetch_rss? I sure am not :). I get “Warning: Invalid argument supplied for foreach()” about 75% of the time. Which, as you know, looks fantastic when it breaks my site, lol.

    I don’t have any other problems with any other feeds, just Pipes RSS output. I wonder what is special in the RSS that causes that?

  8. By Russell posted on June 13, 2008 at 9:48 am
    Want an avatar? Get a gravatar! • You can link to this comment

    Hi there,

    Thanks for the tutorial, really excellent and easy to understand. I’ve now used Pipes to aggregate a number of blogs that I publish into one feed.

    However, I’d like to extract the names of each post author from the original source feeds, and display these in the aggregated feed along with the post titles/excerpts as well – to make it clearer that each post in the Pipes feed is by different people. Anybody got any ideas how I can use Pipes to do this?

    Thanks

    Russell

  9. By Michael posted on September 30, 2008 at 8:46 pm
    Want an avatar? Get a gravatar! • You can link to this comment

    Thank you for this post, I used the code you provided on my own site. Some of the titles I’m bringing down are very long, but with php it was very easy to shorten them. If anyone is interested I did it this way…

    if (strlen($item2['title']) > 80)
    echo wp_specialchars(substr($item2['title'],0,80)) . “…”;
    else
    echo wp_specialchars($item2['title']);
    ?>

  10. By Michael posted on September 30, 2008 at 8:52 pm
    Want an avatar? Get a gravatar! • You can link to this comment

    Sorry about the double commment, I hit enter to quickly. I did have one question…

    Pipes stores the post creator in dc:creator. I have tried to show that on my blog using

    echo wp_specialchars($item2['dc']['creator'])

    Does anyone know why this wouldn’t work?

  11. By James posted on November 22, 2008 at 4:50 am
    Want an avatar? Get a gravatar! • You can link to this comment

    Hi,
    These pipes are looking good.

  12. By Mel posted on January 13, 2009 at 6:41 pm
    Want an avatar? Get a gravatar! • You can link to this comment

    is there a way to put the dates on the individual feeds?

  13. By Mel posted on January 13, 2009 at 6:42 pm
    Want an avatar? Get a gravatar! • You can link to this comment

    actually, the dates and the descriptions. thanks!

  14. By Carter posted on April 23, 2009 at 4:03 pm
    Want an avatar? Get a gravatar! • You can link to this comment

    I’m trying to add Pipes in my wordpress sidebar – not sure why this isn’t working.

    Why can I not add the Pipes RSS to the RSS Sidebar Widget in wordpress?

  15. TrackbackYahoo! PipesNotes from a Teacher: Mark on Media » Monday squibsMohd Huzairy : Blog Archive : Mashing Up Feeds Using Yahoo Pipes | DevloungeRetazos de la web del 2008-07-16 (microblogging) | hombrelobo, una mente dispersaIdeas for the website : Stephen’s Personal BlogWordPress Developer’s Toolbox | rafdesignWordPress Developer’s Toolbox - 山歌好比春江水逐浪·网络 » Blog Archive » WordPress 开发者工具箱- WordPress 开发者工具箱(上)【联讯网】WordPress Developer’s Toolbox | Fusuy.com || Webmaster Accessary Platformcssframework » WordPress 开发者工具箱【完整版】11 Non-Traditional Uses of WordPress135+ Ultimate Round-Up of Wordpress Tutorials | About Us | instantShift我想网 » Blog Archive » WordPress 开发者工具箱(上)250 Wordpress Tutorials135+ Ultimate Wordpress Tutorials11 Non-Traditional Uses of WordPress | brainstorming magazine | use the brain to get ideasWordPress Hacks: Build a Techmeme River of News Clone, Part 1 : PerformancingWordPress Hacks: Techmeme River of News Clone, Part 2 : Performancing250 Wordpress Tutoriels et liens pour tous | Guppytrucs Freeware11 Non-Traditional Uses of WordPress | Blog of Jon  WordPress Developer’s Toolbox  by Oshoamy