Mashing Up Feeds Using Yahoo Pipes
Yahoo Pipes may just be in beta, but it's already a great tool for web publishers that wants to add a bit of more flair and life to their websites. It's in no way hard to mash up RSS feeds using Yahoo Pipes, and I'll walk you through the most basic way of using the service in this tutorial.
But first of all, why would you want to mash up several RSS feeds into one? Let's do a list of reasons, shall we?!
- You've got a bunch of sites you like and would like to monitor all in one. Mash them up!
- You publish a handful of sites and wants your users to subscribe to them all. Mash them up and offer the feed, through Feedburner if you like!
- You'd like to display relevant stories in your sidebar. Nab the feeds and mash them into one!
That latter one is exactly what we'll be doing here on the next version of Devlounge, and something I'm adding to the new Wisdump design as I'm writing this.
Let's Mash Those Feeds
It's really easy. Go to http://pipes.yahoo.com, click Create a pipe, and sign in with your Yahoo ID (get one if you don't have one). You get a big grid, a debugger, and stuff to the left for inclusion using drag and drop. Yikes!
So let's get started then. We want to mashup these feeds:
- Devlounge's RSS feed (http://feeds.feedburner.com/Devlounge)
- The Blog Herald's RSS feed (http://feeds.feedburner.com/blogherald)
- Performancing's RSS feed (http://feeds.feedburner.com/performancing)
- Blogger Jobs' RSS feed (http://feeds.feedburner.com/blogger-jobs)
We could add more, but that'll do for now. I've collected the feeds as you can see.
Now, let's add them. Grab the Fetch Feed badge under Sources in the left menu, and drag it top the big grid. A box gets added, and we also get a second box called Pipe Output. That's where we want the feeds to land.
Let's add the feeds. Paste the feed URL in the input field within the Fetch Feed box. Then click the + sign thrice beside the URL to get three more input fields. Add the RSS feed URL:s in these.
All these feeds are powered by Feedburner, as you can tell by looking at their URL:s, and the little icon to the left of them.
Good.
Now let's sort them so that the most recent one is on top. Click Operators to the left, and drag the Sort field to the grid. For this we want to sort by item.pubDate (when the items in the mashed up feeds where published) so we'll pick that from the first dropdown in the Sort box, and we want it in descending orders so we'll select that in the second one.
Let's make something off this! Click the circle in the bottom of the Fetch Feed box, and drag it to the top circle in the Sort box. Voilá, they're connected! Now do the same from the bottom circle in the Sort box, to the Pipe Output box.
There we are. If you want to test it you can click the Refresh link in the debugger below the grid.
Now click Save in the top right, and name your pipe. When it has finished saving, click Run Pipe at the top of the screen. A new window opens, where you can do stuff with your pipe.
Looks good? Then click Publish to make it accessible. Now, what we want is the pipe's RSS feed, so click the More options link to the right and Get as RSS.
There you go, one mashed RSS feed containing the latest from four different feeds!
Adding The Feed To Wisdump
So let's add the feed to Wisdump's sidebar then, shall we? The pipe's feed is this: http://pipes.yahoo.com/pipes/pipe.run?_id=bntjQkH73BGSmXmSjknRlg&_render=rss
So how do we add it? With a RSS parsing plugin? We could, but since Wisdump is powered by WordPress, why not just use WordPress' built-in feed parser? Let's do that.
This little code snippet is well known, several blogs have reported variants of it. It's snatched from the RSS feeds in WordPress' admin.
require_once (ABSPATH . WPINC . '/rss-functions.php');
// insert the feed URL here
$rss = @fetch_rss('http://pipes.yahoo.com/pipes/pipe.run?_id=bntjQkH73BGSmXmSjknRlg&_render=rss');
if ( isset($rss->items) && 0 != count($rss->items) ) {
?>
<?php
// set the number of items from the feed to display (10)
$rss->items = array_slice($rss->items, 0, 10);
foreach ($rss->items as $item ) {
?>
<li>
<a href='<?php echo wp_filter_kses($item['link']); ?>'>
<?php echo wp_specialchars($item['title']); ?>
</a>
</li>
<?php } ?>
<?php } ?>
This will spit out a list where each item in the pipe's feed is a li list item, so you'll probably want to insert an ul around it.
The key is:
$rss = @fetch_rss('http://pipes.yahoo.com/pipes/pipe.run?_id=bntjQkH73BGSmXmSjknRlg&_render=rss');
...where you insert the feed's URL, and:
// set the number of items from the feed to display (10)
$rss->items = array_slice($rss->items, 0, 10);
foreach ($rss->items as $item ) {
?>
The number of items you want to display. I went with 10 here.
Here's the code used in Wisdump's sidebar:
<h2>News From The Network</h2>
<?php
require_once (ABSPATH . WPINC . '/rss-functions.php');
// insert the feed URL here
$rss = @fetch_rss('http://pipes.yahoo.com/pipes/pipe.run?_id=bntjQkH73BGSmXmSjknRlg&_render=rss');
if ( isset($rss->items) && 0 != count($rss->items) ) {
?>
<?php
// set the number of items from the feed to display (10)
$rss->items = array_slice($rss->items, 0, 10);
foreach ($rss->items as $item ) {
?>
<li>
<a href='<?php echo wp_filter_kses($item['link']); ?>'>
<?php echo wp_specialchars($item['title']); ?>
</a>
</li>
<?php } ?>
<?php } ?>
</ul>
You can see it in action over at Wisdump (down a bit in the sidebar), which incidentally has got a new design, so check that out too.
Good luck with your feed mashing!
Want an avatar? Get a gravatar! • You can link to this comment
Nice Article.
It seems that you like Yahoo Pipes. I managed to implement almost 30 Yahoo Pipes in my site MacrosReader (http://reader.macrostandard.com/). Just take a look and run a few of it! Thank you.
Want an avatar? Get a gravatar! • You can link to this comment
Excellent. Note that Geckotribe’s stuff is capable of this too, and with a lot more flexibility. Carp and Grouper, along with their new Wordpress plugin can give you tons of control.
Want an avatar? Get a gravatar! • You can link to this comment
Thanks.
There are several services that could do this. I do like Yahoo Pipes though, it’s stable enough.
Want an avatar? Get a gravatar! • You can link to this comment
Thord, I’ve written no fewer than 20 articles on Pipes, including video screencasts, and none of them looked as clear as yours. Good work, especially with that bit of code explanation.
Want an avatar? Get a gravatar! • You can link to this comment
Thanks Raj.
I urge everyone interested in tweaking WordPress to check out Raj’s series on http://performancing.com
Want an avatar? Get a gravatar! • You can link to this comment
Love this Post. Great tips! Surely will try this
Want an avatar? Get a gravatar! • You can link to this comment
Are you all having luck using the Yahoo Pipes RSS feed with fetch_rss? I sure am not
. I get “Warning: Invalid argument supplied for foreach()” about 75% of the time. Which, as you know, looks fantastic when it breaks my site, lol.
I don’t have any other problems with any other feeds, just Pipes RSS output. I wonder what is special in the RSS that causes that?
Want an avatar? Get a gravatar! • You can link to this comment
Hi there,
Thanks for the tutorial, really excellent and easy to understand. I’ve now used Pipes to aggregate a number of blogs that I publish into one feed.
However, I’d like to extract the names of each post author from the original source feeds, and display these in the aggregated feed along with the post titles/excerpts as well – to make it clearer that each post in the Pipes feed is by different people. Anybody got any ideas how I can use Pipes to do this?
Thanks
Russell
Want an avatar? Get a gravatar! • You can link to this comment
Thank you for this post, I used the code you provided on my own site. Some of the titles I’m bringing down are very long, but with php it was very easy to shorten them. If anyone is interested I did it this way…
if (strlen($item2['title']) > 80)
echo wp_specialchars(substr($item2['title'],0,80)) . “…”;
else
echo wp_specialchars($item2['title']);
?>
Want an avatar? Get a gravatar! • You can link to this comment
Sorry about the double commment, I hit enter to quickly. I did have one question…
Pipes stores the post creator in dc:creator. I have tried to show that on my blog using
echo wp_specialchars($item2['dc']['creator'])
Does anyone know why this wouldn’t work?
Want an avatar? Get a gravatar! • You can link to this comment
Hi,
These pipes are looking good.
Want an avatar? Get a gravatar! • You can link to this comment
is there a way to put the dates on the individual feeds?
Want an avatar? Get a gravatar! • You can link to this comment
actually, the dates and the descriptions. thanks!
Want an avatar? Get a gravatar! • You can link to this comment
I’m trying to add Pipes in my wordpress sidebar – not sure why this isn’t working.
Why can I not add the Pipes RSS to the RSS Sidebar Widget in wordpress?