Home » Internet » Web Programming » Other Web Programming

A Guide to Yahoo Pipes

Ranked #292 in Internet, #13,069 overall

Advanced RSS Feed Processing

Yahoo Pipes is a powerful composition tool to aggregate, manipulate, and mashup content from around the web. Simple commands can be combined together to create output that meets your needs.

For example, you can combine many RSS feeds into one, then sort and filter it. The resulting feed can then be used to provide dynamic content in your lens.

This lens will give you an introduction to Pipes, and how to create your own.

New: How to change the URLs of your pipes.

Where to Begin?

The first thing you should do is search for some existing Pipes, because you never know - someone may have already created the Pipe you need. Just enter in some keywords, and a whole bunch of Pipes will be presented to you. For example "amazon" or "squidoo".

Searching for Pipes is a great way to find ideas for your pipes. You can view the source of any pipe you find, and see how it was put together. This is a great way to learn how to create your own.

When you're ready to take the plunge, click on the large "Create a pipe" button. You'll be shown a blank sheet, on to which you can drag and drop the modules you need from the menu on the left. Modules are connected to each other by click-n-dragging a link between them.

At the bottom of the window is a debug window, which will show you the intermediate output of which ever module you have selected. Click on Refresh if it doesn't seem to have updated yet.

Below are a series of tutorials that take you step by step to creating your own Pipe, followed by some more advanced tutorials. At the end these is a Guestbook for your feedback and comments, as well as Plexo for your best Pipes.

  1. Simple User Input
  2. Creating a URL from User Input
  3. Fetching an RSS Feed
  4. Filtering a Feed
  5. Dealing with Dates...
  6. Remapping Elements in a Feed
  7. How To Customise Your Pipe URLs
  8. Working with Yahoo! Pipes (Book)

Tutorial Video for Yahoo Pipes

Yahoo Pipes tutorial
by LonewackoDotCom | video info

26 ratings | 33,051 views
automatically generated by YouTube

Simple User Input

One of the most flexible features of Yahoo Pipes is the ability to prompt for user input. This allows you to create a more generic Pipe that you can publish for anyone to use.

User InputTo add one to your Pipe, expand the "User inputs" in the left hand menu, and select the type you want. Here, I've picked the "String Input" field.

This field has 5 parameters: the 1st is just a name, so it doesn't matter what you call it; the 2nd is the prompt that the user will see when they run the Pipe; 3rd is for the order in which you want the inputs to appear, if you have more than one; 4th is the default value that the user will see; and the 5th is the value used for debugging.

In the debug window at the bottom of the screen, you should see that debug value appear.

Creating a URL from User Input

Once you have the user's input, you can use it in a number of places, for example building a URL.

Add the "Url Builder" from the "Url" sub-menu. In this example, I've set the Base to be "http://squidutils.com/lensmaster". I then linked the previous Text Input field by clicking on the blue dot at the bottom of the field, and dragging it to the dot on the right of the Path Element.


Add another Path Element by clicking on the (+) icon, and enter the remainder of the URL "lenses.rss". Note that you do not put in any additional "/" characters.

In the debug window, you should see "http://squidutils/lensmaster/thefluffanutta/lenses.rss", depending on what default value you specified in the Text Input.

Fetching an RSS Feed

The core of the Pipe is the RSS Feed. This module will fetch any RSS feed by URL, which you can then filter and manipulate as you desire. You'll find the "Fetch Feed" module under the "Sources" sub-menu.

You can enter a fixed URL, or connect it to the "URL Builder" module in the previous example.

The debug window should show the first few items in the feed, and you can expand them by clicking on them.

Filtering a Feed

In this example, I've added a "Filter" module, and linked it to the "Fetch Feed" module that I described above. I also created another "Text Input" field, which prompts for a "Category" from the user.


The filter is configured to only show items where the "category" element contains whatever string the user has entered in the "Text Input" field.

The debug window should show only the items which match. Note the special "Pipe Output" box at the bottom, which serves as the end point for your Pipe, and is from where your resulting output will emerge.

This Pipe has been published as Squidoo Lenses by Lensmaster and Category, and you can see the complete source code here.

Dealing with Dates...

Filtering by date is a useful feature, but it might not be immediately obvious how it is done. What helps are the special elements that Yahoo add to your feed when they read it in, prefixed with "y:". You can browse all of the extra elements by clicking on the arrows in the "Filter" module.


You'll see that in this example, I've used "item.y:published.utime", which has been automatically generated from "item.pubDate". This allows me to do a greater than comparision against the result from the "Date Builder" module.

Remapping Elements in a Feed


This quick example shows how you can take a feed, and replace the description for each item with its category tag.

How To Customise Your Pipe URLs

The default URLs for pipes are pretty unwieldy, but Yahoo has provided the ability for you to change them into something more manageable. Here's how:


The first step is to customise the URL of your profile. Start by going to the "My Pipes" page (assuming you're logged in), and look for your profile URL near the top. There should be an (edit) link at the end.

How to change your Yahoo Pipes Profile address



Once you've done this, click on the "View Results" of any of your pipes. Just below the title and description, there should be a link similar to before:

How to change your Yahoo Pipes address



Important Note: If you want to seperate your keywords in the URL, you must use underscores; dashes will get filtered out when you save it.

Once you have your new addresses, you can bookmark them, show them to your friends, etc. If you want to get the RSS output, then simply add ?_render=rss to the end of your URL.

Working with Yahoo! Pipes (Book)

No Programming Required by Tony Loton

Yahoo! Pipes is a powerful composition tool to aggregate, manipulate, and mashup content from around the web. Many of the chapters conclude with an Exercise for the Reader, which helps you discover more "hands on". And don't worry, the answers are given too!

In this book you will learn how to:
* Create Pipes by selecting modules from the library, and connecting them together.
* Publish your Pipes for the benefit of others.
* Discover and take advantage of already-published Pipes.
* Embed Pipe results directly in your own web pages.
* Combine Yahoo! Pipes with Microsoft Popfly to create more innovative mashups
Working With Yahoo! Pipes, No Programming Required

Working With Yahoo! Pipes, No Programming Required

by: Tony Loton

Amazon Price: $6.56 (as of 02/13/2012)Buy Now

"After reading this book, I actually "get it." I understand what this guy is talking about. The visuals really help."

Usually ships in 1-2 business days

What to do with your Pipes?

Once you've created your pipe, you can then save and run it. If you've got user input field, then you'll need to fill them in. If eveything is working, then you should see the results on the screen. You can then click on "More options" to get the new RSS feed.

If you think that other people would benefit from your creation, then you may Publish it. Make sure you add a good description and some tags to help people find it.

You can also add your pipe to this list below...

Some Example Pipes

If you have a Yahoo Pipe you'd like to show off, then feel free to add it here:

1

Vote Up
Vote Down

Pipes from thefluffanutta

All of the Pipes that I have created - go on, take a look...4 points

2

Vote Up
Vote Down

Pipes: Squidoo Lenses by Lensmaster and Keyword

Enter your lensmaster name and a keyword. You will then have a list of all of your lenses that contain that keyword in the title.3 points

3

Vote Up
Vote Down

Pipes: Ones - five random friends

A pipe made to display the top lens from each of 5 lensmasters feeds3 points

4

Vote Up
Vote Down

Pipes: Squidoo Lenses by Lensmaster and Category

Enter your lensmaster name and a category. You will then have a list of all of your lenses in that category.2 points

5

Vote Up
Vote Down

Pipes: Amazon: Show Similar Items (US and UK)

This will generate an RSS Feed containing Amazon items that are similar to a specified ASIN or ISBN.0 points

6

Vote Up
Vote Down

Pipes: eBay Multiple Country Price Search

This Pipe uses 10 of Ebay's RSS feeds from all over the world and US. The feeds are form UK, Canada, France, Indian, Spain, Italy, Singapore, Netherlands and Australia's Ebay Auctions. No need to jump from country to country you will get all the results here. If you can't find it in your country you...0 points

7

Vote Up
Vote Down

Pipes: eBay Simple Price Search

This Pipe was designed to search for a price range in eBay.com. It will allow you to add your search terms and Minimum amount and Maximum amounts. Quick Search on eBay for your favorite Item.0 points

Learn More about Yahoo Pipes

Pipes - About the Editor
The Pipes editor is a JavaScript authoring tool that lets you create and edit Pipes in an intuitive visual interface.
Pipes - Module Reference
The modules in the Pipes Editor have been grouped into categories based on their functionality.
Pipes - Tutorials
Many excellent tutorials for Pipes have been written by users...
Pipes - Frequently Asked Questions
Why can't I connect some modules to others?
Why can't I put some modules inside a ForEach container?
My Pipe is broken, what do I do?
Yahoo Pipes And How To Merge, Filter, And Sort Multiple RSS Feeds
Today's video is the next video in our RSS series, which shows you how to use Yahoo Pipes, a free service to manipulate multiple RSS Feeds to create another unique RSS Feed.  This allows you to drive more backlinks to your website among other benefits.

Mashup Case Studies with Yahoo! Pipes

Featured Author

Tony Loton has devised several Yahoo! Pipes case studies, each comprising one or more Pipes plus web pages that display their results.

Pipes developers can look at the web pages, examining the underlying HTML code, and then following through to the Pipes themselves.

"Mashup Case Studies with Yahoo! Pipes", available as paperback or PDF.

Here are the links to some of them. They're not meant to look nice, but to demonstrate various techniques:
Pipe JSON
Shows items for sale on eBay.
Trading Cockpit
Displays the output from two pipes. To the left is a list of stock sector charts, and to the right is related news for whichever sector link you click.
LOTONtech Finance Portal
Shows how a Yahoo! Pipe can screen-scrape information from a third-party web site, to be displayed in your own portal.
Product Download Form
It shows how a HTML form can capture user input to be used as a Pipe's parameter(s). Enter a username to see the downloads available to the fictional user.
Google Maps
Displays a Google Map showing locations sourced from a Yahoo! Pipe.

Reader Feedback

submit
Show All

More RSS Stuff

Loading

by

thefluffanutta

I write about what I know, and I build tools for Lensmasters at SquidUtils.com. Read more about me on my Lensography, and follow me on twitter. more »

Feeling creative? Create a Lens!

Explore related pages

Related Tags