Advanced RSS Feed Processing
Yahoo Pipes is a powerful composition tool to aggregate, manipulate, and mashup content from around the web. Simple commands can be combined together to create output that meets your needs.
For example, you can combine many RSS feeds into one, then sort and filter it. The resulting feed can then be used to provide dynamic content in your lens.
This lens will give you an introduction to Pipes, and how to create your own.
Where to Begin?
Searching for Pipes is a great way to find ideas for your pipes. You can view the source of any pipe you find, and see how it was put together. This is a great way to learn how to create your own.
When you're ready to take the plunge, click on the large "Create a pipe" button. You'll be shown a blank sheet, on to which you can drag and drop the modules you need from the menu on the left. Modules are connected to each other by click-n-dragging a link between them.
At the bottom of the window is a debug window, which will show you the intermediate output of which ever module you have selected. Click on Refresh if it doesn't seem to have updated yet.
Below are a series of tutorials that take you step by step to creating your own Pipe, followed by some more advanced tutorials. At the end these is a Guestbook for your feedback and comments, as well as Plexo for your best Pipes.
Tutorial Video for Yahoo Pipes
Simple User Input
To add one to your Pipe, expand the "User inputs" in the left hand menu, and select the type you want. Here, I've picked the "String Input" field.This field has 5 parameters: the 1st is just a name, so it doesn't matter what you call it; the 2nd is the prompt that the user will see when they run the Pipe; 3rd is for the order in which you want the inputs to appear, if you have more than one; 4th is the default value that the user will see; and the 5th is the value used for debugging.
In the debug window at the bottom of the screen, you should see that debug value appear.
Creating a URL from User Input
Add the "Url Builder" from the "Url" sub-menu. In this example, I've set the Base to be "http://squidutils.com/lensmaster". I then linked the previous Text Input field by clicking on the blue dot at the bottom of the field, and dragging it to the dot on the right of the Path Element.
Add another Path Element by clicking on the (+) icon, and enter the remainder of the URL "lenses.rss". Note that you do not put in any additional "/" characters.
In the debug window, you should see "http://squidutils/lensmaster/thefluffanutta/lenses.rss", depending on what default value you specified in the Text Input.
Fetching an RSS Feed
You can enter a fixed URL, or connect it to the "URL Builder" module in the previous example.The debug window should show the first few items in the feed, and you can expand them by clicking on them.
Filtering a Feed
The filter is configured to only show items where the "category" element contains whatever string the user has entered in the "Text Input" field.
The debug window should show only the items which match. Note the special "Pipe Output" box at the bottom, which serves as the end point for your Pipe, and is from where your resulting output will emerge.
This Pipe has been published as Squidoo Lenses by Lensmaster and Category, and you can see the complete source code here.
Dealing with Dates...
You'll see that in this example, I've used "item.y:published.utime", which has been automatically generated from "item.pubDate". This allows me to do a greater than comparision against the result from the "Date Builder" module.
Remapping Elements in a Feed
This quick example shows how you can take a feed, and replace the description for each item with its category tag.
How To Customise Your Pipe URLs
The default URLs for pipes are pretty unwieldy, but Yahoo has provided the ability for you to change them into something more manageable. Here's how:
The first step is to customise the URL of your profile. Start by going to the "My Pipes" page (assuming you're logged in), and look for your profile URL near the top. There should be an (edit) link at the end.
Once you've done this, click on the "View Results" of any of your pipes. Just below the title and description, there should be a link similar to before:
Important Note: If you want to seperate your keywords in the URL, you must use underscores; dashes will get filtered out when you save it.
Once you have your new addresses, you can bookmark them, show them to your friends, etc. If you want to get the RSS output, then simply add ?_render=rss to the end of your URL.
What to do with your Pipes?
If you think that other people would benefit from your creation, then you may Publish it. Make sure you add a good description and some tags to help people find it.
You can also add your pipe to this list below...
Some Example Pipes
If you have a Yahoo Pipe you'd like to show off, then feel free to add it here:
Pipes: Squidoo Lenses by Lensmaster and Category
Enter your lensmaster name and a category. You will then have a list of all of your lenses in that category.1 point
Pipes: Squidoo Lenses by Lensmaster and Keyword
Enter your lensmaster name and a keyword. You will then have a list of all of your lenses that contain that keyword in the title.1 point
Pipes from thefluffanutta
All of the Pipes that I have created - go on, take a look...1 point
Pipes: Amazon: Show Similar Items (US and UK)
This will generate an RSS Feed containing Amazon items that are similar to a specified ASIN or ISBN.0 points
Pipes: eBay Multiple Country Price Search
This Pipe uses 10 of Ebay's RSS feeds from all over the world and US. The feeds are form UK, Canada, France, Indian, Spain, Italy, Singapore, Netherla...0 points
Pipes: eBay Simple Price Search
This Pipe was designed to search for a price range in eBay.com. It will allow you to add your search terms and Minimum amount and Maximum amounts. Qui...0 points
Learn More about Yahoo Pipes
- Pipes - About the Editor
- The Pipes editor is a JavaScript authoring tool that lets you create and edit Pipes in an intuitive visual interface.
- Pipes - Module Reference
- The modules in the Pipes Editor have been grouped into categories based on their functionality.
- Pipes - Tutorials
- Many excellent tutorials for Pipes have been written by users...
- Pipes - Frequently Asked Questions
- Why can't I connect some modules to others?
Why can't I put some modules inside a ForEach container?
My Pipe is broken, what do I do? - Working with Yahoo! Pipes, No Programming Required by Tony Loton (Book)
- In this book you will learn how to:
* Create Pipes by selecting modules from the library, and connecting them together graphically on the Pipe Editor's canvas; no programming required.
* Publish your Pipes for the benefit of others.
* Discover and take advantage of already-published Pipes.
* Embed Pipe results directly in your own web pages.
* Combine Yahoo! Pipes with Microsoft Popfly to create more innovative mashups
Mashup Case Studies with Yahoo! Pipes
Featured Author
Pipes developers can look at the web pages, examining the underlying HTML code, and then following through to the Pipes themselves.
"Mashup Case Studies with Yahoo! Pipes", available as paperback or PDF.
Here are the links to some of them. They're not meant to look nice, but to demonstrate various techniques:
- Pipe JSON
- Shows items for sale on eBay.
- Trading Cockpit
- Displays the output from two pipes. To the left is a list of stock sector charts, and to the right is related news for whichever sector link you click.
- LOTONtech Finance Portal
- Shows how a Yahoo! Pipe can screen-scrape information from a third-party web site, to be displayed in your own portal.
- Product Download Form
- It shows how a HTML form can capture user input to be used as a Pipe's parameter(s). Enter a username to see the downloads available to the fictional user.
- Google Maps
- Displays a Google Map showing locations sourced from a Yahoo! Pipe.
Reader Feedback
|
triathlontraining
I'd only just heard about Yahoo Pipes (from adez7!) and now it's one of my next things to learn. Your tutorial is perfect! :) Posted May 19, 2008 |
|
Yahoo-Web-Hosting
Thanks for the great info... Posted May 14, 2008 |
|
Chadrew
Great tutorial. I love Yahoo Pipes, I was so amazed that the tool I was searching for already existed. Plus the interface is very nice. Posted March 22, 2008 |
Anyone interested in Yahoo! Pipes might be interested in my book "Working with Yahoo! Pipes, No Programming Required".
It's not yet on Amazon (as I write this) but is available as paperback or PDF from http://www.lotontech.com/it_books.htm
Posted March 03, 2008
|
mulberry
Fabulous lens, NOW I'm starting to get it! Posted February 28, 2008 |
|
hypnoplace
Awesome tutorial, and thank you so much for sharing your valuable knowledge. Posted February 21, 2008 |
|
sajego
I love Pipes! It was so much simpler to figure out than the other sites I looked at. I'm actually using it for Measured Fit (lens) because I liked how the Amazon RSS feeds could be built on the fly and then pulled together with a Pipe. Posted February 01, 2008 |
|
clouda9
Not sure if I did this right...huge learning curve. I guess time will tell :) Great big hug for sharing this information! **Awaiting approval on the Plexo :) Posted January 29, 2008 |
|
Frankster
Great lens, fluff. Thanks! 5 stars and I'm a FAN! You should add it to the plexo in my How to Squidoo.... Posted January 28, 2008 |
|
flaminglacer
Thank you - I can get my aging head round it now! Squid Angel blessing for an excellent lens.... Posted December 06, 2007 |
|
flowergardener
Yea! I finally did it! Here's my first pipe: Posted December 05, 2007 |
|
adez7
Pipes is a very powerful tool and a great asset to have in ones arsenal of tricks. Combined with Dapper and others there is no limit as to what you can do. THis is a five star lens and a great resource. Thanks :) Posted December 05, 2007 |
|
dougeby
Thanks for this great info and the developed pipes. Will come in handy! Posted December 05, 2007 |
|
Loyalis
Yahoo pipes is one of the most powerful tools out there for creating dynamic content. Great tutorial! Posted December 05, 2007 |
More RSS Stuff
-
Lensmaster Tools and Utilities
-
SQUIDUtils.com Utilities and Tools for the Expert Lensmaster SquidUtils is the place to find complementry utilities and tools for your Lenses on Squidoo. Here you can see all of your lenses and groups, with shortcuts to some of the most vital fu...
-
How To Get Deep Links with RSS Feeds
-
There are a number of online syndication services that will display an RSS feed as a web page. These will typically contain deep-links to pages on to the originating website, which is a valuable source of backlinks. I originally wrote about this...
-
Amazon RSS Feeds for Your Squidoo Lens
-
I have this special service for Squidoo Lensmasters whom are also Amazon Associates - RSS Feeds containing products from Amazon. You can put these feeds into your lens to show books, DVD, electronics and more from specific categories. "But Why?&q...
News from Yahoo! Pipes
Fetching RSS feed... please stand by
