...begins in wonder

code, poetry, philosophy, folly (Andrew Kuklewicz)

Yahoo pipes foray

10 Feb 2007

New free toys!
Everyone can appreciate the Unix pipe concept, even if it was a slow start to get it added. It's nice to see |s inspiring folks still, as with Yahoo pipes.
Did my own based on prx and flickr, give it a run.
I find it only works right about 50% of the time - seems like it doesn't retrieve all the feeds each time I request, so sometimes all I get are PRX listings, other times I get prx with the associated flickr images based on the pieces.
A few things I think could improve:

  • When dragging the end of a control, like a Text Input, to be the input to another form value, like in a URLBuilder, the drag and drop is very imprecise. In a list of three Query Parameters, I could never drop onto the middle element, it just wasn't able to only select that one middle parameter
  • Add a For Each: replace that does an element in a feed item, rather than the whole element. Or adds to it, hard to actually combine feedw without something like this
  • Have a smarter Union with options like interleaving rather than just putting all of one feed after another
  • The URLBuilder does not put the arguments into the query string in the order you have them in the form
  • While I love the debugger, there is no way to immediately run a pipe, you can only see debugger output, I want a way to preview the run
  • The output from a given source can only go to one widget - it would be great if the output of a fetch could be used for different processing, then recombined. I did this, but needed 2 almost identical Fetch controls to make it work, and adds another request to the processing time
  • Save is buggy - I have made changes to the URLBuilder, but Save would be disabled till I changed a different control. I also had once where I had saved some changes, then gone back to edit after running, and the changes were goe...
  • Consistent behavior of the fetch/content analysis/for each replace. I see times when I run it that no results come back, other times they work great. I have often seen that running a Content Analysis operation would flake out and show no results, even though the fetch above it would have results when I debug it. This same inconsistency seems possibly related to the general inconsistency of results mentioned above.
  • Also saw another bug after adding a For Each where I could not add the sub source to the For Each. Had to save, close up, and come back in to get it to work.

So all in all a good toy, but not ready for anything real - but I see the potential. Really the UI is unbelievable, and while a bit buggy still, I do not know how they made it such a rich UI - the pipes alone are an amazing bit of work.
Having used lots of GUI based tools like Informatica Powermart or the Weblogic Integration tools, or even just Rational Rose or Visio, I see where something like this is not far from being a very competitve interface, and incredibly light weight and responsive.