Monday, September 14, 2009

Pipes and Filters

One of the first systems I thought about when reading this paper was Yahoo! Pipes. I stumbled across this a few years ago and never really used it, but I can't argue with the power this sort of system provides. I also really never heard any huge buzz about it. It is a graphical editor to create data mashups that aggregate feeds and websites.

A system that would not benefit from using this pattern are applications that depend on throughput. It is possible that a system has to push data through hundreds of filters and pipes. That could potentially have a negative effect on the throughput of the application.

As for benefits of parallelism, as long as the data sets being worked on are independent you can see a benefit. That way each one can be worked on safely in parallel and then combined at the end.

No comments:

Post a Comment