Skip navigation

Category Archives: Projects

Hey guys,

I’ve been playing around with using yahoo pipes to scrape data from the internet and have processing visulize that data looking for emergent patterns. here’s a little how to if anyones interested:


THE INTERNET -> YahooPipes ->Processing

This is a quick how-to on how you can write a custom yahoo pipes to scrape data from various websites and rss feeds then dynamically load them into a processing application for representation.

Basically you can use yahoo pipes to do what it’s good at, collect, scrape  and collate data from around the internet then return have it return only what data your interested in to a processing application in a nice standardised re-usable way though a xml file.

First example (simple):
We have yahoo pipes collect all known earthquakes over the last 7 days from a rss feed provided by usgs, we then use the “location extractor” provided by yahoo pipes to extract the earthquakes location to lat and longs and we also use a simple regex command to extract the earthquakes magnitude from the earthquakes description. All of this is then returned to a processing script where the location and magnitude of each earthquake is visualised.

The Yahoo Pipe: here

The processing Patch: here

Example 2: the average twit
This example searches for conversations about specific topics on twitter and builds up the average icon of those users talking about that topic. because we are not including icons more than once these faces tend towards individuals that twitter about a specific topic a lot, this means that more obscure topics tend to be among specific individuals resulting in a icon heavily weighted to the icons of a few users and more ubiquitous topics produce greater less defined icons.

The Yahoo Pipe: here

The processing Patch: here

requires the proXML library for processing from here


See the Map


Facebook: Lindsey
Twitter: Greg
Blog: Hye Jin, Brenda
Photos: Elise, Stephany, Sara
Gossip sites: Stephany, Joo Hyun
Cyworld: Joo Hyun, Hye Jin

A few of you don’t have roles yet. Go ahead and help out where ever you like.

– Lindsey

edit: ok, I just checked the e-mail and realized that a xanga had already been made too, but I know that people have multiple blogs for different things…

make sure this guy is GENDERLESS!!!!!
i just changed it

we posted a comment on juicy. it’s under Carnegie Mellon tagged as “kurzweil”
find it, vote for it, make replies, and increase its popularity.


Hey guys! Can someone post the task list? Who else is on blog duty?

I was wondering what our facebook picture should be?  We can’t really start friending people outside the class until we have a picture.  Otherwise we wouldn’t be as believable.  I vote we take a picture of everyone in the class and merge it together in photoshop to make a completely new person with little bits of each of us in it.  I have a digital camera and would be able to take pictures next class if you want.  If not, respond with suggestions, or possibly even pictures.


Hey all, I just made the gmail account  The password is currently diaspora13 but we can totally change it.  I had to enter in a name for the person in the account so I entered in Alex Kurzweil after Ray Kurzweil, the man who wrote The Age of Spiritual Machines. I really like that name but I would be happy to change any of this if you want, just let me know.

There is also a facebook page with the same email, name, and password used.  They are in the Carnegie Mellon network and no one can see their age or sex.  I said that they graduate in 2010 but if anyone wants to change that it’s fine too.