Skip navigation

Category Archives: FYI

View it

Advertisements

BabyBook on Facebook

First ever HDD

This is your brain on the internet

– Lindsey

link

Hey guys,

I’ve been playing around with using yahoo pipes to scrape data from the internet and have processing visulize that data looking for emergent patterns. here’s a little how to if anyones interested:

from:http://gregsaul.co.nz/

THE INTERNET -> YahooPipes ->Processing

This is a quick how-to on how you can write a custom yahoo pipes to scrape data from various websites and rss feeds then dynamically load them into a processing application for representation.

Basically you can use yahoo pipes to do what it’s good at, collect, scrape¬† and collate data from around the internet then return have it return only what data your interested in to a processing application in a nice standardised re-usable way though a xml file.

First example (simple):
We have yahoo pipes collect all known earthquakes over the last 7 days from a rss feed provided by usgs, we then use the “location extractor” provided by yahoo pipes to extract the earthquakes location to lat and longs and we also use a simple regex command to extract the earthquakes magnitude from the earthquakes description. All of this is then returned to a processing script where the location and magnitude of each earthquake is visualised.

The Yahoo Pipe: here
http://pipes.yahoo.com/pipes/pipe.info?_id=a135e8a8730f782fb817d792298562a4

The processing Patch: here

Example 2: the average twit
This example searches for conversations about specific topics on twitter and builds up the average icon of those users talking about that topic. because we are not including icons more than once these faces tend towards individuals that twitter about a specific topic a lot, this means that more obscure topics tend to be among specific individuals resulting in a icon heavily weighted to the icons of a few users and more ubiquitous topics produce greater less defined icons.

The Yahoo Pipe: here
http://pipes.yahoo.com/pipes/pipe.info?_id=rJkmdk2b3RGn_ShI1ZzWFw

The processing Patch: here

requires the proXML library for processing from here

XKCD Mapping – Just for fun

<a href=”http://www.stltoday.com/stltoday/news/stories.nsf/stlouiscitycounty/story/C8A4053D0406FFDA862574DC00109BD1?OpenDocument”>A lawsuit seeks $25000 from fake Facebook account</a>; Internet harrassment as a crime

Here are two articles about Google’s censorship policies.

First, a Google censorship FAQ and then, something about Google’s censorship policy in China.