Comments on: Easy Map-Reduce With Hadoop Streaming

By: charles

charles — Thu, 18 Feb 2010 19:53:13 +0000

That is, there seems to be no notion of a Sawzall externalized aggregator…the Sawzall table keyword. I guess it would be OK if the actual aggregation was accomplished with a hadoop reducer…as long as it was transparent to the user. Here, it is unclear, and I did not see a def for table in the source.

By: charles

charles — Thu, 18 Feb 2010 19:52:55 +0000

That is, there seems to be no notion of a Sawzall externalized aggreator…the Sawzall table keyword. I guess it would be OK if the actual aggregation was accomplished with a hadoop reducer…as long as it was transparent to the user. Here, it is unclear, and I did not see a def for table in the source.

By: charles

charles — Tue, 16 Feb 2010 23:21:24 +0000

In order for this to be Sawzall, it needs to run on top of HBase, not just Hadoop, so that aggregation can be externalized

By: Mia Rencontre

Mia Rencontre — Mon, 22 Jun 2009 12:13:13 +0000

I don’t agree; at the moment I still consider it easier to try to build my own distributed processing layer

By: Ilya Grigorik

Ilya Grigorik — Tue, 09 Jun 2009 02:52:03 +0000

Tommy, thanks a lot for the link! Wukong looks amazing!

By: Tommy

Tommy — Mon, 01 Jun 2009 18:52:18 +0000

Also, take a look at Wukong (http://github.com/mrflip/wukong/tree/master) for a hadoop map reduce framework in ruby. I find the data stream feature of Wukong more flexible.