Comments on: Keith Rarick: Building Causes.com

By: Kristján

Kristján — Wed, 27 Aug 2008 02:53:42 +0000

Nope, your good, I would come out ‘kp’. ‘kr’ is the illustrious Mr. Rarick ;)

By: Ilya Grigorik

Ilya Grigorik — Wed, 27 Aug 2008 02:29:07 +0000

Kristjan, my bad, didn’t realize you’re ‘kr’. ;)

We’ve been using curl-multi at AideRSS for about a month now (extended it to do head requests), and its been rock solid.

By: Kristján

Kristján — Tue, 26 Aug 2008 18:23:54 +0000

Ilya, we wrote bindings for the C client:

http://github.com/kr/curl-multi/tree/master

It’s worked really well, but is one of the hardest parts of our infrastructure to debug when something goes wrong. Finding segfaults after they’ve pushed into the Ruby layer is akin to finding a needle in a pile of other needles.

By: Ilya Grigorik

Ilya Grigorik — Tue, 26 Aug 2008 12:46:36 +0000

Todd, new curb interface looks awesome. Have you guys done any benchmarking in terms of IO throughput / CPU overhead? I’m yet to find a ruby client that can saturate a 100mb pipe.

JB, I’ll ping Keith.

Kristjan, how were you guys running CurlMulti before? Native client?

By: Kristján

Kristján — Fri, 22 Aug 2008 00:51:35 +0000

@Todd - Thanks for the tip! We’ve been hoping/meaning to switch to an all Ruby implementation of CurlMulti for a little while now.

By: Glenn

Glenn — Thu, 21 Aug 2008 21:43:31 +0000

Great post, and thanks for the tip off on curl-mutli and curb, I’ll have to check them both out and see how they perform.

By: JB

JB — Thu, 21 Aug 2008 19:51:21 +0000

I would really like to hear Keith’s explanation of why he considers python “the most beautiful among the popular languages”, and why he feels it’s superior to Ruby. I’ve done my own assessment, and found the inconsistencies of Python very confusing compared with Ruby’s approach. If I missed something, I’ve love to know what.

By: Todd

Todd — Thu, 21 Aug 2008 19:09:42 +0000

If you’re using curl-multi, you should really have a look at the updated curb library on git-hub, here: http://github.com/taf2/curb/tree/master

It adds support for libcurl’s multi interface, but also all the options of the easy interface. Meaning you can do POST/GET and set any HTTP options through Curb and still run through a Multi loop. I also added support for running the select loop through ruby’s built in thread scheduling select loop - rb_thread_select, meaning it won’t block other ruby threads while perform, is waiting in the kernel.