hoodwink.d enhanced


I Thought Process::detach Was My Friend #

by why in inspect

So, Try Ruby has been riddled with problems. At about 9 AM EST yesterday the whole thing was swapping pretty heavily. This was right in the epicenter of the Digg stampede. I mean imagine trying to manage all these hundreds of Ruby processes opening up. CPU is fine, but a single Ruby process will gobble up a good share of memory lickety split.

Try Ruby itself is pretty small, about 350 lines of Ruby code. But yesterday it was consuming 32M of memory per process. The dispatcher would start at 8M, but as it grew, forked children would swell as well.

The problem was Process::detach, which was used to ensure zombie kids get killed. More here. The method moves the $SAFE up and starts a thread which calls waitpid every second. I’m not sure what’s so expense. I think constant switching of $SAFE will do it.

Anyway, the code switched from something like this:

 FCGI.each_request do |req|
   if req.new_session? 
     pid = fork { ... }
     Process.detach pid

To a tip from Nobu on ruby-talk:

 trap("CHLD") {Process.wait(-1, Process::WNOHANG)}
 FCGI.each_request do |req|
   if req.new_session?
     fork { ... }

And memory consumption was sliced back down to 8M, the bare allocation. Blink. Blink. I guess I should probably add a waitall at the end of the script, too. Had this not worked, I was gonna do a telethon for another stick of memory. (Anyway, you can read the joyous parade of the diggers enjoyably trampling me until all was well an hour and a half later.)

said on 30 Nov 2005 at 14:31

Ouch, man. Why doesn’t rb_detach_process use SIGCHLD instead of polling? It’s just the way you’re supposed to do on Unix.

Since it’s part of the Ruby core, it could surreptitiously trap SIGCHLD without interfering with any actual Ruby-user traps of it.

Just do a rundown of waitpid(pid, &blah, WNOHANG) for all the detached pids every time it got a SIGCHLD , and then if it was none of those what died, pass it on down to whatever Ruby handler was installed.

said on 01 Dec 2005 at 05:35

Yeah! What he said!

Comments are closed for this entry.