jeffr_tech's Journal

Feeling nostalgic.
10 years ago this year I started at my first programming job. 5 years ago this year I became a freebsd committer.

Today I had some of W. Richard. Stevens's (TCP/IP illustrated etc.) famous cookies. Google them. They're tasty.

I'm about half-way through a patch to remove FreeBSD's global scheduler lock. I started it about 3 years ago. Hopefully it won't take me another 3 years to finish. The mechanism is mostly ripped off of Solaris, although I didn't look at their code. The idea is to have a per-thread lock pointer that protects that thread's state. The lock transitions from one container to another. For example, while you're running it's a pointer to the per cpu run queue lock. While you're sleeping it's a pointer to the sleep queue lock, or a turnstile lock if you're blocked on a mutex.

This arrangement actually reduces the amount of locking you have to do, while giving you excellent parallelism. For example, previously when you wanted to wakeup a sleeping thread, you had to lock the sleep queue it was on, lock the scheduler to protect the thread state, and perform the wakeup. Now locking the thread has the side effect of locking the queue. Pretty fancy eh?

As it turns out, priority propagation is the only really hard part to do without the global lock. This is because you're traversing a chain of locks and operating on threads holding those locks and this whole operation must be done with a clearly defined lock order or with the aid of a deadlock detector. I'm working out a way to do it with a clearly defined lock order. I understand solaris has some fancy deadlock detector code that probably allows them to do fewer lock operations.

It's too bad solaris has such a nice kernel and such a crap user experience. Really if I had solaris on alphas with my scheduler, I probably wouldn't feel like I needed to work on free software, because it'd work the way I wanted. ;-)

Advertisement