Home
jeffr_tech's Journal
 
[Most Recent Entries] [Calendar View] [Friends View]

Monday, June 18th, 2007

    Time Event
    3:08p
    select on kqueue
    Anyone who has written a large sockets app knows that select is really slow with a lot of file descriptors. Various operating systems have solved this in incompatible ways. For example, FreeBSD has kqueue, linux has epoll, solaris has /dev/poll, etc. Apple also has kqueue, as it is somewhat FreeBSD derived. Rather than have duplicate interfaces in the kernel for eventing, they wrapped it all up together.

    Select is implemented by creating a kqueue per thread and adding fds from the select bits to the kqueue event set. They actually persist the events and cache the old bits and then only do updates based on the delta. That means if your application is passing in a relatively static set of bits you get the scalability of kqueue! See http://www.kegel.com/dkftpbench/Poller_bench.html for an old benchmark on kqueue vs epoll vs select vs dev poll.

    We're presently trying to get patches out of apple rather than re-invent the wheel here. It has a secondary advantage; kqueue does not have any global locks in the FreeBSD kernel while select does due to historical implementation issues. Fixing the select locking makes it internally just like kqueue except without persistence.

    Overall, I think it's very clever and quite exciting.
    7:33p
    I have updated my scaling results with Linux 2.6.21.5 and glibc 2.6. You can see them at http://people.freebsd.org/~jeff/sysbench.png.

    This also has results comparing the old FreeBSD scheduler with no affinity, SCHED_4BSD. SCHED_ULE is the version of ULE that doesn't use per-cpu locks. And SCHED_SMP is actually ULE with per-cpu locks and various other tuning. I actually have had better results than those on the graph but I don't feel like regenerating it again right now.

    Linux has definitely improved but they still have some significant problems. I will try with tcmalloc later.

    << Previous Day 2007/06/18
    [Calendar]
    Next Day >>

About LiveJournal.com

Advertisement