jeffr_tech ([info]jeffr_tech) wrote,
@ 2008-03-07 00:22:00
Previous Entry  Add to memories!  Tell a Friend  Next Entry
More sysbench noise.
http://www.kernel.org/pub/linux/kernel/people/npiggin/sysbench/

Nick Piggin has been doing some benchmarking of recent linux kernels and FreeBSD 7.0 on a 2xquad core barcelona opteron. He verified that the CFS problems seem to be fixed and FreeBSD's performance on this box with mysql is really very similar up to about 20 threads. I feel confident that the test was conducted fairly and I'm happy with these results. Our stable release is doing very well even if fresh-out-of-git linux is showing better on this platform. We already have some good gains in this workload in 8.0-CURRENT as well. What's most important to me is that we stay relevant on common server hardware and we're doing a good job at that.

I'm also happy to see some collaboration and competition between linux and bsd kernel developers. I hope that continues. We're really more alike than we are different.

Next up, we now have a 16 way xeon and 16 way opteron system to tune and test with. More points of contention are being removed. The code marches on.

(Read 4 comments) - (Post a new comment)


[info]jeffr_tech
2008-03-07 10:35 pm UTC (link)
Typically we work on commodity server hardware. So intel has plans for a 6 or 8 way chip and then you're likely to see 4 socket version of this. So 24 or 32 processors. There have also been some special 8 socket opteron motherboards so you could conceivably have 8 quads for 32 processors but these are very rare and very expensive. The 4x4 machines are hard enough to come by.

The other option for more cores is sun's T2 chips which have 64 threads on them. This is a very different beast from a real 64 processor machine but it might be useful for working on general kernel scalability. We do have a port for these platforms but it's not well maintained.

In the end it comes down to what we can get our hands on. I can't afford to purchase every new generation of parts. Although I will probably try to get something with 32cores once that is mainstream as an upgrade over my current 8way systems.

Linux has the advantage of being backed by some companies with very large platforms of less tightly connected (NUMA) nodes with as many as 1024 cpus. We just don't have this resource and that isn't our core focus anyway.

(Reply to this) (Parent)(Thread)

[info]sniket
2008-03-08 03:39 pm UTC (link)
Jeff, you never went beyond 20 threads in the tests you published in your journal. Can you verify the gap between peek (~4300) and low point (~3000) in your own environment?

I also agree that the results are pretty close and I really liked Nick's conslusion:
"In other words, I can't say definitively that Linux is faster than FreeBSD. My primary interest is to see that Linux's performance problems on this workload are under control."

Regarding T2, I found this today: http://milek.blogspot.com/2008/03/16-core-intel-system-vs-niagara-2.html

(Reply to this) (Parent)(Thread)

[info]jeffr_tech
2008-03-09 05:07 am UTC (link)
I'm glad you asked about the low point. We actually just discovered today that it was simply our pthreads implementation not adaptive spinning for long enough. Changing an env var LIBPTHREAD_SPINLOOPS to 2000 puts our performance back in line with linux at these higher thread counts.

On this processor I still think we're about 4-5% slower than linux in 7.x due to non-core aware scheduling algorithms. In 8.0 we should be doing better again in a week or two. But Nick's conclusion is right. We both have our performance under control and in the same ballpark. What we wanted all along was for bad scalability not to be a reason to avoid bsd.

That's interesting news on the niagra. I can probably get one of those and we have a port mostly ready for it. Maybe we'll see if we can get some from sun to do further testing. Thanks for the link.



(Reply to this) (Parent)

(Read 4 comments) - (Post a new comment)

Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…