jeffr_tech: More subjective scheduler tests.

jeffr_tech (

jeffr_tech) wrote,
@ 2007-09-20 02:27:00

More subjective scheduler tests.
So I've just installed fedora core 7 on my laptop so I can do a side by side comparison of scheduler performance. The test I like to do with ULE is to make -j64 kernel while playing a dvd with mplayer and browsing the web. I have verified that make -j64 in the base directory of the linux kernel gets the load average up to the same level. The actual source involved matters little. Both systems are using a gcc 4.x compiler. The laptop is an IBM T42 with a 1.8ghz PentiumM,2 gigabytes of ram, and a 7200 rpm drive.

With the FreeBSD 7.0 configured without debugging options INVARIANTS and WITNESS, but with SCHED_ULE, I get no skipping or glitching of any kind.

With the O(1) scheduler in the default fedora core 7 kernel I get intermittent skipping but it's generally pretty tolerable.

I installed a 2.6.23-pre7 kernel built with 'make defconfig'. Feel free to suggest configuration options that may be relevant. At -j64 the ui was basically unusable. At -j4 it skipped more than the older kernel using -j64. It is my understanding that this is using the CFS scheduler.

I must say that in general the linux experience was very good. The installer was easy enough, although neither it or the installed system seemed to detect my atheros wireless card. I didn't care to futz with it so I just plugged it into my switch directly. It sure looked pretty otherwise. Maybe I'll try PC-BSD when they update to 7.0.

The issue with CFS is that the simple algorithm works very well as long as your interactive tasks consume less cpu in proportion to the other tasks. So for example mplayer takes 10% of the cpu playing a dvd on my machine. If you have 9 cpu hogs and mplayer running, without any information other than runtime or %cpu you can't distinguish between them.

Whether this is an important workload or not is subject to debate.

(28 comments) - (Post a new comment)

danfe
2007-09-20 10:09 am UTC (link)

Speaking of fair scheduling, I wonder whether CFS or any other recent Linux schedulers (and ULE, of course) take into account task hardware activity (e.g. mouse and keyboard). I'm under impression this is what most gamers (both under Linux and FreeBSD) complain to date: performance is sluggish compared to Windows, where it's much smoother on the same hardware.

I've been thinking of instrumenting Quake II source with high-precision time counters to get "process heartbeat distribution" that can be statistically comparable to Linux'/Windows' case. I yet need to instrument (KTR'ace) the kernels as well, as I'm not sure if my timing results would be of any significant help without such instrumentation.

Feel free to correct me if I'm wrong. Thanks.

(Reply to this) (Thread)

jeffr_tech
2007-09-20 10:30 am UTC (link)

I had not heard of that complaint. FreeBSD actually gives a very high static priority to threads sleeping waiting for input from devices like this. The priority is lowered aAfter the thread services the device, but the thread is not immediately switched out in favor of other higher-priority threads. This has been around in BSD since well before FreeBSD started.

Is it possible to run quake 2 or 3 on bsd? I don't know. It might be interesting to try. I don't know if I need a hardware accelerated driver even?

There are other issues that may effect this. How frequently the mouse is sampled, for example. Games may also benefit from higher hz rates and more precise timers. There may also be some effect because the games are really designed and optimized for windows even if they may work on unix.

(Reply to this) (Parent)(Thread)

danfe
2007-09-20 10:44 am UTC (link)

Yeah, sure, it is possible: Quakes 1-3 are opensourced (under GPL), Quake 4 runs fine under Linuxulator, just search the ports. Accelerated driver is probably important, but not essential. Considering that even Quake 3 is considered old nowadays, anything like GF2/i850 would suffice.

I'll try to collect more data WRT mouse rate and kernel HZ. However, I do remember that I had to lower HZ from 1000 to 400 to fix sound issues on 5-STABLE with EDuke32 (games/eduke32). *Sigh*, it's not all that simple.

(Reply to this) (Parent)

(Reply from suspended user)

	katiebipef 2008-07-16 02:17 pm UTC (link)
	Anyway hardware support under windows is allways easier than linux, all you have to do is to insert the disk/download the drivers for that hardware and you are set to roll in windows. (Reply to this) (Parent)(Thread)

	isaiahhaynen 2008-08-11 11:04 am UTC (link)
	WARNING : Don't set the timer to 0 seconds if you also have Windows set as the default boot option or you might not be able to boot Ubuntu anymore. (Reply to this) (Parent)

	rashadmarciliu 2008-08-06 05:41 am UTC (link)
	Desktop Linux is really terrible compared to Windows. It's much better than before but it needs a lot of work. (Reply to this) (Parent)(Thread)

	sheltonlathem 2008-08-06 06:00 am UTC (link)
	Linux ain’t bad but it needs a little more time to getter better. Sumesh March at pm If it weren’t for my EVDO wireless internet card that I use always, I’d have been on Ubuntu full time. (Reply to this) (Parent)

	ulyssesblair 2008-08-11 04:14 am UTC (link)
	Joel March at pm @Emory Well, Linux might be good by its certainly not better than windows. If you say that its better than windows then you probably: a. (Reply to this) (Parent)(Thread)

	moseshayward 2008-08-11 05:25 am UTC (link)
	Joel March at pm @Emory Well, Linux might be good by its certainly not better than windows. If you say that its better than windows then you probably: a. (Reply to this) (Parent)(Thread)

	hoseaweerlie 2008-08-11 05:44 am UTC (link)
	Joel March at pm @Emory Well, Linux might be good by its certainly not better than windows. If you say that its better than windows then you probably: a. (Reply to this) (Parent)

	chelseaglovier 2008-08-11 07:32 am UTC (link)
	Joel March at pm @Emory Well, Linux might be good by its certainly not better than windows. If you say that its better than windows then you probably: a. (Reply to this) (Parent)

	daphnemundefor 2008-08-11 01:52 pm UTC (link)
	Joel March at pm @Emory Well, Linux might be good by its certainly not better than windows. If you say that its better than windows then you probably: a. (Reply to this) (Parent)

	samuelamerindi 2008-08-11 06:05 am UTC (link)
	Joel March at pm @Emory Well, Linux might be good by its certainly not better than windows. If you say that its better than windows then you probably: a. (Reply to this) (Parent)

	modestobennet 2008-08-11 03:37 pm UTC (link)
	Vista has some hiccups compared to windows XP but compared to Windows 98 its way superior. If any of you are on win you are nuts to be on that crap. (Reply to this) (Parent)

Desktop scheduling
(Anonymous)
2007-09-20 10:24 am UTC (link)

I couldn't understand the logic behind a fair scheduler being better for a desktop (desktop scheduling should be unfair by nature), so I asked about it in LKML (http://lkml.org/lkml/2007/6/22/406).

The consensus was that the scheduler should not make decisions regarding which tasks should have more CPU time than others. That should be done via nice levels.

Of course, this sounded strange to me: you can't expect desktop users to renice their tasks according to their needs. It should be done automatically somehow. They said that distributions or application programmers should take care of using the right nice level for each application. As I'm no expert I accepted this explanation, but I see (think) you don't agree much with it and my feeling is that your approach is better.

(Reply to this) (Thread)

Re: Desktop scheduling

jeffr_tech
2007-09-20 10:40 am UTC (link)

I generally dislike nice. I really only see utility in using nice +20 as a pseudo idle priority thing. In most cases users are not going to change tunables and static allocations are almost never going to scale properly. This is the same reason I argued against any kind of threshold/watermark based memory allocation schemes. I favor adaptive algorithms wherever possible.

The ULE algorithm is simple, efficient, and has few edge cases where it's not likely to do what you want. It has some tunables available via sysctl but I consider those more for development experiments than for users. However, if the interactivity algorithm isn't doing what you want, you can disable it with a sysctl and use static nice values.

The only really inconvenient edge case is when the task you'd like to be interactive is a cpu hog and you have a number of other cpu hogs running. However, the scheduler is not omnipotent. In this case tricks like boosting the priority of foreground windowing applications are reasonable but we fortunately have not had to resort to that.

(Reply to this) (Parent)

	jamelvilett 2008-08-11 02:05 am UTC (link)
	"I know you can't endorse me, but I want you to know that I endorse you. " The quote launched the Christian Conservative movement as a political force in the U. (Reply to this) (Parent)

	evan 2007-09-29 05:56 pm UTC (link)
	Saw this, thought you'd be interested: http://mail-index.netbsd.org/tech-kern/2007/09/28/0014.html (Reply to this) (Thread)

(Anonymous)
2007-09-29 09:10 pm UTC (link)

Interesting, thanks!

One strange thing is that in Jeff's tests both FreeBSD and Linux peaked at the same number of threads as cores the system had (8 in his case). But in this one, while FreeBSD has a similar behavior (except a small fall after 4 threads), Linux behaves very differently. At 4 tps its performance is very low. It then equals FreeBSD at 7 tps and peaks at about 14 tps in a 4 core box. NetBSD performs only slightly higher than FreeBSD at 4 tps but then peaks at 7 tps. Strange.

Another thing that seems clear is that when a test is run on the system that the developer is using to develop a particular feature the behavior is perfect. Not strange since all the bottlenecks found in that setup have been solved. But then you test in different hardware and the behavior can change dramatically. Here's probably where Linux takes advantage of a broad base of testers. In a particular setup/hardware it might not perform as well as others, but its behavior across many different setups is quite good.

(Reply to this) (Parent)(Thread)

jeffr_tech
2007-09-29 09:22 pm UTC (link)

I thought I had actually made a post with this link but I guess I didn't.

The test is interesting indeed. However, it's not the same as the tests I ran. The client is always remote in this test. So the results are not directly comparable. The test that I ran was very repeatable across a range of hardware as long as the number of cpus was at least 4 or so. I tested on dual HTT xeons and dual quad core xeons as well as other opterons of varying sizes.

(Reply to this) (Parent)(Thread)

	NetBSD outperforms FreeBSD .. ?!? (Anonymous) 2007-10-01 08:03 pm UTC (link)
	I wonder how NetBSD manages so high TPS -- 1.5x better than FreeBSD .. ?!? (Reply to this) (Parent)(Thread)

Re: NetBSD outperforms FreeBSD .. ?!?

jeffr_tech
2007-10-01 10:57 pm UTC (link)

Well given that the vast majority of the NetBSD kernel is still under a giant lock I suspect something other than kernel scalability is at play here. This sysbench test spends about 25% of the time in kernel on a modern machines so with 4 cores you definitely need concurrency in the kernel.

It's hard to say what's going on. We probably have a problem but it's not so simple as bad concurrency. We're in the process of setting up a quad 600mhz xeon of the same generation so that we can run our own tests and see if we can reproduce this.

I suspect it will be something related to the particular machine architecture.

(Reply to this) (Parent)

alidalyjog
2008-07-16 05:38 am UTC (link)

You can try to get a perfomance baseline by writing a small program that moves data between 2 threads (threads share memory easily, and you don't need to write SYSV shmmem code) and i) affining the threads to cores on the same chip, then ii) moving one of the threads to a core on another chip in the same board, and then iii) moving it to a core in another board (I browsed the Tyan site and the 8-way you got is built from 2 4-way "boards").

(Reply to this) (Parent)

	matthewtrotter 2008-08-11 02:05 am UTC (link)
	SuSE Linux Enterprise Server 10 might be an interesting thing to try over Ubuntu, as they have some cpu/mem scalability patches that aren’t in mainline yet IIRC. (Reply to this) (Parent)

	jeffr_tech 2007-10-01 11:59 pm UTC (link)
	We just reproduced better numbers than this on an older 4x 500mhz pIII xeon. So I think he must've had some debugging enabled. It's impressive that their numbers are this good given how early their SMP project is. (Reply to this) (Parent)

	Great work (Anonymous) 2007-10-15 04:27 pm UTC (link)
	Jeff - I am sitting at a pre-release 7 machine with SCHED_ULE complied in. Running many Gnome apps. Very, very nice. Thanks for all your hard work. Arjen (Reply to this)

	Thanks cacala 2009-06-27 05:38 pm UTC (link)
	sesli sohbet sesli chat sesli chat seslichat sesli panel tatil otelleri ucuz oteller kiralık tekne tekne kiralama ajans oyuncu tekirdağ nakliyat tekirdağ evden eve nakliyat evden eve nakliyat sağlık (Reply to this)

(28 comments) - (Post a new comment)

Username:		Create an Account Forgot your login or password? Login w/ OpenID
Password:
	Remember Me
	English • Español • Deutsch • Русский…

About

Help

Get Involved

Legal

LJ Labs

Store