jeffr_tech ([info]jeffr_tech) wrote,
@ 2007-09-14 14:59:00
Previous Entry  Add to memories!  Tell a Friend  Next Entry
proud
http://linux.slashdot.org/comments.pl?sid=297945&cid=20605013

"Linux, even with CFS, it's still fairly easy to 'upset' it by just producing a fairly large (2-4) amount of load. Solaris did notably better. While it seemed to have a few quirks with scheduling in general, it could sustain a load of around 8-12 without producing video/audio frame drops. FreeBSD, with the experimental SCHED_ULE 2.0 scheduler (as of March 2007) could sustain a load of over 80 with no problems, frame drops, or even glxgears slowing down to a complete crawl"

(22 comments) - (Post a new comment)

Great, but...
(Anonymous)
2007-09-15 11:31 am UTC (link)
Those numbers are great for FreeBSD and the new scheduler, so congrats. This is really a great job!

As with all benchmarks, they should be taken with a grain of salt, though.

I am a Linux user. Mainly because in my hardware FreeBSD doesn't work very good. And my hardware is very common and open source friendly, but still I've traditionally had many performance problems with FreeBSD, especially IO related.

A few weeks ago I decided to test FreeBSD 7 to see if performance *in my hardware* has improved. And make 3 little tests:

1) Extract a gzipped tarball: FreeBSD: 40s. Linux: 10s.
2) Glxgears: FreeBSD: 320fps and choppy. Linux: 480fps and smooth.
3) Apache bench (with tine file to prevent IO problems): FreeBSD 820 threads/s. Linux 2240 threads/s.

I obviously don't claim this to be scientific and the *real* performance of FreeBSD. It obviously has problems in my hardware. The difference is too big to be taken seriously.

So though I understand you feel happy about those numbers from that user, I doubt they represent reality very faithfully. I'm sure that the new scheduler is *very* good. But really, not *that* good as compared to Linux's 0(1) and CFS. Differences must exist, but not in that range.

By the way, I'm soon getting a new computer and I'm looking forward to test FreeBSD 7 on it. I'm sure it will perform fine there and I might stick with it :)

(Reply to this) (Thread)
Re: Great, but...
[info]jeffr_tech
2007-09-15 08:33 pm UTC (link)
You are right that anecdotes are anecdotes and benchmarks should be taken with a grain of salt. However, this comment matches my experiences and experiments. Even as far back as my original ULE paper in 2002 I documented many of the priority and interactivity problems in the O(1) scheduler that are just recently being fixed by CFS/SD. I did this not to take a shot at linux, but because originally ULE was partially a clone of O(1) and I discovered these problems. It has zero resemblance to anything else these days as I came up with my own solutions.

Frankly, there are differences this big and larger in many cases. BSD and linux have optimized for different things. We are probably much slower in some areas and much faster in other areas. I believe our desktop scheduling is superior however, and was even superior during the time of the 4BSD scheduler vs linux's O(n) scheduler.

As far as your testing goes, did you rebuild your kernel? Even the snapshots these days have invariants and witness enabled. This is to catch bugs before the release. They are associated with as much as 400% kernel slowdown in some cases.

Anyway, I don't claim we're faster at everything. I know that's not true. That, however, does not mean we're not better at some things.

(Reply to this) (Parent)
Re: Great, but...
(Anonymous)
2007-09-16 04:43 am UTC (link)
"A few weeks ago I decided to test FreeBSD 7 to see if performance (...)"
Which is a development version and have a default kernel filled by all possible debuging options ...

Rebuild FreeBSD base system and ports with same flags as you did for your Linux and without debuging options and try again.

(Reply to this) (Parent)
Re: Great, but...
(Anonymous)
2007-12-25 03:45 pm UTC (link)
That's great. You admit that your test PC has problems with FreeBSD 7, yet you felt the need to share FreeBSD versus Linux benchmarks from it? And even draw the conclusion that you doubt some other guys tests represent reality?

The reality is that your setup was not fit to be used to draw any conclusions about an OS it does not support well.

You've wasted your time and the time of anyone who bothered to read what you wrote.

Your post looks like you were following some recipe for blind Linux advocacy...


1. Start out appearing friendly and supportive. Get on their good side first and hopefully give the appearance of being a reasonable person who really likes BSD.

2. Gently discredit gains of opposing "team". Was it a benchmark? YES! They can't argue with this! Benchmarks are always to be taken with a grain of salt.

3. Provide disclaimer that you are a Linux user, but give the BSD a jab by claiming you use Linux because BSD has poor hardware support. As if you would choose the BSD system otherwise. This continues the reasonable man masquerade and the friendly "I like BSD" fib.

(BTW, if your hardware really was very common and open source friendly, it would work on any BSD nicely)

4. Provide numbers supporting Linux and discrediting BSD. Make them up if need be. Or if you really want to test them, do it poorly, on hardware which is supported well by Linux and poorly by the BSD system.

5. Find some argument to claim that your test is better then theirs. No matter how adverse the conditions may be for the BSD system.

6. End by appearing friendly and supportive. Remain on their good side and hopefully continue the appearance of being a reasonable person who really likes BSD.


I seem to remember Microsoft unfairly testing Linux this way.

(Reply to this) (Parent)(Thread)

[info]juliettepizag
2008-07-16 06:34 am UTC (link)
Sometimes a PC running BSD or Linux is what you need. You can't benchmark that. For example Yahoo uses BSD to serve web pages.

(Reply to this) (Parent)
What did he mean saying "amount of load"?
[info]poige
2007-09-15 12:26 pm UTC (link)
Load avg.? If so, this is crap, no doubt for me. Speaking about Solaris; I've recently run it for a while (they sent to me DVD "Solaris Express Developer Edition") and I can say that using it as Xorg * Gnome (which is default there) runner is *disappointing*. It's *abrupt*, meanwhile Linux with {CFS,Realtime} patchset from Ingo Molnar, or SD from Con Kolivas performs really smoothly. Never tried to run FreeBSD on this machine, though. :)

You can be proud of course, but you'd better have facts for it.

(Reply to this) (Thread)
Re: What did he mean saying "amount of load"?
[info]nikolasco
2007-09-15 01:27 pm UTC (link)
If you, uh, read the full comment that was linked (and the excerpt is from), you'll see that the numbers are load averages. The poster is comparing responsiveness of multimedia desktop apps while compiling things (I think). It's far from formal.

[info]jeffr has frequently posted here with sysbench information.

(Reply to this) (Parent)(Thread)
> If you, uh, read the full comment that was linked
[info]poige
2007-09-15 03:50 pm UTC (link)
Well, so I guessed right. Load avg. isn't a reliable measurement tool at all, so that's why I call all the 'full comment' a crap. And as I personally tried Solaris (well, x86 edition I must say), I can make a conclusion that not only the full comment is a crap due to load avg. issue, but also due to its "Solaris did notably better"

> jeffr has frequently posted here with sysbench information.

Yeah, I know. Those posts are facts to be proud, not this one. :)

(Reply to this) (Parent)(Thread)
Re: > If you, uh, read the full comment that was linked
[info]nikolasco
2007-09-15 05:11 pm UTC (link)
Yeah, I took this as "supposedly this made a big difference for someone, good for them." In addition to the crap metric, their use case (multiple compilations with multimedia) seems to be far from the normal ones (servers, desktop users with a few apps, distributed crunch farms).

I don't mind jeffr taking joy in making someone's day better (which seems to be what the title indicates), especially considering the amount of poo flung at him by the usual monkey pack. I'd worry that he was letting this "go to his head" or something, but he's been consistent with benchmarking.

(Reply to this) (Parent)
Re: > If you, uh, read the full comment that was linked
[info]jeffr_tech
2007-09-15 08:23 pm UTC (link)
Load average is a reliable measurement of how many processes are on the run-queue over time.

My experience agrees with these results although I have not tried with CFS. On a single processor machine I have been able to do a make -j64 buildworld while simultaneously browsing the web and playing a video without skipping. On linux while building the kernel my desktop is often choppy.

(Reply to this) (Parent)(Thread)
> of how many processes are on the run-queue over time.
[info]poige
2007-09-16 03:53 am UTC (link)
Sure, man. But if I'm not mistaken, it even doesn't explain why they were in the run-queue -- blocked on I/O or due to lack of CPU power for very this bunch of running processes.

> I have not tried with CFS

I'd like to ask have your tried RT-patches by Ingo?

> On a single processor machine I have been able to do a make
> -j64 buildworld
[...]

I wonder do you use "nice'd" build?

> On linux while building the kernel my desktop is often choppy.

You mean using the very same machine in both cases, right?

(Reply to this) (Parent)(Thread)
Re: > of how many processes are on the run-queue over time.
[info]jeffr_tech
2007-09-16 04:55 am UTC (link)
The run-queue has only threads waiting to run, not threads blocked on io. The load average reflects the number of processes scheduled to run during a given period.

I have not tried any particular linux patches. I generally only compare to stock kernels. It would take too much time for me personally to investigate various linux patches. I leave that to linux developers and users.

In general I never use nice except to make sure I have not broken it in my scheduler. I don't believe the user should typically be asked to hint which workloads they care about.

Yes, I frequently dual boot machines between linux and BSD as I often need to run both operating systems for various contracts.

I usually don't talk about anecdotal evidence of poor io and thread scheduling in linux because it's difficult to be objective and it's an inflammatory subject as this thread shows. However, I find it more prone to skipping and stuttering. IO is very bursty and tends to saturate the disks for short periods rendering operations that require fsync() temporarily hung.

(Reply to this) (Parent)(Thread)
> It would take too much time
[info]poige
2007-09-16 09:13 am UTC (link)
Well, how much time would it take to type in bzcat some_patch.bz2 | patch -p1 -s && make bzImage. I guess much less than make buildworld. :)

BTW, can FreeBSD be installed to an 'extended' partition? I guess "no", but might be I've forgotten this.

(Reply to this) (Parent)(Thread)
Re: > It would take too much time
[info]jeffr_tech
2007-09-16 09:52 am UTC (link)
I don't believe there is any problem using extended partitions.

The problem with testing linux patches is that there are so many of them and there are already many volunteers who are testing them anyway. I am only interested in linux insofar as it provides a reasonable point of comparison. It is free and opensource and generally performs well.

I'm really not interested in trashing linux's scheduler as my post may indicate. I'm really more pleased that mine is doing well.

(Reply to this) (Parent)(Thread)
> I don't believe there is any problem using extended partitions.
[info]poige
2007-09-16 10:17 am UTC (link)
Using -- nope, but installing FreeBSD into one could be. I don't know, though.

> is that there are so many of them

3 of them worth trying: SD, CFS, RT. dot. :)

(Reply to this) (Parent)
> In general I never use nice except to make sure
[info]poige
2007-09-16 09:18 am UTC (link)
You know that CPU's power differs; there are machines which would be loosing no frames, playing video while compiling "world" and others, at contrary, being unable to do it. And it depends pirmarily on CPU, not scheduler; the only way to play video smoothly would be using nice to lower priority of other tasks.

> I don't believe the user should typically be asked to hint
> which workloads they care about.

Hell yeah! Might be this should be done automatically? :) I don't know.

(Reply to this) (Parent)(Thread)
Re: > In general I never use nice except to make sure
[info]jeffr_tech
2007-09-16 09:49 am UTC (link)
CPU speed and efficiency may differ but schedulers use real wall time to schedule them. It is true that things improve with faster processors but latency problems tend to show themselves regardless.

(Reply to this) (Parent)
Re: > of how many processes are on the run-queue over time.
[info]cks
2007-09-18 02:44 am UTC (link)

At least in Linux, the load average counts both processes that are runnable and processes that are in uninterruptible sleeps. Traditionally this is processes that are waiting on 'disk IO', including NFS, but I believe that Linux has a number of other things that will put a task into that state.

(Reply to this) (Parent)

[info]yvettenilas
2008-07-17 06:16 am UTC (link)
Has become an inseparable attribute of information management," says EMC CEO Joe Tucci. But one analyst notes that until EMC articulates a security strategy, it's too early to tell whether the purchase was a good idea.

(Reply to this) (Parent)

(Reply from suspended user)

[info]kart
2007-09-15 06:47 pm UTC (link)
Coincidentally, I just happened to have mod points today.

(+1: SCHED_ULE is great)

(Reply to this)


[info]cnst
2007-09-17 02:01 am UTC (link)
Good job, Jeff! I'm looking forward towards RELENG_7 branching -- then there'll be official builds to compare! :)

C.

(Reply to this)

(22 comments) - (Post a new comment)

Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…