When the only tool you have is a hammer, everything looks like a nail.
I've had the good fortune to play around with Windows Vista a little over the past couple of weeks. I've been considering the upgrade to Windows 7, so I've read up a little on the newer features in Windows.
As a security-conscious developer, all the new security features in Windows sound very impressive. You've got your DEP (data execution prevention), your ASLR (address space layout randomization), your stack canaries, stack-smashing protection, sandboxing, virtualization, etc...
When I first saw all these 'technologies' described in one place, an image leaped into my mind.
Picture a guy holding a hammer. He's wearing special metal gloves to stop from smashing his fingers. He's got a sophisticated helmet and face shield to keep from bouncing the hammer back into his own head. He's got a safety harness on, and there's a 10-ft exclusion zone around him to make sure nobody else gets hurt. It takes him about 5 minutes to hammer a nail in.
Of course, the guy is a developer, and the hammer is C. (a more apt metaphor might be a hand-made black powder rifle, but stay with me here...). Instead of fixing the inferior tool that's used to build things, they try to mitigate the damage it causes. Every one of these 'technologies' is needed because the C language is inherently unsafe. Even a highly skilled programmer with decades of experience can still make the kind of mistakes that lead to your system getting pwned by the Russian Mafia, and it's all because of this antiquated, unstable-land-mine of a language.
There's no good excuse for writing critical infrastructure like operating systems (or applications) in a language like C. There are plenty of bad excuses. For example, "it's difficult to hire people to write code in better languages, there aren't enough of them". That's like complaining that everybody's trained to use a hammer, and it's too hard to train people to use a nail gun.
It's not like the technology isn't here. Microsoft themselves have been working on an entire suite of tools to replace C (C#,F#,.NET, Singularity, etc...), but they haven't been able to pull the trigger yet. Why is that? It's true that the development timeline for a huge project like a new operating system could easily span 10-15 years. But let's not make perfect the enemy of good. They could start by rewriting IE in C#. They have the talent, they have the tools, the ability. That one simple step would make Windows far more secure.
You'll notice that I haven't mentioned either Apple or Linux yet. That's because they're nowhere near as far along as Microsoft is on this front. Yup, I said that.
The Great Divide
One big reason for the foot-dragging (or should I say knuckle-dragging) are systems programmers. These guys are true Luddites, and they cling, snarling, to their 40-year old tools. They want nothing to do with the highly mathematical, cryptic, and slow-as-molasses-in-January high-level languages like ML. The thought of garbage collection gives them that sinking feeling of lost control you get just after the plane takes off and throttles back a little. These two camps - the systems programmers and the high-level language guys - don't interact at all. And the high-level language guys are just as much to blame. They're often ignorant of the needs of systems programmers, and even when they are aware of them they dismiss them. Their languages are too hard to use, too hard to read, and too hard to write, and one reason is that they still live mostly in academia, where the motivations are all twisted the wrong way.
I've spent some time in both camps, and would dearly love to bridge them, if for no other reason than this: until it happens there will be no secure operating system, and no secure applications. Also, it's a little lonely being one of the few guys that knows how to use the nail gun.
I know of only one project that's trying to bridge the gap, and that's BitC. If you're interested in both sides of the bridge, you should read The Origins of the BitC Programming Language.