Friday, December 11, 2009

Performance, Stability, Security and Conclusion

Measuring performance of an OS is something difficult and very prone to subjectivity. It also depends wildly on hardware, drivers, and more importantly, the software you run on top of the OS. Still, overall a few things are probably safe to say: Windows 7 is a vast improvement over Vista in boot times and responsiveness, especially on machines with 2 GB ram or less, and machines that have been used for months. Windows 7 is overall pretty snappy, its comparable again to a fresh XP install and generally Ive not often felt I was being slowed down by the OS. Quite a relief when compared to Vista. That said, on my machines Ubuntu still boots faster and shuts down a whole lot faster. Its just as snappy, if not more so, and once it has booted, my harddisk remains quiet.

Ubuntu clearly requires less RAM. With all desktop effects enabled I boot into my desktop with barely over 300Mb ram used, roughly half of what 7 uses. Unless you run some large, memory hungry apps, Ubuntu will run just fine with 1 GB of ram and even 512 MB is twice the recommended minimum and is real world usable. Its probably not a stretch to say Ubuntu needs less than half the amount of ram 7 needs, and on modern machines with 2+ GB ram, swap space is usually not needed, nor used. Windows has this tedious habit of swapping applications from RAM to disk no matter how much free RAM you have, resulting in almost non stop disk activity whatever you do. Ubuntu will only swap when its really necessary, and a result, almost never does.

BTW, if you're truly RAM constrained, or have an old windows 2000 or 9x era machine you want to put to use, then check out specialised distributions like Puppylinux which happily runs with 128 MB, or Slitaz and Damn Small Linux that require less than 32MB (!).

There is more to “performance” than boot times and ram usage however, and in some other area's Ubuntu may disappoint. A lot of users report sluggish web browsing in firefox, especially scrolling is sometimes perceived as being less smooth as in windows. Flash playback is another notorious weak point, adobe just doesnt seem to invest as much effort in its Linux player as it does in the windows variant, and it shows, especially on low spec'd hardware.

In the end, overall Ubuntu feels significantly faster to me, especially on older or slower hardware like netbooks or machines with limited amounts of RAM, but YMMV and on well supported modern hardware neither OS should disappoint.

Ubuntu: 9/10
Windows 7: 9/10 on modern hardware, 7/10 for netbooks and older hardware.

Stability and reliability
Linux has a strong reputation for stability and reliability, because of its popularity in the server market and its unix heritage. Windows has the opposite reputation mostly due to its horribly unstable win 3.x, and 9x predecessors. But reputations are just that, and in reality windows has matured to the point where I consider it rockstable for desktop use, and any crashes or blue screens nowadays are usually caused by faulty hardware, malware or in some rare cases, third party driver bugs, but rarely issues with the OS itself. Likewise, and unfortunately, Ubuntu in my experience doesn't quite live up to its reputation either. If you expect unix server like stability, Ubuntu may disappoint you - in some cases, badly. Google on “Ubuntu freezes” or “Ubuntu won't boot” and you will find countless horror stories, proportionally many more than you will find for windows.

To be fair, the main cause of this, is not bugs or a badly written OS like windows 9.x but very often it is related to bios incompatibilities, typically incomplete or buggy ACPI implementations, and in some cases badly broken firmware of some devices. Its not “fair” to blame Linux for your motherboard vendor cutting corners by shipping boards with a broken or non compliant bios which they only tested with, or only made compatible with windows (often compiled with a known buggy microsoft acpi bios compiler). Its not "fair" to blame Linux for hardware with incomplete or buggy firmware that gets patched up through windows drivers, while those firmware patches are not made available for Linux. Its not “fair”, but it doesnt change your experience if you happen to own such a piece of hardware and Ubuntu (and other linux distro's) exhibit behaviour like not waking up from standby, flaky touchpads, losing network connectivity randomly or freezing more frequently than windows 3.11 ever did.

With the increasing popularity of Ubuntu, and the upcoming Linux based Google Chrome OS and intel's Moblin, I do suspect OEMs will test more thoroughly with Linux kernels and these issues will slowly fade away. In most cases such issues can also be worked around by updating your bios or firmware, or by disabling some specific functionality in the kernel. If you know what you're doing, you can sometimes even patch it yourself, and if its new hardware, its likely a patch or workaround will be provided quickly. But here and now, it can be frustrating, very hard to diagnose and cure, especially for a new user who will often not even bother to look for a solution and just concludes "Ubuntu sucks".

Not all stability problems can be blamed on the hardware vendors though, Ubuntu is certainly not without its own faults. In fact I have the impression Canonical is often pushing too hard to maintain their aggressive 6 month release cycle no matter what, and this goes at the expense of time to test their software properly. New releases often have severe bugs that impact a non trivial amount of users, and in some (admittedly, rare) cases just an update renders your machine unbootable or without functional GUI. If you spend some time on the Ubuntu support forums, one will quickly wonder if Ubuntu's regular 6 month releases do not deserve a semi permanent “beta” label in the google tradition; sure it works fine and is rockstable for the majority of users, but its anything but flawless or bug free for all. LTS (long term support) versions are a lot less prone to such bugs, so if absolute stability and reliability is what you are looking for, Ubuntu 8.04 LTS might be preferable over the latest 9.10.

In the end, generally calling Ubuntu unstable or unreliable would be a stretch, provided your hardware is properly supported under Linux. From personal experience, I have seen very few serious problems over the last years on any of my machines or the machines I manage, but when it comes to quality control other distributions like Debian or OpenSuse do a better job than Ubuntu, and if it weren't for virus and malware problems, it pains me to say, but IMHO Microsoft nowadays does a better job as well.

Ubuntu: 7/10 for regular versions, 9/10 for LTS
Windows 7: 9/10 if you can keep your machine malware free.

Security and privacy
Microsoft has made significant progress when it comes to securing their OS over the last decade(s). They made security a top priority and their work does bear fruit. Despite the common jokes, the amount of gaping security holes seems to shrink constantly and by including features over the years like a firewall, decent malware protection, the hated but important UAC and drive encryption, Windows 7 is no longer the security joke it once was.

However, due to the closed-source nature of Windows, your perception of security and privacy does depend in no small amount on your faith in Microsoft and other third parties, as there is no way to verify if there are no hidden backdoors in the OS or in third party software. When you install drivers or other software, you can usually not verify or have anyone else independently verify for you that these programs do not offer attack vectors or compromise your stability, security or privacy, either by accident or intent.

Even with complete trust in the ISVs, achieving reasonable security requires some work, knowledge and common sense. If you open pamela-blowjob.avi.exe or you set a 3 character password on your administrator account, no OS is going to protect you 100%. Not even Linux is fundamentally different here.

What is different is the fact that in the real world, for a variety of reasons, the risk of getting hacked, infected by a virus, worm, trojan, malware or rootkit on a Linux PC is almost nil, even if you are stupid and ignorant. To some extent, this is due to a fundamentally better security model of the OS, and partially because of the relative unpopularity of the OS compared to windows. More important however is the opensource nature of Linux and its software, and the mechanism most Linux distro's use to distribute software: if you download something through Ubuntu software center, you actually connect to a trusted and managed library of software, a so called repository. This repository contains both compiled binaries and the source code, which is free for anyone to inspect.

Repositories also digitally sign and checksum their software, so even if someone would manage to hack Canonical's servers to infect the binaries with a virus or rootkit, the signature verification mechanism would still reject the download. Software packages are also frequently (re)built from source code and source code is a damn difficult spot to hide malware or backdoors.

A good analogy I read somewhere, is that of a biological virus. For it to exist and spread its reproduction rate must exceed its death rate. The same is true for a computer virus, so while in theory linux is not 100 % immune to malware, in reality the environment is so hostile to malware that no one has managed to write a successful virus or worm that actually works and spreads (and don't think no one ever tried). As a result you simply do not need an antivirus or antispyware for Ubuntu, although some do exist (mostly to detect windows viruses which are harmless on Linux).

Likewise, Ubuntu doesn't need a firewall by default, for the simple reason that unlike windows, by default it doesn't have any open ports with software listening that an intruder could attack. Not having open ports renders a firewall completely pointless. If you do install software that listens to a port (say a webserver or ssh server) you obviously want that port open, so a firewall is still unnecessary, unless for more advanced configurations, for instance if you want to specify which network or IP address can access those ports.

The fact Ubuntu doesn't need a firewall doesn't mean it is completely immune to remote hacking or worms; just like in the windows world, security flaws do exist and if you do run server software and that software has an unpatched flaw, it will provide an attack vector, although the extent of the damage that can be done will usually be quite limited courtesy of a rigorous security model. Also the opensource nature of linux does ensure thousands of people look at the source code and can identify and quickly patch security holes. Lastly the diversity of Linux distributions and its countless versions means that any single security hole will typically only affect a relatively small fraction of all users, making it rather impractical to exploit for any potential hacker or worm.

One more point about security; the old mantra "physical access equals root access" applies equally to Windows and Linux. Anyone who gains physical access to your machine is able to compromise your data. If nothing else someone could always boot from a CD and format your harddrives. Unless you encrypt your document folders, with either OS it is easy to copy or access your personal files, and with some effort its usually possible to obtain passwords and steal your digital identity. Both windows and Ubuntu provide easy to use and secure drive encryption, which protects your identity and privacy (although it obviously does nothing to protect its integrity). Unfortunately, Microsoft restricts this functionality only to the expensive Windows ultimate and enterprise editions, and you still need to believe MS on their word that it is not providing backdoors by intent or accident.

To summarise, is Ubuntu really more secure than windows? Yes, definitely. But not infinitely; enabling ssh and setting a 3 letter password will still open the front door to your personal files for any potential hacker. Giving someone physical access to your machine will compromise any unecrypted data. As usually with security the weakest link is the user, and Linux is no exception to that rule.

Ubuntu: 9/10
Windows 7: 7/10 if you know what you're doing and have trust in MS and the NSA. 2/10 if you are ignorant or frequently see black helicopters circling your house.

Maintaining the OS
Most of us are part-time network admins on our home network -if nothing else. Keeping our machines up and running and up to date does require some work. In the windows world, this means installing windows updates now and then, doing application updates, updating your AV, de-fragmenting your drive, and, let's be honest, every 6 or so months, when our registry has bloated to hundreds of MBs and windows crawls to a standstill, do a complete reinstall. I've not used windows 7 for long enough to be sure, but it does seem to have improved in that last aspect at least, as after a few months of use, its still fairly snappy and hasnt slowed down nearly as much as I was used to with XP. Then again I have no used it all that much either, but Ill give it the benefit of the doubt.

Ubuntu further improves on some of this. For starters there is obviously no need to update your AV and antispyware as you dont need any. Similarly, you dont have to defragment -ever. Ubuntu's filesystem (ext4) is immune to fragmentation. The small price you pay for that luxury is that you can not use 100% of your drive or partition's capacity as by default 5% will remain reserved (though this can be changed and in reality you want at least as much free space on windows anyhow). There is no slowdown when your drive is almost full, in fact there is no slow down when its entirely full (though weird things start to happen when your root partition is full). In general there is hardly any perceptible slowdown in Ubuntu over time. It pretty much remains as fast after a year as it was the day you installed it, regardless of how many applications you installed, uninstalled and reinstalled.

But if you do want to do a fresh Ubuntu install, you have some interesting options not available in windows. For starters, you can simply backup and restore your home folder (or better, just keep a separate /home partition). That will ensure 99% of your OS and application settings remain preserved even if you re-install the OS. To a large extent this will retain your OS and application settings even when you install a different version or even a different linux distribution.

You can also export a list of all the programs and packages you have installed, and import that list in to your package manager to automatically install all the same packages and programs again with one simple command.

Finally, you can even remaster your own Ubuntu liveCD. You do an initial install, add the applications and utilities you want, configure the OS the way you like it, download any patches that may have been released, then make your own custom live CD based upon that install, and use it for backup, deployment or give it to your friends. Its surprisingly easy to do and a very powerful tool.

Another advantage of Ubuntu is that updates are managed through the repositories. That means that the OS itself, as well as all the applications you installed through the package manager (or software center), are all kept up to date in the same way, using the same update manager. Once per day (or week or month, however you configure it) the update manager will prompt you with bug fixes and security updates for all your installed software, whether it is applications, drivers, plugins or the OS. In fact, even a new version of the OS is installed in the same way. Every ~6 months, update manager will tell you a new distribution release (Ubuntu version) is available and you can upgrade your OS to the latest version.

A minor downside of this centralised approach is that the list of updates tends to get very long and selectively installing updates can be a bit of a pain. Updates are also very frequent, if you have a lot of applications installed don't be shocked to be downloading 100+ MB of updates on a given week.

Another potential point of criticism is the application update philosophy that Canonical uses: by default Ubuntu will not update your applications to newer versions, it will only provide security and bug fixes. Application versions remain frozen until the next release of Ubuntu. This means if you installed Ubuntu 9.04 which came with firefox 3.0 and openoffice 3.0, you would remain on those versions until the next Ubuntu version is released and you upgrade your OS. If you want firefox 3.5 or OO 3.1 or other versions of applications released after the Ubuntu 9.04 freeze, you will need to download and install them manually, which in some cases may not be as easy as it seems. A way around this, is enabling the backport repositories, which will provide you with tested and packaged application version updates for at least some popular applications.

To recover your system from user error or anything else rendering your system unbootable, both OS's offer some tools. In Windows you have the good old safe mode, there is the option to roll back to a previous restore point and when you boot from the CD there are some (annoyingly) automated recovery tools. I say annoying because they are anything but verbose, they will just diagnose and attempt something without telling you even what its doing, potentially causing more harm than good.

Ubuntu also offers a recovery mode which will give you a menu with a few automatic fixes for frequent problems, or a root shell that will enable a savvy user to fix just about anything. You can also boot into older kernels from the boot menu if a new kernel update should give you any problems. There is however, no real equivalent to Windows' system restore and sometimes it would be good to have that.

Another glaring omission in the default application set of Ubuntu is a backup utility. I'm anything but fond of Windows backup (does anyone really use that?), but its better than not having one. Fortunately this is easily fixed by selecting Back In Time or Simple Backup from the software center. More powerful backup utilities based on rsync are also available.
All in all, IMHO its definitely easier to maintain Ubuntu, especially the unified update manager is a huge step forward over having 235 different and independent update managers popping up every day, but I do look forward to seeing a "system restore" equivalent for Ubuntu in the future.

Ubuntu: 8/10
Windows 7: 6/10

Closing thoughts
I started experimenting with Linux about 5 or 6 years ago. I was intrigued by the concept of an open operating system, and I craved for something other than Windows. Not because Windows didnt work for me, but after decades of using Microsoft OS's, I just wanted to try something different, it seemed unlikely Microsoft were the only ones having all the good idea's.

My initial impressions of Linux weren't that great. The distributions I tried back then seemed so rough and unfriendly that I didnt have enough motivation to get everything installed and working properly. I just couldn't get my head around even the basics and I would always struggle to get my videocard or sound working. It all seemed too much effort for too little benefit.

To be honest, its only when I first saw a demo of Beryl, with a 3D desktop cube that I set myself the goal of achieving the same on my own PC. Sure the cube was fairly pointless, but it did look cool and it was the challenge that I needed, the concrete goal that would motivate me. The whole process actually took me several weeks, if not months of experimenting, reading, trying and usually... failing; but I learned so much along the way. I had fun, in a very geekish kind of way. A whole new world opened up to me Though I still saw Linux more as a toy, a hobby, rather than an OS to actually use, I got hooked. Over time I began booting Windows less and less, to the point where I only used it for games, and even that became increasingly rare.

Its amazing how fast Ubuntu has progressed in those short few years. I wouldnt have dreamed this 5 years ago, but about 2 years ago, when I grew tired of providing tech support for the Vista laptop of my 70-something year old mother, I installed Ubuntu as an experiment. A month later the Vista partition was wiped and my tech support effort dropped to almost nil. Soon after her nerdy grand children, aged 5 and 9 discovered the "new laptop" was so much more fun than the "old laptop". The youngest one literally taught himself to read and do basic math using GCompris on Ubuntu. By now, both nephews run ubuntu as well as Windows on their own computers, and astonishingly, they didnt need my or their father's help to install it. Barely 10 years old, never installed an OS in their life, and they achieved what once took me weeks of hard work.

Anyway, as much as Ubuntu and Linux in general have progressed over the last years, it obviously still isn't the best OS for everyone. It probably never will be. Yet, perhaps ironically, both geeks and noobs should seriously consider it. For the noobs there is the ease of finding and installing free software, as well as the effortless security and low maintenance. For the geeks, its like an endless kindergarten. Everyone else will have to try for themselves and weigh the pro's and con's.

Which brings us to our final verdict. I don't believe any single OS can be ideal for everyone and every machine, but in the end there can only be one winner in this comparison. There is absolutely no doubt in my mind who that is; the only winner in this contest between two great operating systems, is you.

No comments: