13 December 2005

Dear John letter

Background: John and I regularly twit each other in a friendly manner over Linux® vs MS-Windows; this time I thought that, since I'd gone to the trouble of writing a lengthy response, I’d publish it.

On Tuesday 13 December 2005 01:00, John Xxxxxxxxx wrote:

Reliability: Analyzing Solution Uptime as Business Needs Change (PDF version)
http://list.windowsitpro.com/t?ctl=1A2C8:358B67

Uh, thanks, John. (-:

It seems obvious from this that they are not making effective use of the uniquitous packaging systems available for the various Linuces:

At a high level, the Windows systems were dramatically more reliable than the Linux systems. On average, patching Linux took six times longer than patching Windows, and there were almost five times as many patches to apply on Linux (187) as there were on Windows (39). More important, perhaps, the Linux systems suffered from 14 “critical breakages,” software dependency failures in which software simply stopped working on those systems. Windows had no dependency failures.

In my own experiences with Linux over the past decade, I’ve seen this sort of dependency problem arise numerous times. A certain software component requires a certain version of another software component, so you upgrade. But that upgrade breaks another component that depends on the upgraded component.

In the case of SuSE, that will be RPMs as manages by YAST. In my case, it’s RPMs managed by URPMI. In the case of Debian, it would be DEBs managed by APTitude. In all cases, sorting out the dependencies is the worry of the packaging people affiliated with the particular Linux distribution, not the admins.

The fact that they didn’t do this tells me that either their admins didn’t know what they were doing, or the eCommerce package in question was closed-source and/or unduly picky. The direct equivalent under MS-Windows would be a package that required specific versions of VBRUN300.DLL or the like. I’m sure you've run into those.

I would only ever install such under protest, and after making the managers concerned very well aware of the manageability risks that this package entailed for them. I presume you’d do the same, under the circumstances.

To illustrate: upgrading from Mandriva 10.1 (a nominal equivalent to SuSE 8) to Mandriva 2006.0 (a nominal equivalent to SuSE 10) involves doing this as superuser:

urpmi.removemedia -a
urpmi.addmedia --distrib [URL]
urpmi.addmedia --update updates [URL]
urpmi --auto-select

Done. You can do this on a live, running server (I know, because I have), and it will bounce any exposed services once or twice (typically sub-second interruptions) as it upgrades them. You will also, if you think it necessary, need to tell it to upgrade the kernel and reboot into the new one.

Now adding... oh, what shall we say? The English version of OpenGroupware is about as complicated a set of dependencies as you’ll be likely to run across; normally you’d pick what you wanted from a list (with descriptions) in a GUI and then click on Install, but let’s do this the hard way. The first line is what I type, the rest is the system’s response:

urpmi opengroupware-webui-resource-en opengroupware-webui-mailer
opengroupware-webui-scheduler opengroupware-webui-project
To satisfy dependencies, the following 19 packages are going to be installed (21 MB):
libopengroupware-core4-4.2-20040523.2mdk.i586
libopengroupware-xml0-4.2-20040523.1mdk.i586
opengroupware-docapi-5.0.0-20040523.1mdk.i586
opengroupware-env-1.0-20040523.1mdk.noarch
opengroupware-gstep-db-4.2-20040523.1mdk.i586
opengroupware-gstep-make-4.2-20040523.2mdk.i586
opengroupware-gstep-objc-2.95.3-20040523.1mdk.i586
opengroupware-js-1.5.5-20040523.1mdk.i586
opengroupware-libfoundation-1.0.45-20040523.1mdk.i586
opengroupware-logic-5.0.0-20040523.1mdk.i586
opengroupware-sope-4.2-20040523.1mdk.i586
opengroupware-webui-common-5.0.0-20040523.1mdk.i586
opengroupware-webui-forms-5.0.0-20040523.2mdk.i586
opengroupware-webui-libs-5.0.0-20040523.1mdk.i586
opengroupware-webui-mailer-5.0.0-20040523.1mdk.i586
opengroupware-webui-project-5.0.0-20040523.1mdk.i586
opengroupware-webui-resource-en-5.0.0-20040523.1mdk.noarch
opengroupware-webui-scheduler-5.0.0-20040523.1mdk.i586
opengroupware-xml-4.2-20040523.1mdk.i586
Is this OK? (Y/n)

In this case, I have most of the prerequisites already install, so answering “Y” will only install 19 packages, all of them directly related to the final apps; but on a bare naked system this would also imply the Apache webserver, Python scripting language, Zope CMS, PostgreSQL database, an email transport (PostFix by default), XML parsing libraries and so on.

Probably up around eighty packages when all was said and done, with zero extra effort from the administrator.

When upgrade time came, the packaging system would upgrade the lot, automatically.

If it thought a configuration item needed updating, but you'd made changes to the default that were complex enough to worry it, the packaging system would add a copy of the config file with “.rpmnew” and a few quality minutes with the diff utility would make any required hand-editing obvious.

Security updates are also simple. If you suspect that your package indices are out of date, do this:

urpmi.updates updates

Either way, do this:

urpmi --auto-select

Done. And trivial to automate.

When compared with Windows systems in the real world, Linux distributions are more complex to manage and can thus be less reliable.

Speaking to real-world experience and watching the MS-Windows admins around me wiping and reinstalling stuff wholseale “just in case” it impacts the issue they’re facing, I call bulldust.

Putting a system this robust, consistent and straightforward up against Microsoft's usual game of Pick-a-Patch or HotFix Roulette implies a walkover. The fact that a walkover wasn't reported implies — pardon the pithy language — that someone buggered things up somewhere, big time.

My guess would be that the people designing the test picked a particularly cantankerous eCommerce app, and this reasonably dispassionate analysis of the analysis seems to support my guess:

http://www.itjungle.com/breaking/bn112105-story01.html

<sarcasm weight="heavy">How surprisingly unlike Microsoft it would be to so specify.</sarcasm>

Reading ITJungle article, I wonder if the study priced the cost and trauma involved in updating MS SQL Server across a major version change, or simply ignored the difference.

In the case of updating MySQL without updating the rest of the distribution, I would pull down the source RPMs for the later MySQL packages, rebuild them on the target system, and install the resulting binaries rather than ginning about trying to reinvent the wheel by hand. This is generally a straightforward operation.

OTOH, if it had been up to me, I wouldn't choose MySQL in the first place, I’d choose PostgreSQL. This is my experience only, the MySQL people may disagree, but PostgreSQL doesn’t have to get so deep into the system chasing performance that it needs a new glibc version. Also, it would have been relatively trivial to set up a new glibc version just for the new MySQL instance, without ploughing up the rest of the system — including the normal packaging and updates.

None of these avenues seems to have been explored by the Linux admins used in the study, which makes this point particularly telling:

Why should I trust a company that is being paid by Microsoft to pick the so-called Linux experts? If you really wanted to do the Worldwide System Admin Smackdown properly, you would have Microsoft pick three admins and a Linux vendor such as Novell pick three admins, and name them and show their qualifications.

There are many other points which beg to be addressed; e.g. the 187 Linux “patches” (actually complete updates to the relevant subsystem rather than piecemeal file replacements) vs 39 patches for MS-Windows is more than a bit misleading. If the Linux update touches 4 packages, that is often counted as “4 patches” when in reality it is but one, and because the Linux people are perfectionists, a small change to a deeply buried subsystem may trigger updates to lots of dependent packages which Microsoft would deem unneccessary. The Linux people are also in the habit of patching trivial security holes and marginal risks where Microsoft will generally only patch showstoppers unless some seriously large client holds a gun to their collective heads.

Addressing all of these, every time, is a lot of work — generally wasted effort, too, since they involve actually understanding the issues, and many managers won’t — so it isn’t often done, or at least not systematically. However, rest assured that a lot of the confident prose accompanying such reports is about as solidly backed as a free-standing sheet of polystyrene.

1 comment:

JJ said...

Thanks Buddy

I'd call that a typically comprehensive answer and would have expected
nothing less.

I agree with the main point you seem to me making though. Linux or Windows,
if it's administered by morons, you won't see a happy result.

Luv JJ xx