One of the things that I've always liked about Linux is its excellent package management systems. Building an RPM or DEB is definitely a lot harder than NSIS or InstallShield, but the end result is a solid thing indeed. Thanks to the dependency checking in these package formats, I've been able to do successful in-place upgrades for many years on Mandrake (now Mandriva), Red Hat, and Debian systems. It's really simple -- you just change the source media from the current release to the release you want and tell it to go. If it's a major upgrade, you can even do it in chunks, rebooting between steps.
So, I was rather disappointed a few months ago when I went to upgrade some rather antiquated OpenSuSE releases on a couple of servers and found that the good folks at Novell have specifically disabled this capability. Furthermore, they discourage upgrading at all, recommending a clean install. That's not bad advice for Windows, with its mediocre packaging and 5-year release cycle. It's even okay advice for OSX, with its sadly primitive packaging and 1.5 year release cycle. But a Linux distribution with a 6-month release cycle and perpetual beta codebase is another matter.
As long as I had to go down into the crawlspace, take the server apart to get a DVD drive into its boot order, and reformat the system partition anyway, I might as well change the OS to something that can handle upgrading. This was back in November, and I switched it to a stable release of Xubuntu. The speed increase was immediate, and I've since done two in-place upgrades without disrupting anything or physically touching the box. I won't be using OpenSuSE again... having to physically visit and rebuild the white-box junkpile in my crawlspace is one thing, but the rackmount server at a data center in another state is far worse. Too bad its hardware is too old to handle IDE-R.