So, I've noticed there have been a fairly large number of Xen bug related outages posted recently.

The good...
I like that Linode is open about the reason for reboots and outages. The more info the better.

It appears good strides are being made in getting the Xen bugs resolved in newer releases.

But...
I haven't seen any indication that the Linode staff is proactively trying to protect us from these bugs, even though they seem to be hitting rather frequently. My Linode is not a "high priority," so a random outage during the day at some point wouldn't really bother me. But, with that said, I can imagine people with high traffic or ecommerce sites would decidedly *not* want a server to randomly disappear in the middle of the day. It would make more sense to me to eliminate known bugs with a simple server reboot during off-peak hours, rather than waiting for them to randomly crash.

A server reboot doesn't necessarily protect against bugs - plenty of bugs can be caused by unusual circumstances that are not compounded by time. I don't know what kind the Xen bugs in the past have been, myself, though :)

Uhm... I think my reboot comment was taken out of context. I'm not suggesting randomly rebooting servers makes them work better. I'm alluding to the many updates recently that read like:

"X hit a Xen bug and went unresponsive. We took this opportunity to update it to our latest Xen stack." See the System and Network Status forum for several of these.

Only a small percentage of the of the Xen hosts have hit Xen bugs. Even within that group, their uptime before hitting one of the potential known Xen bugs is quite long.

We're still determining if our latest stack fixes all known bugs. The longer we wait to reboot machines, the more chance there is that something gets fixed in the meantime.

We'd rather take one, potentially random, reboot -- extending the time between Xen builds/fixes -- than force a reboot on all of the Xen hosts every single time we have a bug fixed, or think we may have fixed a bug. For instance, we've developed 19 production-deployed Xen stacks over the past 12 months. Multiply that by half the number of Xen hosts, and that would have been something like 1500 reboots. Not cool.

We're making progress. Our latest Xen stack shows promise that all of our known issues have been resolved -- but, it's going to take time for it to prove itself. Once things look 100% solid, we'll roll out planned upgrades on all the Xen machines.

-Chris

But when is off-peak? Linode has customers all round the globe. OK, the majority are in the USA, but people outside the US definitely don't want servers going down in the middle of their business day for a scheduled reboot, just because it's 3am in the states.

_________________
/ Peter

For these scenarios you want more than one Linode in a high-availability configuration, to avoid hitting a single point of failure.

Fortunately Linode provides very nice features like a private network (within the same datacenter) with out-of-band traffic and movable IP addresses. These can help a lot with this kind of setups.

BTW, you'll have to do this even with dedicated servers if you want to minimize downtime.