Linode Backup Service (beta)

If you're participating in the beta and discover an issue, please open a new thread.

We want you to have backups. Hell, I want backups. But they're tedious to configure and monitor. We've all heard the stories (and possibly experienced them) when you need a backup long after configuring them you realize they haven't been running successfully for months. So, our goal for the Linode Backup Service was ambitious: create a reliable, redundant and highly available system, that's easy to use (set-it-and-forget-it), affordable, and Just Works.

Description
The Linode Backup System is designed to be an easy to use, reliable and redundant on-site backup solution for your Linode. It performs backups without causing any interruption of your running system, and is seamlessly integrated into the Linode Manager.

Backing Up
There are four backup slots: Three of the slots are executed and rotated automatically: a daily backup, a 2-7 day old backup, and an 8-14 day old backup. The fourth backup slot is a user-initiated snapshot and remains in place until another user-initiated snapshot is taken.

You can configure the time upon which the automatic backups are initiated from a list of 2 hour windows -- you'll want to perform any database dumps before this window. You can also configure which day of the week to consider for the weeklies.

Restoring
You can restore a backup to any of the Linodes attached to your account, even if it does not have backups enabled. Currently only a full restore is possible.

Redundancy
We built a custom distributed and scalable storage engine which will replicate your files across at least two storage nodes. Even your backups are backed up!

Features and Limitations
The backup system must be able to mount your disk images on the host. If you've used fdisk on your images to create partitions, or created encrypted volumes, or done anything other than use our deployment or disk image creation tools, we won't be able to back up the data. The backup system operates on files, not at the block level.

A failed backup will never rotate out a good one. If a backup fails on the day of a weekly backup, the next oldest backup will be used for that weekly slot.

Files that have been modified, but are the same length and without any metadata changes (like mtime) will not be considered "changed" during a subsequent incremental backup.

Currently, only ext2/3 volumes can be backed up. This limitation will be removed in an upcoming release.

ACLs are correctly backed up and restored.

Pricing Structure (tentative)
Linode 360: $5.00/mo
Linode 540: $7.50/mo
Linode 720: $10.00/mo
Linode 1080: $15.00/mo
Linode 1440: $20.00/mo
Linode 2880: $40.00/mo


Backup Service Beta

How do I participate in the Backup Service Beta?
Participating in the beta is free of charge.

We've only deployed the storage hardware in the Newark, NJ facility. So, for now, you must have a Linode in Newark to participate in the Backup Service Beta. Open a ticket under that Linode and request that we enable backups.

What happens at the end of the beta?
The backup service will be scheduled to be turned off automatically. However, we'll give everyone the chance to retain their backup service and will start charging for it.

What kind of risk is there? Is this thing going to work?
There's little risk to your existing data. However, I wouldn't rely on our backup service as your only form of backup just yet. During the beta period, backups may be inconsistent or incomplete, or we may need to wipe all of the backup data and start fresh. In other words, keep your own backups and don't rely on this thing working. You've been warned!

What about the other facilities?
Atlanta and Dallas sometime later this month, and Fremont early May.

-Chris

Yay Caker you've made my day - thanks a lot !

Will this take an actual "snapshot" or will we need to stop MySQL, etc. in order to prevent file corruption.

Also, will the backups be available outside of the linode network - i.e. can we access them from home?

Awesome, thanks! Looking forward to checking it out! And I'm glad it's in Newark

This may not work very well for things like MySQL InnoDB tables, which allocate chunks of disk. I realize a database is not the best example, but let's say it's a big, mostly read-only database.



It doesn't sound like it's an LVM-based snapshot, and instead is running rdiff/rsync or something simpler. The original post says we'll need to dump databases to a file prior to the backup window.

Great idea!
Any idea of how long a backup should take?

I'm backing-up my Linode 360 atm. So far it's taken 1.25 hours.

This will work fine because the mtime changes every time a change is made.

-Chris

I beg to differ . This is a very sophisticated system which *does* involve LVM snapshots on the host. But what's the difference? Live database files can't be snapshotted, or copied, or rsync consistently, anyway.


Doing a snapshot does NOT always mean you're doing to get your mysql DB files in a consistent state. It can depend on which type of table engine you're using, for instance...

So dump your DBs to a file before the backup window just like you would with any other backup configuration.

-Chris

I like "very sophisticated" here!

Just making it clear: ext4 disk images are not currently supported, correct?

My understanding is that LVM snapshots can be made of running InnoDB tables and restored successfully. The InnoDB tables will be in "crashed" state, but the data will recover automatically.

There's also the option to mysqldump --single-transaction with InnoDB and get a consistent backup without blocking applications.
http://dev.mysql.com/doc/refman/5.1/en/ ... ransaction

Other MySQL storage formats (like the default MyISAM) don't have this ability.

By "simple" I didn't mean simplistic, I meant like K.I.S.S.!

I'm somewhat more interested in being able to just do a backup on demand (I backup my important files to AWS daily) and grab a copy of that backup for my own archives (i.e. burn a dvd set).

The first is taken care of by the user-selectable backup, but will the second be available? I.E. Will I ever be able to grab a disk image?

...Chris

Yeah, but there's a potential snag when involving a virtualization layer:

We're issuing the LVM snapshot on the host -- not from within your Linode's kernel. Performing a snapshot flushes dirty buffers, but since that's being done on the host your Linode kernel is unaware that this is happening, so it still has dirty buffers that were never written to disk.

This may be moot because of journaling (both at the filesystem and the db level), but, speaking for myself, I'd dump my DBs to a file...

-Chris

Restoring restores to disk images, which you can then mount and pull down on your own with whatever (rsync, scp, tarball, etc).

-Chris