Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin 

Post new topic This topic is locked, you cannot edit posts or make further replies. Go to page 1, 2, 3  Next
Author Message
PostPosted: Fri Apr 03, 2009 12:31 am 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3017
Location: Galloway, NJ
Linode Backup Service (beta)

If you're participating in the beta and discover an issue, please open a new thread.

We want you to have backups. Hell, I want backups. But they're tedious to configure and monitor. We've all heard the stories (and possibly experienced them) when you need a backup long after configuring them you realize they haven't been running successfully for months. So, our goal for the Linode Backup Service was ambitious: create a reliable, redundant and highly available system, that's easy to use (set-it-and-forget-it), affordable, and Just Works.

Description
The Linode Backup System is designed to be an easy to use, reliable and redundant on-site backup solution for your Linode. It performs backups without causing any interruption of your running system, and is seamlessly integrated into the Linode Manager.

Backing Up
There are four backup slots: Three of the slots are executed and rotated automatically: a daily backup, a 2-7 day old backup, and an 8-14 day old backup. The fourth backup slot is a user-initiated snapshot and remains in place until another user-initiated snapshot is taken.

You can configure the time upon which the automatic backups are initiated from a list of 2 hour windows -- you'll want to perform any database dumps before this window. You can also configure which day of the week to consider for the weeklies.

Restoring
You can restore a backup to any of the Linodes attached to your account, even if it does not have backups enabled. Currently only a full restore is possible.

Redundancy
We built a custom distributed and scalable storage engine which will replicate your files across at least two storage nodes. Even your backups are backed up!

Features and Limitations
The backup system must be able to mount your disk images on the host. If you've used fdisk on your images to create partitions, or created encrypted volumes, or done anything other than use our deployment or disk image creation tools, we won't be able to back up the data. The backup system operates on files, not at the block level.

A failed backup will never rotate out a good one. If a backup fails on the day of a weekly backup, the next oldest backup will be used for that weekly slot.

Files that have been modified, but are the same length and without any metadata changes (like mtime) will not be considered "changed" during a subsequent incremental backup.

Currently, only ext2/3 volumes can be backed up. This limitation will be removed in an upcoming release.

ACLs are correctly backed up and restored.

Pricing Structure (tentative)
Linode 360: $5.00/mo
Linode 540: $7.50/mo
Linode 720: $10.00/mo
Linode 1080: $15.00/mo
Linode 1440: $20.00/mo
Linode 2880: $40.00/mo


Backup Service Beta

How do I participate in the Backup Service Beta?
Participating in the beta is free of charge.

We've only deployed the storage hardware in the Newark, NJ facility. So, for now, you must have a Linode in Newark to participate in the Backup Service Beta. Open a ticket under that Linode and request that we enable backups.

What happens at the end of the beta?
The backup service will be scheduled to be turned off automatically. However, we'll give everyone the chance to retain their backup service and will start charging for it.

What kind of risk is there? Is this thing going to work?
There's little risk to your existing data. However, I wouldn't rely on our backup service as your only form of backup just yet. During the beta period, backups may be inconsistent or incomplete, or we may need to wipe all of the backup data and start fresh. In other words, keep your own backups and don't rely on this thing working. You've been warned!

What about the other facilities?
Atlanta and Dallas sometime later this month, and Fremont early May.

-Chris


Last edited by caker on Mon Mar 29, 2010 3:44 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 4:13 am 
Offline
Senior Member
User avatar

Joined: Tue Mar 17, 2009 5:11 am
Posts: 129
Location: UK
Yay Caker you've made my day - thanks a lot ! :D


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 6:00 am 
Offline
Senior Member
User avatar

Joined: Sat Oct 16, 2004 11:13 am
Posts: 176
Will this take an actual "snapshot" or will we need to stop MySQL, etc. in order to prevent file corruption.

Also, will the backups be available outside of the linode network - i.e. can we access them from home?


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 6:56 am 
Offline
Senior Member

Joined: Mon Feb 28, 2005 7:21 pm
Posts: 76
Awesome, thanks! Looking forward to checking it out! And I'm glad it's in Newark :)


Top
 Profile  
 
PostPosted: Fri Apr 03, 2009 7:20 am 
Offline
Senior Member

Joined: Mon Feb 28, 2005 7:21 pm
Posts: 76
caker wrote:
Files that have been modified, but are the same length and without any metedata changes (like mtime) will not be considered "changed" during a subsequent incremental backup.


This may not work very well for things like MySQL InnoDB tables, which allocate chunks of disk. I realize a database is not the best example, but let's say it's a big, mostly read-only database.

saman007uk wrote:
Will this take an actual "snapshot" or will we need to stop MySQL, etc. in order to prevent file corruption.


It doesn't sound like it's an LVM-based snapshot, and instead is running rdiff/rsync or something simpler. The original post says we'll need to dump databases to a file prior to the backup window.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 9:29 am 
Offline
Senior Member

Joined: Mon May 14, 2007 8:20 am
Posts: 81
Great idea!
Any idea of how long a backup should take?


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 9:56 am 
Offline
Senior Member
User avatar

Joined: Tue Mar 17, 2009 5:11 am
Posts: 129
Location: UK
jcr wrote:
Great idea!
Any idea of how long a backup should take?


I'm backing-up my Linode 360 atm. So far it's taken 1.25 hours.


Top
 Profile  
 
PostPosted: Fri Apr 03, 2009 11:34 am 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3017
Location: Galloway, NJ
phvt wrote:
caker wrote:
Files that have been modified, but are the same length and without any metedata changes (like mtime) will not be considered "changed" during a subsequent incremental backup.


This may not work very well for things like MySQL InnoDB tables, which allocate chunks of disk. I realize a database is not the best example, but let's say it's a big, mostly read-only database.

This will work fine because the mtime changes every time a change is made.

-Chris


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 11:37 am 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3017
Location: Galloway, NJ
phvt wrote:
It doesn't sound like it's an LVM-based snapshot, and instead is running rdiff/rsync or something simpler. The original post says we'll need to dump databases to a file prior to the backup window.

I beg to differ :). This is a very sophisticated system which *does* involve LVM snapshots on the host. But what's the difference? Live database files can't be snapshotted, or copied, or rsync consistently, anyway.

saman007uk wrote:
Will this take an actual "snapshot" or will we need to stop MySQL, etc. in order to prevent file corruption.

Doing a snapshot does NOT always mean you're doing to get your mysql DB files in a consistent state. It can depend on which type of table engine you're using, for instance...

So dump your DBs to a file before the backup window just like you would with any other backup configuration.

-Chris


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 11:49 am 
Offline
Junior Member

Joined: Mon Sep 22, 2008 8:41 am
Posts: 48
Location: London, UK
caker wrote:
I beg to differ :). This is a very sophisticated system which *does* involve LVM snapshots on the host.


:D :D :D I like "very sophisticated" here!


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 11:55 am 
Offline
Senior Member

Joined: Tue Aug 12, 2008 8:41 am
Posts: 56
Just making it clear: ext4 disk images are not currently supported, correct?


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 12:01 pm 
Offline
Senior Member

Joined: Mon Feb 28, 2005 7:21 pm
Posts: 76
caker wrote:
I beg to differ :). This is a very sophisticated system which *does* involve LVM snapshots on the host. But what's the difference? Live database files can't be snapshotted, or copied, or rsync consistently, anyway.


My understanding is that LVM snapshots can be made of running InnoDB tables and restored successfully. The InnoDB tables will be in "crashed" state, but the data will recover automatically.

There's also the option to mysqldump --single-transaction with InnoDB and get a consistent backup without blocking applications.
http://dev.mysql.com/doc/refman/5.1/en/ ... ransaction

Other MySQL storage formats (like the default MyISAM) don't have this ability.

By "simple" I didn't mean simplistic, I meant like K.I.S.S.! :)


Top
 Profile  
 
PostPosted: Fri Apr 03, 2009 12:22 pm 
Offline
Newbie
User avatar

Joined: Fri Apr 03, 2009 12:18 pm
Posts: 3
I'm somewhat more interested in being able to just do a backup on demand (I backup my important files to AWS daily) and grab a copy of that backup for my own archives (i.e. burn a dvd set).

The first is taken care of by the user-selectable backup, but will the second be available? I.E. Will I ever be able to grab a disk image?

...Chris


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 03, 2009 12:22 pm 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3017
Location: Galloway, NJ
phvt wrote:
caker wrote:
I beg to differ :). This is a very sophisticated system which *does* involve LVM snapshots on the host. But what's the difference? Live database files can't be snapshotted, or copied, or rsync consistently, anyway.

My understanding is that LVM snapshots can be made of running InnoDB tables and restored successfully. The InnoDB tables will be in "crashed" state, but the data will recover automatically.

Yeah, but there's a potential snag when involving a virtualization layer:

We're issuing the LVM snapshot on the host -- not from within your Linode's kernel. Performing a snapshot flushes dirty buffers, but since that's being done on the host your Linode kernel is unaware that this is happening, so it still has dirty buffers that were never written to disk.

This may be moot because of journaling (both at the filesystem and the db level), but, speaking for myself, I'd dump my DBs to a file...

-Chris


Top
 Profile  
 
PostPosted: Fri Apr 03, 2009 12:24 pm 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3017
Location: Galloway, NJ
xoff00 wrote:
The first is taken care of by the user-selectable backup, but will the second be available? I.E. Will I ever be able to grab a disk image

Restoring restores to disk images, which you can then mount and pull down on your own with whatever (rsync, scp, tarball, etc).

-Chris


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic This topic is locked, you cannot edit posts or make further replies. Go to page 1, 2, 3  Next


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS
Powered by phpBB® Forum Software © phpBB Group

Home | Manager | Contact Us | Jobs | Terms of Service | Privacy Policy | ™ © 2003-2012 Linode, LLC. All rights reserved.