jaden wrote:
That's a good point, with a NodeBalancer in each datacenter, DNS failover would be great (assuming data replication was in place). I've thought about switching to another database entirely, like MongoDB, that appears to have better replication support, but that means rewriting the DB layer of the application and figuring out how to get MongoDB set up and running smoothly.
Yeah, I'm not too keen on doing that yet.
We primarily use Django, and although our dataset would fit nicely and
efforts are afoot to bring non-relational database support to the ORM, I'm probably not going to use it until it hits trunk.
Quote:
Do you currently use MySQL master-slave replication across datacenters? If so, have you had (m)any issues with it? From my reading it sounded like setting up a VPN or SSH tunnel was the preferred way to not have to open MySQL up and keep your data secure, but I have yet to set it up.
Not yet, but I do have code and procedures in place to do it. Turns out that this particular platform is operating on Rackspace Cloud Servers, and (for reasons unknown) location is an account-wide setting. It's easier for us to deploy a new instance on Linode than it is to deploy in a different Rackspace datacenter, which is kinda my medium-term plan
To facilitate this, we store nightly mysqldumps on S3. We use InnoDB exclusively (if you're going to have a relational database, you might as well have referential integrity and transactions!) and have binlogging enabled by default, so throwing some additional options on the mysqldump gives us an exact snapshot at a specific point, along with a CHANGE MASTER TO command that points to the exact correct point to start replication:
Code:
/usr/bin/mysqldump --defaults-extra-file="/etc/mysql/debian.cnf" --single-transaction --quick --master-data=1 --all-databases --add-drop-database > <%= @localfile %>
/usr/bin/s3cmd del <%= @s3file %>.old
sleep 1
/usr/bin/s3cmd --acl-private mv <%= @s3file %> <%= @s3file %>.old
sleep 1
/usr/bin/s3cmd --acl-private put <%= @localfile %> <%= @s3file %>
sleep 1
(Yes, that needs better error handling. Urk.)
Deploying a new slave, whereever it is, consists of pulling this file from S3. Good news is that almost everything on the site today was prepared last night, so this morning's backup would be almost entirely complete.
Speaking of server-id, we generate it based on the public IP address of the database server, which damn well better be unique.
Code:
set_unless[:mysql][:replication][:server_id] = "#{node[:ipaddress].split('.').collect(&:to_i).pack('C*').unpack('N').first}"Also, MySQL supports SSL natively, so you can skip the VPN/SSH stuff
if you do it right. Here's
some documentation on it. You're going to want to use certificates and some sort of trust architecture.
Quote:
I like the idea of having read-only mode be the norm, and special-casing writes. I may just do that too
They tell me a "Service-Oriented Architecture" is the way to go. I think they might be right.
Heck, might want to do load-balancing between two or more locations for the primary read-only stuff (
www.example.com), while keeping writable stuff (checkout.example.com, comments.example.com, stalking.example.com, etc) localized to the master site... hmm... maybe have the master site at Rackspace Chicago with read-only sites at Linode Fremont and Atlanta... *twirls mustache*
_________________
Code:
/* TODO: need to add signature to posts */
FAQ
Search
Members
Register
Login [ Anonymous ]
