scaling image uploads from one to two servers + redundancy

Problem:

1 server that takes uploads or images/attachements that need to be serverd and the server has:

a) no redundancy; and/or

b) is reaching CPU/Memory capacity for delivering Pages

Grab another linode for a second identical web server and nodebalancer.

Set up lsyncd both ways between each webserver (the rsync checks the timestamp the second time so you don't get in a bit loop. TIAS).

If you've got heaps of directories you may need something like this to increase the number of kernel inotify watches:

/etc/sysctl.d/notify-sync.conf

fs.inotify.max_user_watches = 1024000

Then set up the nginx fallback for a file not found to proxy through to the other web server to eliminate the timing hole where an image is uploaded but not yet copied to the other server.

  location ~ ^/img/(.............)\.jpg {
     expires 2592000;
     add_header Cache-Control public;
     alias /var/www/images/$1.jpg ;
     error_page 404 @fallback;
  }

location @fallback {
    proxy_set_header  X-Real-IP  $remote_addr;
    proxy_pass http://myotherserver;
}

A similar proxy configuration on another web server works the same way.

Now you have a redundant web server. I've got two configurations like this that work well.

originally from: http://forum.linode.com/viewtopic.php?f … 792#p57089">http://forum.linode.com/viewtopic.php?f=20&t=9792#p57089