My employer got DDOS'd yesterday - thousands of simultaneous requests from someone using the 80legs spider (which ignored robots.txt).

I had to deal with the assault by blocking addresses via iptables - individually at first, then eventually using a published list of netblocks registered to Russia and Ukraine - we banned 13,000 netblocks. I dealt with the rest using an Apache RewriteRule that matched the user-agent.

To be better able to scale in the future, we've decided to migrate to a NodeBalancer setup with at least two back-end servers. Yet this raises some concerns about reacting to DDOS attacks.

How would we block a range of malicious remote IPs without root access? IPTables can handle thousands of rules, but apache's config files likely wouldn't.

mod_evasive, which we've recently installed in Apache, can take countermeasures against an IP that sends too many requests too quickly - but this won't work if they're all coming from the same place.

Are there tools for dealing with DDOS attackers in a Nodebalancer setup?

Linode is not a DDoS mitigation service; your only course of action is to scale up your infrastructure sufficiently to deal with the load. NodeBalancer, for its part, scales up to 10,000 simultaneous connections. Beyond that, you'll need to use a dedicated DDoS mitigation service. Real ones cost a lot, but can absorb massive attacks. There are some services like CloudFlare that might help. You can put CloudFlare between your linode (or node balancer) and the internet. They have some tips:

http://blog.cloudflare.com/ddos-prevent ... the-origin

Thanks for the Cloudflare suggestion, I'll look into them. But, my inclination is to keep everything on Linode rather than adding another layer hosted elsewhere.

If there's no ability to use something like iptables on a NodeBalancer, then that cuts down on our ability to deal with attacks.

And we expect to have more like this in the future - the attacker was probably trying to clone our site through content scraping, and we're in an industry where this sort of thing happens a lot. Yesterday's incident was the third in a year. The smarter content thieves will scrape a site slowly and go undetected; the stupid ones that are more dangerous as they'll think nothing of unleashing a few thousand scrapers at the same time.

I think I may end up getting another regular Linode, where I'll have root & ability to use iptables, and install something like HAProxy on it. It's less convenient, but we'll have more control.

I highly recommend cloudflare. You need something off the Linode stack because you do run the risk of getting your IP blocked for 24 hours if you start affecting other customers.

null routed != banned.

It's really best for everyone involved when a null route is applied. Null routes are only applied when an attack is large enough to negatively affect our infra or other customers. This means that even if we didn't null route your Linode's IP for an attack that should be null routed, your shiz is probably down anyways. We check null routes constantly to make sure that we restore service ASAP.

_________________
me | voltaireMC

If a Linode is exposed via CloudFlare, then the Linode's IP is a secret, and a DDoS attack can't impact other customers beyond whatever traffic CloudFlare does forward. If the level of forwarded traffic does become sufficient to cause the IP to be null-routed, then CloudFlare can still present a static copy of the site since the Linode will appear to be down. This is presuming that the DDoS isn't large enough to take out CloudFlare itself, but since DDoS mitigation is one of their selling points, that'd be rather difficult.

If you want to survive DDoS attacks without interruption of service, Linode alone is not the solution. They're a fantastic host, but they're not a DDoS mitigation service.

Changed my wording after you replied apparently