We're an ISP, we put our SMTP server on linode...

Lets say someone spoofs our clients IP. We would IP table the IP (as we did) to stop incoming traffic.

However, the client then has to use a different SMTP server.

Question:

How would you deal with this situation?

Note:

As an ISP, If the server was internal - we would be able to stop spoofing since we would know where the IP originates.

If someone hates you and knows you use linode, they coud send you tons of incoming traffic with spoofed IPs, you'd be forced to take in the traffic and pay the bandwidth...

Not use a box outside of your control to run a business critical app.

Get (or rent in a data center) a rack, put your OWN boxes in it, including a hardware firewall, and MANAGE your business services.

somehow i don't think linode would appreciate that answer, but i could be wrong...

Why?

Linode is a VPS provider. I would hope (and believe they) are honest and ethical enough to know when their service is not the best tool for the job.

A screwdriver doesn't cut it if you need a hammer, no matter how hard you pound with it.

they offer infrastructure as a service. The point of linode is to allow a business to reduce their hardware footprint by leveraging linode as the physical server manager.

I know most people on here are running 1 or 2 linodes for webservers , but not everyone is using linode that way, or at that scale.

We were asking to get some constructive conversation about this topic and not just a snarky low ball answer.

thanks.

Then lets see what ideas pop up to handle this scenario in a VPS environment.

Unless Linode starts offering an external Firewall service, I don't see how you can stop this problem since the packets have already passed Linodes counting and billing tracker before they hit anything that you control in order to filter or drop them.

Of course if they offer a Firewall service, then they're not exactly a UNmanaged VPS service.

What you're trying to do can't be controlled in a virtual environment. So you either except the limitations, or you step up your platform.

I don't get your snarky comment - this is real world advice. If you figured out how to turn the OSI model into some magically controlled system - please share.

anyone have something constructive to say?

It seems to me that vonskippy was being constructive, but that's just me...

To your original question, I'd probably implement some form of SMTP authentication and not worry about filtering at the IP level unless I really absolutely had to, in that case probably looking at something like fail2ban to make it as dynamic as possible. I would certainly try to avoid making clients change servers due to some third party. Yes, stray traffic might hit the server, and I'd have to worry about (D)DOS attacks, but that's part of running a public server. Heck, eventually IPv6 will make this moot anyway since there's no way you're going to track all the myriad of dummy addresses people are hitting you from.

Yes, Linode offers a form of infrastructure as a service, but as you say that gives you a virtual hardware footprint on their physical servers. The network fabric is common and your management control starts at your virtual server, not out at the network edge. That's just the nature of the service. I understand your desire to capture the errant traffic closer to the source, but that's where you leave the Linode model (as vonskippy pointed out), since there just isn't currently any configurable piece of hardware you have control over beyond your Linode, and by then the network traffic is accounted for.

It's not clear to me if the traffic was actually taking your system down (though I suspect not) or just accruing undesirably against your bandwidth. If the latter, I certainly sympathize with you, but am not sure there's any resolution that would address that issue. If it's a big enough attack that might be impacting the host itself, I expect Linode would definitely help take it down, or may even have some monitoring/tools that would trip automatically due to load, but otherwise I think you get to deal with it locally and count the traffic.

If it were me, I'd also try to decide if I really cared about the accounting, outside the theoretical. You may already have done this (or your bad traffic may in fact already be extreme in nature), but it seems to me it might not really hurt all that much, if you run some numbers. Odds are that your valid traffic is going to easily use more bandwidth than the failed attempts (except in extreme attacks).

If you shut the bad traffic down quickly (either blackhole the initial syn, or fail authentication early - within a round trip or so - in the protocol), it should take an awful lot of requests to put a significant dent in the bandwidth limits. At ~40 bytes for a TCP SYN packet that's 25 million connection attempts per GB. Assume a full connection (3-way handshake) and maybe 100 bytes of payload to fail to authenticate and we're still around 4-5 million.

Even if you buy additional Linodes just to add to your bandwidth limit, the plans scale at $10/100GB (with a minimum of $20 for the 512/200GB). So essentially each $10/mo (per 100GB) could cover 2.5 billion SYN connection attempts or ~450 million protocol failures (roughly) each month. That's ~900 (or ~175) connection attempts a second, every second, the entire month. That's probably good bang for the buck compared to dedicated hardware.

In summary, yes, Linode permits you to reduce your hardware footprint, but in return you lose some level of hardware control and are subject to its own accounting in terms of bandwidth. To the specific question of filtering out traffic before it accounts against your limits, I don't think there is any way to do that today. So you'll have to decide if the possibility of increased network load/charges - compared to dedicated equipment where you control the outer network layer - outweighs the benefits of the virtualized hardware and other management and cost efficiencies.

-- David

And what? Heh. You will still pay for incoming traffic, want you this traffic or not, anyway

With Linode you can in few minutes run new node with new IP and change DNS, or deploy the balancer node. With your own "real" server in datacenter you will spend hours to make same things.

Enforcing authentication doesn't stop the traffic from coming to you...

Let's say this, someone spoofs an ip of a client who needs to use your service, and let's say its 2mbps, which adds up fast... What do you do?

1. Allow the ip so your client can use your service while you eat the additional bandwidth.

2. Deny the client and spin up a new server linode and dedicate a node for them, which is not very scalabale.

3. Bring the server in house and defend with a router doing antispoofing?

Yes, "server in the house" is a most solid solution. And stable. And... ok, it was sarcasm.

With TCP, "spoofing" in the traditional sense isn't possible: bidirectional communication is required to set up the stream. The biggest threat is an accumulation of half-open connections in the queue, which can be mitigated with SYN cookies. You don't need to block the SYN, you just need to forget you sent the SYN+ACK unless you get an ACK back.

This would, in effect, be a SYN-flooding DoS attack with single (forged) origin IP. IP spoofing is difficult to do, useless with TCP, and easy enough to thwart that an attacker would probably just use real IP addresses and bring connections up all the way (a real DDoS attack!) if they wanted to make an impact.

You mention you're an ISP... if you want to be quite sure you can trust the source IP address, you could tunnel traffic from your customer-facing network(s) to your servers and drop anything arriving via the Internet that ought to be coming via the tunnel. This would make your Linode a part of your internal network, in a way, and allow you to use similar ingress/egress policies. Bonus points for giving it a customer-facing IP within your AS...

_________________

/* TODO: need to add signature to posts */

... and no matter which way you cut the mustard, you're going to have to pay for some amount of undesired traffic. 2 Mb/sec is greater than average, of course, but it happens. Keep an eye on it and purchase extra early and often: at 2 Mb/sec, that's 642 GB/mo, or about $64/mo.

I wouldn't lose sleep over it, although I would wonder what the hell you and/or that customer did to piss someone off enough to execute something so difficult and contrived, just to burn through a couple bucks each day...

_________________

/* TODO: need to add signature to posts */

How much traffic are you really talking about?