I noticed my outgoing traffic went from nearly 0 to a constant 1k/s outgoing. Also my disk I/O is now about 8 blocks/sec.

They aren't coming from http requests, I think it's from ssh requests because I found endless entries like this one from /etc/log/auth.log:

Nov 10 12:10:15 li200-196 sshd[32075]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:10:43 li200-196 sshd[32078]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:11:13 li200-196 sshd[32081]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:11:42 li200-196 sshd[32084]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:12:12 li200-196 sshd[32087]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:12:41 li200-196 sshd[32090]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:13:11 li200-196 sshd[32093]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:13:41 li200-196 sshd[32096]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:14:11 li200-196 sshd[32099]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:14:40 li200-196 sshd[32102]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:15:10 li200-196 sshd[32105]: refused connect from 222.143.26.249 (222.143.26.249)
Nov 10 12:15:39 li200-196 sshd[32108]: refused connect from 222.143.26.249 (222.143.26.249)

I'm using denyhosts python script and this IP is already in my /etc/hosts.deny file, but it's causing constant io from all of the requests. What should I do from this chinese IP constantly pinging my server for ssh?


edit: I ended up using this command to block all incoming requests from that ip address:

ufw insert 1 deny from 222.143.26.249

So no more logging of the requests by denyhosts and auth.log, so should fix my problem.

I use ubuntu's ufw as a firewall, here are my current rules:

To                         Action      From
--                         ------      ----
22                         ALLOW       Anywhere
80                         ALLOW       Anywhere
Apache Full                ALLOW       Anywhere

Thinking of adding something like: ufw insert 0 deny in on port 22 from 222.143.26.249

But I'm still trying to figure out ufw's syntax. Any help is appreciated, especially if you don't think adding a firewall rule will help

edit: after adding a deny from all command (see above post) in ufw this is my new rule list:

#ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] Anywhere                   DENY IN     222.143.26.249
[ 2] 22                         ALLOW IN    Anywhere
[ 3] 80                         ALLOW IN    Anywhere
[ 4] Apache Full                ALLOW IN    Anywhere

This is a normal consequence of the way denyhosts works. The daemon (sshd in this case) consults hosts.deny, then acts based on its contents. The result is a bunch of "refused" messages from sshd in your logs.

As you've discovered, adding a firewall rule prevents attempts from the designated IP address from even reaching sshd. Your approach is reasonable to take for seriously offending IPs that you don't ever expect to have a legitimate person behind.

If it is somehow important to you not to see these messages in your logs, you could look into fail2ban. It works by inserting firewall rules rather than using hosts.deny. I've used both, and denyhosts seems more stable (although fail2ban hasn't crashed on me for some time now).

You can also change your ssh to a random high port and also change the ufw setting from allow to limit.

A random port isn't security, but it will keep your logs cleaner.

Thanks for the feedback guys! It's always tough as an inexperienced linux admin to make these kinds of calls, so any insight into others' thought processes is greatly appreciated.

My concern was that the increase in disk i/o and incoming bandwidth in my linode dashboard graphs would mask future problems if I just get used to the new baselines caused by that pesky ip. Does anyone else watch those like a hawk looking for spikes? Right now I don't have any real traffic, but I'm going to be hosting production code soon so I'm trying to work out the kinks.

my limited experience on a no-traffic server is so far that denyhosts is fine, with the occasional need to ban an ip. I looked into firewall rules to deny all from certain countries (especially china), but I hear that's only effective against those not smart enough to run through a proxy (which so far is most of the snoopers in my logs). Instead I think I'm going to go with several fellow linoders' advice and switch to a high port for ssh and just block port 22 altogether.

Are there any programs that will detect a port scan from an IP and block it before it can find the new port sshd is listening on? maybe that's a setting in fail2ban, or I bet iptables/ufw has a setting for that. I also heard something about port sinks/traps? I'll have to research that more. I'll update this threat when I figure out what I'm doing next, so far no breaches! <fingers crossed>

the ufw limit setting is exactly what I'm looking for. The linode community rocks! I'll be switching careers to security expert in no time

Not really. I measure a lot of things, but the only thing I watch really closely is actual performance of the service being provided. If that starts tanking, then I look at all the graphs.

(That's not to say I avoid looking at the graphs unless there's a problem; I've got a browser tab open to munin and a tail -F running on the log server most of the time, but between Nagios, Pingdom, and the best users in the industry, it's not the first line of defense.)



There are no humans behind the ssh connections, nor are they isolated to particular geographic regions. It's a worm (or, more accurately, a number of different worms). It spreads on its own, and while it might report its findings somewhere, it's autonomous. Do not attempt to anthropomorphize it, nor should you attempt to out-clever it. Only way you can "win" is by making it impossibly difficult.

Also remember that "once in a million years" happens annually per each million computers.

_________________

/* TODO: need to add signature to posts */

sudo ufw deny from 222.143.26.249

That will block 'em.

ufw also has built in rate-limiting:

 sudo ufw limit ssh

I don't know about ufw, but with iptables you could rate limit to 2 times per minute with the following:

iptables -A INPUT -p tcp --dport 2200 --syn -m limit --limit 1/m --limit-burst 2 -j ACCEPT
iptables -A INPUT -p tcp --dport 2200 --syn -j DROP


where 2200 is the ssh port, I would suggest, as others have, changing it to something other than 22 to stop the stupid bots that only go for 22. The second line drops subsequent attempts past the rate limit.

_________________
-Abzstrak