Someone managed to exploit code on your website and got it to run a command which downloaded a malicious Perl script and opened a port on the server (port 3131, if you want to block it in iptables) which gives anyone connecting to it a shell as whichever user the script was run under - in this case, www-data. You can see that there's a shell open:
Quote:
www-data 9694 9693 0 15:25 ttyp0 00:00:00 sh -i
After getting a shell your attacker most likely used wget to download files like elflbl and mremap2. I don't know what they do but odds are very likely your attacker will have downloaded a rootkit and attempted to use it too.
Much as it's tempting to kill the scripts first, it's a good idea to perform a little research so you can find the executables. The /proc filesystem will tell you (among other things) where the executables are, by way of a symlink, which can be very handy. For example, running this command as root should tell you where the
mremap2 executable is located:
Code:
# ls -l /proc/9713/exe
After you've figured out where all the suspect executables are, you can now kill all those scripts. As root:
Code:
# killall elflbl mremap2 cgi
# kill 6814 6812
(if killing them this way doesn't work, try again using "-9" after the kill command)
Then to prevent them connecting to the same code again, block port 3131: (It's by no means a proper solution as the port is trivially changed, but it does buy you more time.)
Code:
# iptables -A INPUT -p tcp --dport 3131 -j DROP
Now, if you don't already have it, install
chkrootkit. In fact, even if you do you already have it, uninstall it and reinstall it again to make sure you've got good binaries. Some distros come with it and others have an easy method of installing it (emerge, apt-get, urpmi, etc).
However, before running it, stop your network support entirely and continue working in lish. How you do this depends on your distro but "/etc/init.d/network stop" is normally a good bet. Failing that, "ifconfig eth0 down" should do the trick. Remember that if you're connected via SSH you won't receive a prompt after doing this so you'll need to connect to the lish console to carry on. (This is so that nobody can take advantage of your server while you're investigating it).
After running chkrootkit, it'll tell you if you have any 'infected' programs. Obviously you shouldn't use just one tool to tell if you have any, but I'm not sure of any others.
If you don't have any, the next step is to find the directory where your attacker uploaded this stuff to. Mostly these would be directories that can already be uploaded to - for example, forums have avatar upload functions that could theoretically be vulnerable to an attack like this. If you took note of the location of the executables earlier you've probably already nailed it. Otherwise, You're looking for files named
elflbl,
mremap2 or
cgi specifically, although there'll probably be others too. Also, be aware that they may be scattered around the drive - attackers like to do that.
It may also be worth checking the .bash_history file for root or any other users which may be affected (especially www-data, if you have one for that user). This may not reveal much though as logging to .bash_history files is easily disabled and the files are also easily editable by the user - plus, if you had to kill the sh session with -9 above, it won't have been able to write to the history file.
Before bringing the system back up you should also change your root password and double-check your ps listing to make sure there aren't any suspicious processes, especially ones owned by www-data or root.
You may think this sounds like a lot but in actual fact it's not a lot at all. Your attacker may have edited the /etc/init.d/ or crontab files to download the files again at startup or at regular intervals. chkrootkit may not have detected a rootkit when there actually is one... etc, etc. If you want to be truly secure, back up your data (making sure it *is* just data - no binaries, etc) and reinstall the OS from the Distribution Wizard. Then restore your data and other settings. Yes, it takes a long time but unfortunately it's hard to make sure you're utterly clean otherwise.
Of course, you should also make sure you upgrade whatever script was vulnerable to the attack in the first place otherwise you'd just go through all this again next time round too.
People might call me paranoid. But when it comes to security it's better to be paranoid than to be lax. In my opinion, anyway.
FAQ
Search
Members
Register
Login [ Anonymous ]
Anyway, running the command can give you some insight into whether there are any 'bad' servers running (like the Perl script in this thread).