Also the output of the command iptables -nvL is as follows.

Chain INPUT (policy ACCEPT 4508K packets, 5346M bytes)
pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 3996K packets, 8098M bytes)
pkts bytes target prot opt in out source destination

OK, my statement about the network related to your Linode. If it is blocking hosts which attempt to connect via ssh, then it doesn't matter where the client is coming from - it will be blocked.

It appears that you have no firewall rules, so we can leave the network question aside for the moment.

It is very odd that attempting to re-install sshd did not work. I assume you were running these commands as root? To do so, you need to be logged in as root or put sudo in front of the command.

You can try to fix the problem (as root) with the commands mkdir -p /etc/ssh and apt-get --reinstall install openssh-server. This will create the missing directory that apt-get was complaining about and attempt to re-install sshd.

Hi all --

We are having a 100% identical issue as the original poster.

(1) We have a 512 box with the same Linode's deployment of Ubuntu 10.04 LTS.

(2) All the sudden, SSH started rejecting all login attempts.

(3) Lish does work.

Also, the last time we were able to login via SSH was exactly four days ago, just one day before before this thread was started. So it seems like the problem is more global as all symptoms are absolutely identical.

maxim, were you able to resolve the problem?

I also realized that we have a second LTS 10.04 Linode box -- just checked it, and yes, it's having the same SSH problem, too!

So the issue is consistent across many (all?) Linodes with LTS 10.04

Can it be related to some kind of automatic update from Ubuntu?

I checked two 10.04 LTS Linodes, one an upgrade and the other a new install, and both seem to be working well. Have folks tried the -v option on the ssh client, to print out debugging info? In particular, the "remote software version" and "authentications that can continue" lines will be interesting.

Here's what I see, on the not-upgraded install,

rtucker@tremens:~$ ssh -v sapling.rocwiki.org
OpenSSH_5.9p1 Debian-5ubuntu1, OpenSSL 1.0.1 14 Mar 2012
debug1: Reading configuration data /home/rtucker/.ssh/config
debug1: /home/rtucker/.ssh/config line 10: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to sapling.rocwiki.org [2600:3c03::13:3b01] port 22.
debug1: Connection established.
debug1: identity file /home/rtucker/.ssh/id_rsa type -1
debug1: identity file /home/rtucker/.ssh/id_rsa-cert type -1
debug1: identity file /home/rtucker/.ssh/id_dsa type -1
debug1: identity file /home/rtucker/.ssh/id_dsa-cert type -1
debug1: identity file /home/rtucker/.ssh/id_ecdsa type -1
debug1: identity file /home/rtucker/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3p1 Debian-3ubuntu7
debug1: match: OpenSSH_5.3p1 Debian-3ubuntu7 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA 50:fe:ae:71:fe:a4:2b:40:97:52:0e:dc:ef:e0:27:03
debug1: Host 'sapling.rocwiki.org' is known and matches the RSA host key.
debug1: Found key in /home/rtucker/.ssh/known_hosts:245
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/rtucker/.ssh/id_rsa
debug1: Trying private key: /home/rtucker/.ssh/id_dsa
debug1: Trying private key: /home/rtucker/.ssh/id_ecdsa
debug1: Next authentication method: password
rtucker@sapling.rocwiki.org's password: 
debug1: Authentication succeeded (password).
Authenticated to sapling.rocwiki.org ([2600:3c03::13:3b01]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LC_MESSAGES = en_US.UTF-8
debug1: Sending env LC_COLLATE = en_US.UTF-8
debug1: Sending env LANG = en_US.UTF-8
debug1: Sending env LC_CTYPE = en_US.UTF-8
Linux sapling 3.0.4-linode38 #1 SMP Thu Sep 22 14:59:08 EDT 2011 i686 GNU/Linux
Ubuntu 10.04.4 LTS

And on the upgraded install,

rtucker@tremens:~$ ssh -v framboise
OpenSSH_5.9p1 Debian-5ubuntu1, OpenSSL 1.0.1 14 Mar 2012
debug1: Reading configuration data /home/rtucker/.ssh/config
debug1: /home/rtucker/.ssh/config line 10: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to framboise [2600:3c03::f03c:91ff:fe96:1dc9] port 22.
debug1: Connection established.
debug1: identity file /home/rtucker/.ssh/id_rsa type -1
debug1: identity file /home/rtucker/.ssh/id_rsa-cert type -1
debug1: identity file /home/rtucker/.ssh/id_dsa type -1
debug1: identity file /home/rtucker/.ssh/id_dsa-cert type -1
debug1: identity file /home/rtucker/.ssh/id_ecdsa type -1
debug1: identity file /home/rtucker/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3p1 Debian-3ubuntu7
debug1: match: OpenSSH_5.3p1 Debian-3ubuntu7 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA cd:91:1b:76:45:4c:90:c7:f7:c5:e3:0e:b0:33:a3:55
debug1: Host 'framboise' is known and matches the RSA host key.
debug1: Found key in /home/rtucker/.ssh/known_hosts:18
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Trying private key: /home/rtucker/.ssh/id_rsa
debug1: Trying private key: /home/rtucker/.ssh/id_dsa
debug1: Trying private key: /home/rtucker/.ssh/id_ecdsa
debug1: Next authentication method: keyboard-interactive
Password: 
Duo two-factor login for rtucker

<<redacted>>
debug1: Authentication succeeded (keyboard-interactive).
Authenticated to framboise ([2600:3c03::f03c:91ff:fe96:1dc9]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LC_MESSAGES = en_US.UTF-8
debug1: Sending env LC_COLLATE = en_US.UTF-8
debug1: Sending env LANG = en_US.UTF-8
debug1: Sending env LC_CTYPE = en_US.UTF-8
Success. Logging you in...
Linux framboise 3.0.4-linode38 #1 SMP Thu Sep 22 14:59:08 EDT 2011 i686 GNU/Linux
Ubuntu 10.04.4 LTS

_________________

/* TODO: need to add signature to posts */

No not yet... The issue still persists.. Thank God the problem is seen by other people as well. We were pretty stumped thinking what went wrong.

I'm looking for a post that I was reading awhile back that some kind of Malware was locking out ssh..I will report back when I find it.

Found the post.

http://www.linuxquestions.org/questions ... ts-340366/


Edit: I'm not saying that this is the issue,but its worth looking into.

Ran tail /var/log/messages:

May 6 07:30:44 li374-19 kernel: VFS: Mounted root (ext3 filesystem) readonly on device 202:0.
May 6 07:30:44 li374-19 kernel: devtmpfs: mounted
May 6 07:30:44 li374-19 kernel: Freeing unused kernel memory: 388k freed
May 6 07:30:44 li374-19 kernel: Write protecting the kernel text: 5984k
May 6 07:30:44 li374-19 kernel: Write protecting the kernel read-only data: 1432k
May 6 07:30:44 li374-19 kernel: NX-protecting the kernel data: 3232k
May 6 07:30:44 li374-19 kernel: udev: starting version 151
May 6 07:30:44 li374-19 kernel: udevd (1036): /proc/1036/oom_adj is deprecated, please use /proc/10
36/oom_score_adj instead.
May 6 07:30:44 li374-19 kernel: Adding 262140k swap on /dev/xvdb. Priority:-1 extents:1 across:262
140k SS
May 6 07:30:44 li374-19 kernel: EXT3-fs (xvda): using internal journal

I don't know what it means exactly, but it looks like something happened to the kernel?

@cbrands
Not really sure what it is. However I havent seen anything of that sort on my linode logs.

@hoopycat, here is the debugging info i see with the -v.

[root@li40-97 ~]# ssh -vvvv root@19.x.x.x
OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 19.x.x.x [19.x.x.x] port 22.

..and it times out

That's a Ford IP address.... *confused*

_________________
Rgds
Stephen
(Linux user since kernel version 0.11)

It looks like it rebooted at about 07:30. That would be "something" but probably not directly the cause of the problem. (Unless sshd isn't starting on boot...)

_________________

/* TODO: need to add signature to posts */

Oh, you're trying to be cute and make people's life hard by obscuring details. See hoopycat's sig as to why this is a bad idea. Since you included a hostname in an earlier post, I'm gonna guess you really meant to use 96.126.125.19

You'll always get a timeout trying to use the Ford IP address 'cos it's not reachable from the internet.

_________________
Rgds

Stephen

(Linux user since kernel version 0.11)

I guess I accidentally tested the wrong IP. But here is what I see through: "cat /var/log/auth.log". It opens/closes all SSH connections:

May 6 14:09:01 li374-19 CRON[3101]: pam_unix(cron:session): session opened for user root by (uid=0)
May 6 14:09:01 li374-19 CRON[3101]: pam_unix(cron:session): session closed for user root
May 6 14:17:01 li374-19 CRON[3121]: pam_unix(cron:session): session opened for user root by (uid=0)
May 6 14:17:01 li374-19 CRON[3121]: pam_unix(cron:session): session closed for user root
May 6 14:39:01 li374-19 CRON[3146]: pam_unix(cron:session): session opened for user root by (uid=0)
May 6 14:39:01 li374-19 CRON[3146]: pam_unix(cron:session): session closed for user root
May 6 15:09:01 li374-19 CRON[3189]: pam_unix(cron:session): session opened for user root by (uid=0)
May 6 15:09:01 li374-19 CRON[3189]: pam_unix(cron:session): session closed for user root

These are cron messages; nothing to do with ssh.

_________________
Rgds

Stephen

(Linux user since kernel version 0.11)