After this afternoon's power mishap at Hurricane Electric, my linode came back up with all its daemons... except sshd.

I logged into the console and tried starting it manually by typing:
cd /etc/init.d
./ssh start

It outputs the usual message:
Starting OpenBSD Secure Shell server: sshd.

But there's no sshd running. /var/log/auth.log shows:
May 3 19:26:08 www sshd[446]: fatal: daemon() failed: Success

I have no idea what that means. Anyone have any clues? Corrupted file somewhere perhaps? (*shudder*)

I'm running Debian 3.0, kernel 2.4.26-linode26-1um

Thanks,
Jeff

I'm also running Debian 3.0, so these may be of interest to you. They're the MD5 checksums of all the binaries in the ssh package.

You can just drop this in a file and run "md5sum -c the_file" and it'll test all the listed files.

20e4cf1667b368f28ecb391fe26b3f91  /usr/bin/ssh
a3061844bb22f787d0968fb1ef55236d  /usr/bin/scp
9ab332fb47d037668a72ca6bc675c0d2  /usr/bin/ssh-add
e7f5120c1aa494358c7b95a102e59cfd  /usr/bin/ssh-agent
1164bff544f62150af956700fd7d61fd  /usr/bin/ssh-keygen
6dc707515b5e39e39af1d2e80fc2dade  /usr/bin/ssh-keyscan
9d23427ba0502bfad7349098043bfc29  /usr/bin/sftp
a36ef875ba1c83e0c6d7cbf276e7f0f0  /usr/bin/ssh-copy-id
b7040cbe02fb23e8b8ed02d46824629e  /usr/sbin/sshd
d2288b95c8e1973f920d4adebf6e27fc  /usr/lib/ssh-keysign
e0cd7a8ba000cea530ad8519cd273c8f  /usr/lib/sftp-server

Thanks! The md5sums check out okay.

Try running sshd in debug mode:

sshd -d

Also, by chance does /dev/null not exist?

-Chris

It works in debug mode!

Ah, it's a /dev/null problem. For some reason, it turned into a regular file after the power reboot, so I found some documentation that says to re-create it:

rm -f /dev/null
mknod -m 666 /dev/null c 1 3

sshd starts up now! Thanks much for the help, everyone.

One last question. I noticed this in my /var/log/dmesg:

EXT3-fs: ubd(98,0): orphan cleanup on readonly fs
ext3_orphan_cleanup: deleting unreferenced inode 17229
ext3_orphan_cleanup: deleting unreferenced inode 15141
ext3_orphan_cleanup: deleting unreferenced inode 8101
EXT3-fs: ubd(98,0): 3 orphan inodes deleted

Is there any way to tell what those inodes were for, in case anything else is broken that might not have shown up yet?

Well, unreferenced inodes by definition weren't associated with any filenames, so the only way would be to be able to get to the data. Unfortunately, I don't really know how you would do this - I expect it would be something to do with debugfs, though. Please make sure you have a backup before doing this though, obviously.

hi guys

i just found this thread whilst looking for a solution to my own sshd not starting issue and the above solution worked. this was a new install of ssh on debian sarge (3.1), ssh was installed the easy way with apt and wouldn't work at all until i recreated /dev/null/

thanks to everyone who helped here, you've saved me hours more stress...

_________________
each one teach one...