Hi all,

It appears my Debian squeeze slice is not starting sshd after rebooting. When I connect through Lish, I see that the booting freezes at some point, and won't give me a console. Searching Google for this problem is quite hard, as I can't find any clue to what could be wrong.

startpar: service(s) returned failure: hostname.sh ... failed!
INIT: Entering runlevel: 2                                                                                                                                      
Using makefile-style concurrent boot in runlevel 2.                                                                                                             
Starting portmap daemon...Already running..                                                                                                                     
Starting NFS common utilities: statd.                                                                                                                           
Starting enhanced syslogd: rsyslogd.                                                                                                                            
Starting database: mongodb.                                                                                                                                     
Starting supervisor: supervisord.                                                                                                                               
Starting message broker: rabbitmq-server.

I've tried changing the boot order in rc2.d -- for instance, moving mongo, supervisor and rabbitmq further down, or removing them altogether. This only results in freezing earlier and not loading those services, so my assumption is that it's not those services, and it must be sshd.

Syslog and auth.log do not contain any useful information either.

I'm out of ideas. Has anyone experienced this before? Are there any log files available for sshd or the booting process and any errors that might occur during booting? Any clues how to debug further?

Thanks in advance,
Bas

Do you really need portmap and NFS on a Linode?

Total guess but quite often long delays on boot are caused by non-functioning DNS. Check /etc/resolv.conf and maybe test with the google nameserver 8.8.8.8.

Better answer - Reads the logs under /var/log and find out what is going on. These should be something informative in /var/log/messages.

As a hint, if you can't boot to view log files, you can try booting into Finnix and mounting your linode's disks to access the log files on them.

Thanks guys.

I was able to boot into Finnix and access the log files. However, none of the logs -- syslog, messages, auth, apache's logs -- contained any helpful information. I started removing services from rc2.d and was able to narrow it down to Apache. After removing Apache from rc2.d, I was able to boot just fine. Calling */etc/init.d/apache2 start* manually starts up Apache without any issues or delays.

I'm not sure why Apache is halting the boot process. I thought it might be because networking isn't available at that time, but renaming S..apache to S91apache (making it the last item to be started up during boot) didn't help either.

As a workaround, I am now starting Apache with supervisord, which I was using already for other services. That causes some other issues: supervisor's child processes need to run in the foreground, which isn't possible with apachectl as far as I know, so I have to start the apache2 binary directly, which is a pain because envvars is only loaded with apachectl... Anyway, it works.

I guess the issue is sort of solved, although I'm still open for suggestions on tracking down the booting issues with Apache!