Hi,

I rebooted my linode today and now it won't come back up. There is kernel panic on bootup with the following message that a helpful support rep was able to provide. I hadn't rebooted my linode in about 2 months, running Arch Linux. The only change that I made to init was switching to systemd from initscripts last year and had done a couple of reboots since with no issues. Would really appreciate if someone can throw a pointer as to where I can start with this. I tried a different kernel also (3.6.5), which was the last kernel i was using but no luck.

/bin/sh: nodevfs: No such file or directory
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00

Pid: 1, comm: sh Not tainted 3.7.10-linode49 #2
Call Trace:
[<c078a017>] ? panic+0x87/0x177
[<c013795a>] ? forget_original_parent+0x32a/0x340
[<c015b569>] ? sched_move_task+0x89/0x110
[<c013797e>] ? exit_notify+0xe/0x120
[<c0137c13>] ? do_exit+0x183/0x3a0
[<c01d4f30>] ? vfs_write+0x140/0x170
[<c0137e6c>] ? do_group_exit+0x3c/0xa0
[<c0137ee1>] ? sys_exit_group+0x11/0x20
[<c078c844>] ? syscall_call+0x7/0xb
[<c0780000>] ? klist_put+0x20/0x80

As far as package upgrades go, I have sifted through pacman.log and only notice these packages that could have something to do with it, maybe filesystem/glibc package that could be an issue. But I am not sure. Any help appreciated, thanks in advance.

[2013-02-17 10:52] upgraded linux-api-headers (3.7.1-1 -> 3.7.4-1)
[2013-02-17 10:53] upgraded filesystem (2012.12-1 -> 2013.01-3)
[2013-02-17 10:53] Generating locales...
[2013-02-17 10:53] en_US.UTF-8... done
[2013-02-17 10:53] en_US.ISO-8859-1... done
[2013-02-17 10:53] Generation complete.
[2013-02-17 10:53] upgraded glibc (2.17-1 -> 2.17-3) 
[2013-02-17 10:53] upgraded file (5.11-1 -> 5.11-2)
[2013-02-17 10:53] upgraded findutils (4.4.2-4 -> 4.4.2-5)
[2013-02-17 10:53] upgraded gcc (4.7.2-3 -> 4.7.2-4)
[2013-02-17 10:53] upgraded libksba (1.2.0-1 -> 1.2.0-3)
[2013-02-17 10:53] upgraded gnupg (2.0.19-4 -> 2.0.19-6)
[2013-02-17 10:53] upgraded grep (2.14-1 -> 2.14-2)
[2013-02-17 10:53] upgraded groff (1.21-2 -> 1.22.2-2)
[2013-02-17 10:53] upgraded gzip (1.5-1 -> 1.5-3)
[2013-02-17 10:53] upgraded hwids (20121203-1 -> 20130214-1) 

fstab

 GNU nano 2.2.6 File: etc/fstab
# /etc/fstab: static file system information.
#
# <file system>         <mount point> <type> <options> <dump> <pass>
proc                     /proc         proc   defaults   0      0
/dev/xvda                /             ext3   noatime,errors=remount-ro    0   1
/dev/xvdb                none          swap   sw         0      0 

i don't know anything about arch, but in my fstab, i have two extra lines

/sys /sys sysfs defaults 0 0
none /dev devtmpfs rw 0 0

Thanks so much for this tip, I am going to try this right now and report back.

Nope that didn't do the trick, same kernel panic as before.

/bin/sh: nodevfs: No such file or directory

grep -r nodevfs /etc

btw, it might be quicker if you jump on irc

I think I found the culprit, for some really dumb reason I removed the systemd-sysvcompat package. Now I tried to install it via chroot, but getting the error "Could not determine filesystem mount points". I think due to the symlink of /etc/mtab -> /proc/self/mounts, which is not available in the chroot environment. Would manually extracting and dropping the files from the package work?

THAT WAS IT!!!!! I extracted the files from the systemd-sysvcompat package and cp/mv them into /sbin. Rebooted and VOILA!!

awesome, re the chroot, say your xvda root is mounted in /mnt

mount --bind /dev /mnt/dev
mount --bind /dev/pts /mnt/dev/pts
mount --bind /sys /mnt/sys
mount --bind /proc /mnt/proc
chroot /mnt

for next time