Thank you, irgeek! I'm trying it now (as I write this) and will include notes. It's not quite a how-to (although hopefully I'll be able to make one of those, too), but at the very least I hope it will provide additional documentation for anyone else trying to do this.
So I'm starting with a fresh Debian 5.0 (Lenny) kernel, making a new Linux distribution in the Linode Manager and using the "Latest 2.6 Series" kernel, 2.6.18.8-lnode16. I have /dev/xvdh pointed to a disk image I've created and copied between two linodes.
I'm installing packages needed for kernel work (per irgeek's recommendation):
Code:
apt-get install build-essential flex bison wget unifdef
Then retrieving kernel source, untar'ing it, building the kernel, making the headers, and creating the symlink:
Code:
cd /usr/src
wget http://www.linode.com/src/2.6.18.8-linode16.tar.bz2
tar -jxf 2.6.18.8-linode16.tar.bz2
cd 2.6.18.8-linode16
make -j4
make headers_install
cd ..
ln -s 2.6.18.8-linode16 linux
Then grabbing the DRBD8 sources, untar'ing them, and building the module:
Code:
wget http://oss.linbit.com/drbd/8.3/drbd-8.3.1.tar.gz
tar -zxf drbd-8.3.1.tar.gz
cd drbd-8.3.1
make KDIR=/usr/src/linux/
make install
(Can I just say thank you again, irgeek? Everything has worked perfectly in Lenny exactly as you described.)
And then... modprobe fails.
Code:
# modprobe drbd
FATAL: Error inserting drbd (/lib/modules/2.6.18.8-linode16/kernel/drivers/block/drbd.ko): Invalid module format
My understanding is that an "invalid module format" often means you're not building the module against the kernel you're actually running. I suppose this is possible, but I believe I've grabbed the correct sources and they match the selection I've made in creating my linode's kernel distribution.
However, I've seen something online suggesting that even the "extraversion" information on the kernel can cause this, and that perhaps I need to "build the kernel the Debian way" using make-kpkg with "--append-to-version -linode16".
So to try this, then, I'm running an apt-get for tools used in the Debian way of kernel building. This actually installs 24 packages.
Code:
apt-get install kernel-package libncurses5-dev
Then I go back to the kernel source directory and try building again. I'm skipping kernel configuration and jumping right to the make-kpg. My understanding is that make-kpkg basically does "make dep; make clean; make bzImage; make modules" but I don't know where the "--append-to-version" switch gets placed, so I'm using the whole package.
Code:
make-kpkg --append-to-version=-linode16
It was quick, painless, and ran without errors. So now I'll go back to drbd and try again...
Code:
cd /usr/src/drbd-8.3.1
make KDIR=/usr/src/linux/
make install
modprobe drbd
And I still get an invalid module format.
So I start poking around, and try a modinfo on the new drbd.ko file:
Code:
# modinfo /lib/modules/2.6.18.8-linode16/kernel/drivers/block/drbd.ko
filename: /lib/modules/2.6.18.8-linode16/kernel/drivers/block/drbd.ko
alias: block-major-147-*
license: GPL
description: drbd - Distributed Replicated Block Device v8.3.1
author: Philipp Reisner <phil@linbit.com>, Lars Ellenberg <lars@linbit.com>
depends:
vermagic: 2.6.18.8-linode16 SMP mod_unload Xen PENTIUM4 REGPARM gcc-4.3
parm: minor_count:Maximum number of drbd devices (1-255) (uint)
parm: allow_oos:DONT USE! (bool)
parm: cn_idx:uint
parm: enable_faults:int
parm: fault_rate:int
parm: fault_count:int
parm: fault_devs:int
parm: trace_level:int
parm: trace_type:int
parm: trace_devs:int
parm: proc_details:int
parm: usermode_helper:string
and double check my gcc:
Code:
# gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.3.2-1.1' --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --enable-targets=all --enable-cld --enable-checking=release --build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.3.2 (Debian 4.3.2-1.1)
I notice that the architecture is set to i486-linux-gnu, rather than i686-linux-gnu. But I want to check the kernel too:
Code:
# cd /usr/src/linux/scripts/
# chmod +x ver_linux
# ./ver_linux
If some fields are empty or look unusual you may have an old version.
Compare to the current minimal requirements in Documentation/Changes.
Linux li34-206 2.6.18.8-linode16 #1 SMP Mon Jan 12 09:50:18 EST 2009 i686 GNU/Linux
Gnu C 4.3.2-1.1)
Gnu make 3.81
binutils Binutils
util-linux 2.13.1.1
mount 2.13.1.1
module-init-tools 3.4
e2fsprogs 1.41.3
Linux C Library > libc.2.7
Dynamic linker (ldd) 2.7
Procps 3.2.7
Net-tools 1.60
Kbd 83:
Sh-utils 6.10
Modules Loaded
So I think the gcc I've got is fine, and dpkg -l make tells me I'm at version 3.81-5. Aside from the target architecture being i486 (haven't got a clue if this is relevant!) I think the system's okay...
And then I think I find it!
Code:
# dmesg | tail -1
drbd: version magic '2.6.18.8-linode16 SMP mod_unload Xen PENTIUM4 REGPARM gcc-4.3' should be '2.6.18.8-linode16 SMP mod_unload Xen PENTIUM4 REGPARM gcc-4.2'
Bingo!
Code:
# apt-get install gcc-4.2
# rm /usr/bin/gcc
# ln -s /usr/bin/gcc-4.2 /usr/bin/gcc
# cd /usr/src/drbd-8.3.1
# make KDIR=/usr/src/linux/
# make install
# modprobe drbd
# lsmod
Module Size Used by
drbd 257688 0
# rm /usr/bin/gcc
# ln -s /usr/bin/gcc-4.3 /usr/bin/gcc
Okay, I think that works. I'm rebooting and will check it:
Code:
# lsmod
Module Size Used by
# modprobe drbd
# lsmod
Module Size Used by
drbd 257688 0
I'm calling that good.
So finally it comes down to a question. Is it a bug if the kernel we're using wasn't compiled with the same gcc version installed in the distribution? I understand this could be a problem - Debian Lenny has a newer version off gcc than Etch (at least I presume so) and the Ubuntu 8.04 distro (again, I presume) that irgeek used. So it's probably not a bug so much as an experience I've earned in exchange for virtually all of my labor this week
As a final note, I'll type this up under the Linux Tips, Tricks, and Tutorials section, using irgeek's instructions for a standard install (at least on Ubuntu 8.04) and my revised instructions on Debian 5.0.
FAQ
Search
Members
Register
Login [ Anonymous ]
- and I've spent the better part of two days now trying to get drbd8 running on a Debian Lenny kernel, mostly on the 2.6.18.8-linode16 kernel, and been unsuccessful so far at getting the modules to compile. I've seen a variety of errors when trying apt-get install + module-assistant, as well as compiling from source. I can post more details - no doubt there's something I'm doing wrong - but one thing I haven't found is a Linode-specific write-up or how-to. I've looked for a drbd specific write-up, as well as a little on compiling modules, but I'm still stumped.
