Hi everyone. I'm hoping someone can help me. I created a "vanilla" CentOS distro, and did an "/etc/init.d/mysqld start". Everything works fine. Then, I did a "yum update", updated all the packages (137 of them) that it recommended, rebooted, and now MySQLd is broken. It won't start; I immediately get an error, and the mysqld process hangs and eats 100% of available CPU.

Do you think that updating libraries caused the breakage? How do I get security updates without breaking MySQLd? This is very frustrating to me, because I would expect MySQLd to be backtested against the new updates when they're released.

The following message is in the error log file:

040908 21:15:57 mysqld started
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail

key_buffer_size=12288
record_buffer=131072
sort_buffer=65528
max_used_connections=0
max_connections=100
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (record_buffer + sort_buffer)*max_connections = 19211 K
bytes of memory
Hope that's ok, if not, decrease some variables in the equation

Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x402ab794, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x80e43dc
0x4007ad38
(nil)
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://www.mysql.com/doc/U/s/Using_stack_trace.html and follow instr
uctions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
The manual page at http://www.mysql.com/doc/C/r/Crashing.html contains
information that should help you find out what is causing the crash

I did a stack trace on it and this is what it told me:

[root@pennytracker work]# resolve_stack_dump -s /tmp/mysqld.sym -n mysqld.stack
0x80e43dc _Z18flush_thread_cachev + 876
0x4007ad38 _end + 936751312

Thanks in advance for your help.

Edit: MySQL version is actually 3.23.58-1, not 3.23.53-1

I was able to fix this by doing:

mv /lib/tls /lib/tls.backup

Apparently newer versions of glibc support thread local syscalls, which is not yet supported in UML. Moving this directory disables them. I'm still not sure if updating glibc again will kill my installation though.

Yep, just as I suspected. A yum update killed my mysql again. This time I knew what to look for. Sorry to resurrect this old thread, but is there any chance you can fix this bug? This is pretty major and should be quite embarassing to Linode staff. Doing a glibc update should NOT break User Mode Linux.

Embarrassing to Linode staff? This is just a fact of life for UML as it doesn't have NPTL/TLS support. It's a simple work around to just move /lib/tls out of the way until UML has support for it.

-Chris

Thread local storage is not currently implemented in UML. Unfortunately, NPTL enabled /lib/ld.so assumes that all kernels >= 2.6.0 have TLS available. Doing TLS 'the right way' on X86 means tinkering with the global descriptor table in a regular kernel - a difficult thing to safely emulate in UML. This isn't something for caker to fix - he has to wait for Jeff Dike to impement TLS inside UML.

_________________
/ Peter

Maybe Linode should switch to Solaris 10 for x86 with properly implemented containers...

All joking aside, thanks for the information. Does a 2.4 series kernel run "better" on UML? I know better is a relative term, but would there be any advantages to switching to an older yet more stable kernel?

All of the UML development of late has been going into the 2.6 tree, and 2.4 is lagging behind. Take that for what it's worth, but the only thing preventing me from removing the "Experimental" label on the 2.6 kernels is the pain and suffering the /lib/tls issue causes (wrt answering the same ticket over and over). If I had an acceptible work-around, like a kernel-level hack for ignoring /lib/tls, I'd make 2.6 the default kernel.

-Chris