Home > Admin > A (little) MySQL bug story…

A (little) MySQL bug story…

I just want to share about a strange behavior of one of our MySQL server yesterday.
This server is a 5.1.50 MySQL server on debian 4.0 (Yes, I know…)

When a “mysqld got signal 6” error occurred yesterday, the MySQL server crashed and didn’t want to restart.
Then, I found these informations in the error log file :

/usr/local/mysql/bin/mysqld: File '*** glibc detected ***
malloc():memory corruption: 0x00002aac2d5ab460 ***' not found (Errcode: 2)
120306 17:19:47 [ERROR] Failed to open log (file '*** glibc detected ***
malloc():memory corruption: 0x00002aac2d5ab460 ***', errno 2)
120306 17:19:47 [ERROR] Could not open log file
120306 17:19:47 [ERROR] Can't init tc log
120306 17:19:47 [ERROR] Aborting
120306 17:19:47 InnoDB: Starting shutdown...
120306 17:19:53 InnoDB: Shutdown completed; log sequence number 55 1061584593
120306 17:19:53 [Note] /usr/local/mysql/bin/mysqld: Shutdown complete

Great, let see what is the Errcode 2 through the perror utility :

# perror 2
OS error code 2: No such file or directory

But where is this missing file ? And have you ever heard about a file called “*** glibc detected *** malloc():memory corruption: 0x00002aac2d5ab460 ***” ?!
No, of course not. But this error message  was more meaningful : Could not open log file

The InnoDB log files and binary log files seemed to be ok (file permissions and other checks showed no failure)
But when I did a cat on the binary log index file… surprise ! :

# cat mysql-bin.index
/var/database/bin-log/mysql-bin.126242
/var/database/bin-log/mysql-bin.126243
/var/database/bin-log/mysql-bin.126244
/var/database/bin-log/mysql-bin.126245
*** glibc detected *** malloc(): memory corruption: 0x00002aac2d5ab460 ***

The index file was corrupted during the last crash and an error message was inserted inside this file because of a memory corruption.
Deleting files helped to restart the MySQL server but I found very strange that MySQL wasn’t able to deliver a more accurate message about that.

PS : A MySQL bug report was created : http://bugs.mysql.com/bug.php?id=64579

Congratulations, you've read this post till the end !
If you enjoyed this article, please leave a comment


You can also subscribe to RSS feed or by email to automatically receive real-time notification of new posts

Thanks for your visit on MySQL+. Cédric
Categories: Admin
  1. Joro
    March 8th, 2012 at 06:20 | #1

    This is how a memory corruption error looks like.
    Do you know what have you done to go into this state ?
    If you can repeat the transition from the working state to this corrupted state please file a bug with your findings.

  1. May 9th, 2012 at 06:02 | #1
  2. January 7th, 2013 at 11:34 | #2