MySQL and Secure Linux (SELinux)

Maybe you experienced some strange behaviour with MySQL: Everything is installed correctly and should work. But it does not.

Symptoms we have seen:

• MySQL starts/stops properly when started/stopped with service mysqld restart but MySQL does not start when a server is rebooted.
• Or after upgrading MySQL binaries mysqld will not start at all any more.
• Or after relocating MySQL datadir or changing default port MySQL does not start any more.

shell> service mysqld start
MySQL Daemon failed to start.
Starting mysqld:                                           [FAILED]

shell> grep mysqld /var/log/boot.log 
Starting mysqld:  [FAILED]

If you are lucky you get some error message like: ERROR! The server quit without updating PID file (/data/mysql/server.pid). or:

130620  9:49:14 [ERROR] Can't start server : Bind on unix socket: Permission denied
130620  9:49:14 [ERROR] Do you already have another mysqld server running on socket: /var/lib/mysql/mysql.sock ?
130620  9:49:14 [ERROR] Aborting

This typically happens when you relocate the MySQL data files (datadir), change port, socket, log file, pid file or similar.

The reason for this problem is not too easy to find. You see some traces in /var/log/boot.log. And if you know where to look for you will find something in /var/log/audit/audit.log. But without knowing where to look and what to look for it is quite hard.
If you are lucky the setroubleshoot utility is installed. This will report problems in the syslog (/var/log/messages).

The cause of this problem might be the Secure Linux (SELinux) feature!

SELinux [1], [2], [3] is typically used in Red Hat, CentOS and Fedora Linux. On Debian, Ubuntu and SuSE you have a similar solution called AppArmor.

To see if SELinux is enabled just run the following command:

shell> sestatus
SELinux status:                 enabled
SELinuxfs mount:                /selinux
Current mode:                   enforcing
Mode from config file:          enforcing
Policy version:                 24
Policy from config file:        targeted

To disable SELinux you just have to run the following command:

shell> setenforce 0

And to make this change persistent you have to change it in the following configuration file:

#
# /etc/selinux/config 
#
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=permissive
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 

But possibly you want to move the MySQL datadir to an other location without disabling SELinux? To achieve this proceed with the following steps:

The simple way

If you have just moved datadir or the MySQL port the Blog article SELinux and MySQL of Jeremy Smyth is a good starting point.

Complicated way

If you want to create an other or a new MySQL instance or do some other stuff you have to do some more things manually (possibly there is also an automated way?):

First it is recommended to install the setroubleshoot utility. Then with the command:

shell> tail /var/log/messages
Jun 20 09:38:53 ip-10-39-25-184 setroubleshoot: SELinux is preventing /bin/mkdir from write access on the directory /var/lib. For complete SELinux messages. run sealert -l ef8eae63-7ec3-4b22-87e0-5774120726c3

You will find what is going wrong. Follow the instructions:

shell> sealert -l ef8eae63-7ec3-4b22-87e0-5774120726c3
SELinux is preventing /bin/mkdir from write access on the directory /var/lib.

*****  Plugin catchall_labels (83.8 confidence) suggests  ********************

If you want to allow mkdir to have write access on the lib directory
Then you need to change the label on /var/lib
Do
# semanage fcontext -a -t FILE_TYPE '/var/lib'
where FILE_TYPE is one of the following: var_log_t, mysqld_var_run_t, mysqld_db_t, root_t. 
Then execute: 
restorecon -v '/var/lib'


*****  Plugin catchall (17.1 confidence) suggests  ***************************

If you believe that mkdir should be allowed write access on the lib directory by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# grep mkdir /var/log/audit/audit.log | audit2allow -M mypol
# semodule -i mypol.pp

until MySQL starts properly. And also test a reboot of the machine!