You are here


Download MySQL Enterprise Features

Shinguz - Mon, 2015-02-02 21:47
Taxonomy upgrade extras: enterprise monitorBackupworkbenchenterprise

MySQL provides some great enterprise features beside the MySQL Server. The ones we are asked the most at customers are:

  • MySQL Enterprise Backup (MEB)
  • MySQL Enterprise Monitor (MEM) and
  • MySQL Enterprise Workbench (MWB)
MySQL Enterprise Backup (MEB)

MySQL Enterprise Backup (MEB) is an alternative to the mysqldump backup utility. Its big advantage is its fast backup but even faster restore performance. This is a must for all MySQL users having bigger databases than let's say 10 to 20 Gigabytes and/or having hard requirements for restore times (MTTR).

Last implementation tests we did with a customer for an about 30 Gbyte database were:

 MEBmysqldumpBackup10 minutes18 minutesRestore12 minutes80 minutes

If you need our help implementing MySQL Enterprise Backup into your backup infrastructure please get in contact with us. MySQL Enterprise Backup also seamlessly integrates into the FromDual Backup Manager for MySQL.

MySQL Enterprise Monitor (MEM)

MySQL Enterprise Monitor (MEM) is an Enterprise Monitoring Solution for MySQL which Monitors your business critical MySQL databases. Various predefined advisors rise an alert if something with your precious MySQL database does not work as expected.

Our alternative competitive product is the FromDual Performance Monitor for MySQL.

MySQL Enterprise Workbench (MWB)

MySQL Enterprise Workbench (MWB) is the tool modern MySQL Database administrators use to operate their MySQL databases. Old fashion ones still use the CLI... MySQL developers can easily write and test database queries and develop their data model with the ER diagram modeller.

Download Enterprise tools

But how can we get now to these precious tools? This is quite easy following the screen shots below:

As a fist step you go to

Here you can find a link to Oracle eDelivery which is the MySQL/Oracle download facility. Then you get to the welcome screen:

Before you get access to the software you have to Sign In with your Oracle/MySQL customer account if you have one. If you do not have an account yet you can Create an Account to get to the software:

Then you have to agree (2 times) to the Terms & Restrictions (this is what Oracle is really good in). Once to the Oracle Trial License Agreement and once to the Export Restrictions:

Then you get to the Media Pack Search. Here you can define what product you are interested in and on which platform you are using it. Unfortunately a sub-product filter cannot be chosen. So you get a long list to pick your final package from:

An last you can download your product of desire:

Unfortunately the packages get some silly names like instead of meaningful names. But with the following command you get some information what is included in the package:

unzip -l Archive: Length Date Time Name --------- ---------- ----- ---- 2958631 2014-11-04 13:29 meb-3.11.1-linux-glibc2.5-x86-64bit.tar.gz 185 2014-11-05 08:30 meb-3.11.1-linux-glibc2.5-x86-64bit.tar.gz.asc 77 2014-11-04 13:29 meb-3.11.1-linux-glibc2.5-x86-64bit.tar.gz.md5 2130 2014-11-05 15:06 README.txt --------- ------- 2961023 4 files

Have fun trying the MySQL Enterprise Features. If you need any help installing or integrating them into your infrastructure, do not hesitate to contact FromDual.

MySQL table Point-in-Time-Recovery from mysqldump backup

Shinguz - Sun, 2015-01-25 19:42
Taxonomy upgrade extras: BackupRestoreRecoverymysqldumppoint-in-time-recoverypitr

Sometimes we face the situation where we have a full MySQL database backup done with mysqldump and then we have to restore and recover just one single table out of our huge mysqldump file.
Further our mysqldump backup was taken hours ago so we want to recover all the changes on that table since our backup was taken up to the end.

In this blog article we cover all the steps needed to achieve this goal for MySQL and MariaDB.

Recommendation: It is recommended to do theses steps on a testing system and then dump and restore your table back to the production system. If you do it directly on your production system you have to know exactly what you are doing...
Further this process should be tested carefully and regularly to get familiar with it and to assure your backup/restore/recovery procedure works properly.

The table we want to recover is called test.test from our backup full_dump.sql.gz. As a first step we have to do the recovery with the following command to our test database:

shell> zcat full_dump.sql.gz | --database=test --table=test | mysql -u root

The script is part of the FromDual Recovery Manager to extract one single table from a mysqldump backup.

As a next step we have to extract the binary log file and its position where to start recovery from out of our dump:

shell> zcat full_dump.sql.gz | head -n 25 | grep CHANGE CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000216', MASTER_LOG_POS=1300976;

Then we have to find out where we want to stop our Point-in-Time-Recovery. The need for recover is possibly due to a TRUCATE TABLE command or similar operation executed on the wrong system or it is just a time somebody has indicated us to recover to. The position to stop we can find as follows:

shell> mysqlbinlog -v mysql-bin.000216 | grep -B5 TRUNCATE --color #150123 19:53:14 server id 35622 end_log_pos 1302950 CRC32 0x24471494 Xid = 3803 COMMIT/*!*/; # at 1302950 #150123 19:53:14 server id 35622 end_log_pos 1303036 CRC32 0xf9ac63a6 Query thread_id=54 exec_time=0 error_code=0 SET TIMESTAMP=1422039194/*!*/; TRUNCATE TABLE test

And as a last step we have to apply all the changes from the binary log to our testing database:

shell> mysqlbinlog --disable-log-bin --database=test --start-position=1300976 --stop-position=1302950 mysql-bin.000216 | mysql -u root --force

Now the table test.test is recovered to the wanted point in time and we can dump and restore it to its final location back to the production database.

shell> mysqldump --user=root --host=testing test test | mysql --user=root --host=production test

This process has been tested on MySQL 5.1.73, 5.5.38, 5.6.22 and 5.7.5 and also on MariaDB 10.0.10 and 10.1.0.

FromDual Backup Manager for MySQL 1.2.1 has been released

FromDual.en - Thu, 2015-01-22 21:12
Taxonomy upgrade extras: BackupRestoreRecoveryfromdual_brmanmysql_bman

FromDual has the pleasure to announce the release of the new version 1.2.1 of the popular Backup Manager for MySQL and MariaDB (fromdual_bman).

You can download the FromDual Backup Manager from here.

In the inconceivable case that you find a bug in the Backup Manager please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to

Upgrade from 1.2.0 to 1.2.1 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.1.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.1 fromdual_brman
Changes in FromDual Backup Manager 1.2.1 FromDual Backup Manager

It contains mainly one very critical bug fix, one bug fix for using FromDual Backup Manger in combination with MariaDB and xtrabackup and several minor bug fixes and small improvements.

We STRONGLY RECOMMEND to upgrade immediately to v1.2.1 if you are using FromDual Backup Manager v1.2.0-RC1 and newer. In these versions errors of mysqldump in combination with the --direct-compress option are not handled correctly and a return code of 0 is returned which can be wrong. This leads to incomplete backups and loss of data during the restore.

You can verify your current FromDual Backup Manager version with the following command:

fromdual_bman --version
  • Merges of with the one of MyEnv.
  • Bug with MariaDB using xtrabackup fixed (PM).
  • Lock file handling improved.
  • Very important: max_allowed_packet bug fixed which was introduced in v1.2.0-RC1.
  • Schema names with special characters are now handled properly.
  • Log directory is automatically created if it does not exist.
  • Error comment for --blocking-backup improved.

Introducing Myself: Jörg Brühe

Jörg Brühe - Mon, 2015-01-19 20:56
For some time already, FromDual's "Our Team" page lists me, and it even reveals that I joined in September, 2014. Also for some time, the list of FromDual blogs contains an entry "Jörg's Blog", but it doesn't lead to any entries. It is high time to fix this and create entries, starting with an introduction of myself.

Often, in such introductions people use the phrase of "the new kid on the block". I won't. If I am to use those words, I will arrange them as "the kid on the new block". The reason is that I don't feel as a new kid in the MySQL village (or is it a city?), let alone in DBMS country.

Ever since I left university (Technical University of Berlin, Germany), I have been involved in SQL DBMS development. After my previous product's team had been dissolved in Berlin and maintenance moved to Riga, Latvia, as a cost-cutting measure, I joined MySQL AB in 2004 as a member of the Build Team. Some of you will remember my name from MySQL release announcements, others may have seen it in bug reports, newsgroup entries, or other MySQL communication. Together with the other MySQL colleagues, I got acquired by Sun in 2008 and then by Oracle in 2010.

In 2012, I decided I wanted a culture change, from a highly (and very strictly) organized corporation to a smaller, more flexible entity. That brought me to an internet site, where I worked as a DBA, concentrating on their (more than 100) MySQL installations. It was very interesting to view MySQL from the DBA side now, I got a big load of new experiences and learned a lot about using, running, and automating MySQL (rather than building and testing it).

That might have continued, but as usual the world is changing, and my DBA job was no exception: Company strategy was to shift those tasks more to the various application teams and to do without a central point for MySQL. It is a sign how far MySQL knowledge has spread, and how little effort may be sufficient to run MySQL, that this worked. OTOH, this invalidated the assumptions on which my job was based, and I did not really fit the alternatives left in that company.

In that situation, my contact to FromDual got renewed, and both the company and me felt we should be a good match. Till now, there is no indication we were wrong: I like the work, I feel I'm productive and make customers happy, the tasks are manifold and interesting (so interesting that writing the first blog gets delayed quite long), and I expect this will continue.

"Ad multos annos!"

Impacts of max_allowed_packet size problems on your MySQL database

Shinguz - Sun, 2015-01-18 11:18
Taxonomy upgrade extras: max_allowed_packetconnectionBackupRestoredump

We recently run into some troubles with max_allowed_packet size problems during backups with the FromDual Backup/Recovery Manager and thus I investigated a bit more in the symptoms of such problems.

Read more about: max_allowed_packet.

A general rule for max_allowed_packet size to avoid problems is: All clients and the server should have set the same value for max_allowed_packet size!

I prepared some data for the test which looked as follows:

mysql> SELECT id, LEFT(data, 30), LENGTH(data), ts FROM test; +----+--------------------------------+--------------+------+ | id | left(data, 30) | length(data) | ts | +----+--------------------------------+--------------+------+ | 1 | Anhang | 6 | NULL | | 2 | Anhang | 6 | NULL | | 3 | Anhangblablablablablablablabla | 2400006 | NULL | | 4 | Anhang | 6 | NULL | +----+--------------------------------+--------------+------+

Max_packet_size was set to a too small value then:

mysql> SHOW GLOBAL VARIABLES WHERE variable_name = 'max_allowed_packet'; +--------------------+---------+ | Variable_name | Value | +--------------------+---------+ | max_allowed_packet | 1048576 | +--------------------+---------+

The first test was to retrieve the too big row:

mysql> SELECT * FROM test WHERE id = 3; ERROR 2020 (HY000): Got packet bigger than 'max_allowed_packet' bytes mysql> SELECT CURRENT_USER(); ERROR 2006 (HY000): MySQL server has gone away No connection. Trying to reconnect... Connection id: 6 Current database: test

We got an error message AND we were disconnected from the server. This is indicated with the message MySQL server has gone away which is basically wrong. We were disconnected and not the server has died or similar in this case.

A further symptom is that we get an entry in the MySQL error log about this incident:

[Warning] Aborted connection 3 to db: 'test' user: 'root' host: 'localhost' (Got an error writing communication packets)

So watching carefully such error messages in your MySQL error log with the script from our Nagios/Icinga plugins would be a good idea...

The mysqldump utility basically does the same as a SELECT command so I tried this out and got the same error:

shell> mysqldump -u root test > /tmp/test_dump.sql mysqldump: Error 2020: Got packet bigger than 'max_allowed_packet' bytes when dumping table `test` at row: 2

And again we get an error message in the error log! This is also a good indicator to see if your backup, made with mysqldump failed in this case.

To get a proper dump we have to configure the mysqldump utility properly:

shell> mysqldump --max-allowed-packet=5000000 -u root test > /tmp/test_dump.sql

After the backup we tried to restore the data:

shell> mysql -u root test < /tmp/test_dump.sql ERROR 2006 (HY000) at line 40: MySQL server has gone away

Again we got an error on the command line and in the MySQL error log:

[Warning] Aborted connection 11 to db: 'test' user: 'root' host: 'localhost' (Got a packet bigger than 'max_allowed_packet' bytes)

and further the data are only partially loaded:

mysql> SELECT * FROM test; +----+--------+------+ | id | data | ts | +----+--------+------+ | 1 | Angang | NULL | | 2 | Angang | NULL | +----+--------+------+

Another symptom we can see here is that the MySQL status aborted_clients is increased in all 3 situation:

mysql> SHOW GLOBAL STATUS WHERE variable_name = 'aborted_clients'; +-----------------+-------+ | Variable_name | Value | +-----------------+-------+ | Aborted_clients | 10 | +-----------------+-------+

One positive aspect is that with MySQL 5.7.5 the first 2 symptoms do not appear any more...

Further information you can find here: Communication Errors and Aborted Connections.

Avoid temporary disk tables with MySQL

Shinguz - Fri, 2014-12-19 07:38
Taxonomy upgrade extras: temporary tablediskselectquery tuning

For processing SELECT queries MySQL needs some times the help of temporary tables. These temporary tables can be created either in memory or on disk.

The number of creations of such temporary tables can be found with the following command:

mysql> SHOW GLOBAL STATUS LIKE 'created_tmp%tables'; +-------------------------+-------+ | Variable_name | Value | +-------------------------+-------+ | Created_tmp_disk_tables | 4 | | Created_tmp_tables | 36 | +-------------------------+-------+
There are 2 different reasons why MySQL is creating a temporary disk table instead of a temporary memory table:
  • The result is bigger than the smaller one of the MySQL variables max_heap_table_size and tmp_table_size.
  • The result contains columns of type BLOB or TEXT.
In the following example we can see how the temporary disk table can be avoided without changing the column types: mysql> CREATE TABLE test ( id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY , data TEXT , type TINYINT UNSIGNED ); mysql> INSERT INTO test VALUES (NULL, 'State is green', 1), (NULL, 'State is green', 1) , (NULL, 'State is red', 3), (NULL, 'State is red', 3) , (NULL, 'State is red', 3), (NULL, 'State is orange', 2); mysql> EXPLAIN SELECT data, COUNT(*) FROM test GROUP BY data; +----+-------------+-------+------+---------------+------+---------+------+------+---------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+------+---------------+------+---------+------+------+---------------------------------+ | 1 | SIMPLE | test | ALL | NULL | NULL | NULL | NULL | 6 | Using temporary; Using filesort | +----+-------------+-------+------+---------------+------+---------+------+------+---------------------------------+ mysql> SHOW SESSION STATUS LIKE 'created_tmp%tables'; +-------------------------+-------+ | Variable_name | Value | +-------------------------+-------+ | Created_tmp_disk_tables | 0 | | Created_tmp_tables | 3 | +-------------------------+-------+ mysql> SELECT data, COUNT(*) FROM test GROUP BY data; +-----------------+----------+ | data | count(*) | +-----------------+----------+ | State is green | 2 | | State is orange | 1 | | State is red | 3 | +-----------------+----------+ mysql> SHOW SESSION STATUS LIKE 'created_tmp%tables'; +-------------------------+-------+ | Variable_name | Value | +-------------------------+-------+ | Created_tmp_disk_tables | 1 | | Created_tmp_tables | 4 | +-------------------------+-------+ mysql> SELECT SUBSTR(data, 1, 32), COUNT(*) FROM test GROUP BY SUBSTR(data, 1, 32); mysql> SHOW SESSION STATUS LIKE 'created_tmp%tables'; +-------------------------+-------+ | Variable_name | Value | +-------------------------+-------+ | Created_tmp_disk_tables | 1 | | Created_tmp_tables | 5 | +-------------------------+-------+

This method can be used if changing the table structure from TEXT to VARCHAR or the use of a RAM disk are not possible solutions.

Making HAProxy High Available for MySQL Galera Cluster

Shinguz - Sun, 2014-12-14 18:37
Taxonomy upgrade extras: HAProxyload balancerGalera ClusterVIPvirtual IPHigh Availabilityha

After properly installing and testing a Galera Cluster we see that the set-up is not finished yet. It needs something in front of the Galera Cluster that balances the load over all nodes.
So we install a load balancer in front of the Galera Cluster. Typically nowadays HAProxy is chosen for this purpose. But then we find, that the whole Galera Cluster is still not high available in case the load balancer fails or dies. So we need a second load balancer for high availability.
But how should we properly fail-over when the HAProxy load balancer dies? For this purpose we put a Virtual IP (VIP) in front of the HAProxy load balancer pair. The Virtual IP is controlled and fail-overed with Keepalived.

Installation of HAProxy and Keepalived

First some preparations: For installing socat we need the repoforge repository:

shell> cd /tmp shell> wget shell> yum localinstall rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm shell> yum update shell> yum install socat

Then we can start installing HAProxy and Keepalived:

shell> yum install haproxy keepalived shell> chkconfig haproxy on shell> chkconfig keepalived on

We can check the installed HAProxy and Keepalived versions as follows:

shell> haproxy -v HA-Proxy version 1.5.2 2014/07/12 shell> keepalived --version Keepalived v1.2.13 (10/15,2014)
Configuration of HAProxy

More details you can find in the HAProxy documentation.

shell> cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bak shell> cat << _EOF >/etc/haproxy/haproxy.cfg # # /etc/haproxy/haproxy.cfg # #--------------------------------------------------------------------- # Global settings #--------------------------------------------------------------------- global # to have these messages end up in /var/log/haproxy.log you will # need to: # # 1) configure syslog to accept network log events. This is done # by adding the '-r' option to the SYSLOGD_OPTIONS in # /etc/sysconfig/syslog # # 2) configure local2 events to go to the /var/log/haproxy.log # file. A line like the following can be added to # /etc/sysconfig/syslog # # local2.* /var/log/haproxy.log # log local2 chroot /var/lib/haproxy pidfile /var/run/ maxconn 1020 # See also: ulimit -n user haproxy group haproxy daemon # turn on stats unix socket stats socket /var/lib/haproxy/stats.sock mode 600 level admin stats timeout 2m #--------------------------------------------------------------------- # common defaults that all the 'frontend' and 'backend' sections will # use if not designated in their block #--------------------------------------------------------------------- defaults mode tcp log global option dontlognull option redispatch retries 3 timeout queue 45s timeout connect 5s timeout client 1m timeout server 1m timeout check 10s maxconn 1020 #--------------------------------------------------------------------- # HAProxy statistics backend #--------------------------------------------------------------------- listen haproxy-monitoring *:80 mode http stats enable stats show-legends stats refresh 5s stats uri / stats realm Haproxy\ Statistics stats auth monitor:AdMiN123 stats admin if TRUE frontend haproxy1 # change on 2nd HAProxy bind *:3306 default_backend galera-cluster backend galera-cluster balance roundrobin server nodeA maxconn 151 check server nodeB maxconn 151 check server nodeC maxconn 151 check _EOF
Starting and testing HAProxy

The HAProxy can be started as follows:

shell> service haproxy start

and then be checked either over the socket:

shell> socat /var/lib/haproxy/stats.sock readline prompt > show info > show stat > help

or over your favourite web browser entering the username and password (monitor:AdMiN123) specified in the configuration file above:

To check the application over the load balancer we can run the following command:

shell> mysql --user=app --password=secret --host= --port=3306 --exec="SELECT @@wsrep_node_name;" +-------------------+ | @@wsrep_node_name | +-------------------+ | Node C | +-------------------+ shell> mysql --user=app --password=secret --host= --port=3306 --exec="SELECT @@wsrep_node_name;" +-------------------+ | @@wsrep_node_name | +-------------------+ | Node A | +-------------------+ shell> mysql --user=app --password=secret --host= --port=3306 --exec="SELECT @@wsrep_node_name;" +-------------------+ | @@wsrep_node_name | +-------------------+ | Node B | +-------------------+
Configuration a Virtual IP (VIP) with Keepalived

Now we have 2 HAProxy load balancers. But what happens if one of them fails. Then we do not want to reconfigure our application to work properly again. The fail-over should happen automatically. For this we need a Virtual IP which should automatically fail-over.

More details you can find in the Keepalived documentation and the keepalived user guide.

shell> cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak cat << _EOF >/etc/keepalived/keepalived.conf # # /etc/keepalived/keepalived.conf # global_defs { notification_email { root@localhost } # Change email from on lb2: notification_email_from lb1@haproxy1 router_id HAPROXY } vrrp_script chk_haproxy { script "killall -0 haproxy" interval 2 weight 2 } vrrp_instance GALERA_VIP { interface eth1 virtual_router_id 42 # Higher priority on other node priority 101 # 102 advert_int 1 # notify "/usr/local/bin/" virtual_ipaddress { dev eth1 } track_script { chk_haproxy } authentication { auth_type PASS auth_pass secret } } _EOF
Starting and testing Keepalived

To test the keepalived we can run the following command:

shell> keepalived -f /etc/keepalived/keepalived.conf --dont-fork --log-console --log-detail ^C

To finally start it the following command will serve:

shell> service keepalived start

To check the Virtual IP the following command will help:

shell> ip addr show eth1

And then we can check our application over the VIP:

shell> mysql --user=app --password=secret --host= --port=3306 --exec="SELECT @@wsrep_node_name;"

failed MySQL DDL commands and Galera replication

Shinguz - Tue, 2014-12-09 15:45
Taxonomy upgrade extras: galerareplicationDDLTOIRSU

We have recently seen a case where the following command was executed on a Galera Cluster node:

SQL> GRANT SUPER ON userdb.* TO root@; ERROR 1221 (HY000): Incorrect usage of DB GRANT and GLOBAL PRIVILEGES
2014-12-09 14:53:55 7457 [Warning] Did not write failed 'GRANT SUPER ON userdb.* TO root@' into binary log while granting/revoking privileges in databases. 2014-12-09 14:53:55 7457 [ERROR] Slave SQL: Error 'Incorrect usage of DB GRANT and GLOBAL PRIVILEGES' on query. Default database: ''. Query: 'GRANT SUPER ON userdb.* TO root@', Error_code: 1221 2014-12-09 14:53:55 7457 [Warning] WSREP: RBR event 1 Query apply warning: 1, 17 2014-12-09 14:53:55 7457 [Warning] WSREP: Ignoring error for TO isolated action: source: c5e54ef5-7faa-11e4-97b0-5e5c695f08a5 version: 3 local: 0 state: APPLYING flags: 65 conn_id: 4 trx_id: -1 seqnos (l: 4, g: 17, s: 15, d: 15, ts: 113215863294782)

According to the error message it looks like this command is done in Total Order Isolation (TOI) mode during the Rolling Schema Upgrade (RSU).

Only on the nodes which did NOT receive this wrong command the error log message was written and further they have received a GRA_*.log file.

Analysis of the GRA_*.log (failed transactions) files:

hexdump -C GRA_2_16.log 00000000 f3 fe 86 54 02 53 14 00 00 76 00 00 00 76 00 00 |...T.S...v...v..| 00000010 00 00 00 04 00 00 00 00 00 00 00 00 00 00 2a 00 |..............*.| 00000020 00 00 00 00 00 01 00 00 00 40 00 00 00 00 06 03 |.........@......| 00000030 73 74 64 04 21 00 21 00 08 00 0b 04 72 6f 6f 74 |std.!.!.....root| 00000040 09 6c 6f 63 61 6c 68 6f 73 74 00 67 72 61 6e 74 |.localhost.grant| 00000050 20 53 55 50 45 52 20 6f 6e 20 75 73 65 72 64 62 | SUPER on userdb| 00000060 2e 2a 20 74 6f 20 72 6f 6f 74 40 31 32 37 2e 30 |.* to root@127.0| 00000070 2e 30 2e 31 31 31 |.0.111 |
dd if=bin-log.000001 of=binlog.header bs=1 count=120 cat binlog.header GRA_2_17.log > GRA_2_17.binlog_events mysqlbinlog GRA_2_17.binlog_events ... # at 120 #141209 15:04:54 server id 5201 end_log_pos 118 CRC32 0x3432312e Query thread_id=45 exec_time=0 error_code=0 SET TIMESTAMP=1418133894/*!*/; SET @@session.pseudo_thread_id=4/*!*/; SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/; SET @@session.sql_mode=1073741824/*!*/; SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/; /*!\C utf8 *//*!*/; SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=8/*!*/; SET @@session.lc_time_names=0/*!*/; SET @@session.collation_database=DEFAULT/*!*/; grant SUPER on userdb.* to root@ /*!*/; DELIMITER ; # End of log file ROLLBACK /* added by mysqlbinlog */; /*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/; /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;

It further looks like this command was issues by Connection ID number 4: conn_id: 4.

MySQL Environment MyEnv 1.1.2 has been released

FromDual.en - Thu, 2014-10-23 22:26
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackupcatalog

FromDual has the pleasure to announce the release of the new version 1.1.2 of its popular MySQL, Galera, MariaDB and Percona Server multi-instance environment MyEnv.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to

Upgrade from 1.1.1 to 1.1.2 # cd ${HOME}/product # tar xf /download/myenv-1.1.2.tar.gz # rm -f myenv # ln -s myenv-1.1.2 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/
Changes in MyEnv 1.1.2 MyEnv
  • The MySQL Backup Manager was removed from MyEnv and put into its own package which can be downloaded from FromDual Backup/Recovery Manager.
  • OEM Agent is checked now based on process list and not oemagentctl status any more. Makes it much faster.
  • Alias cdc for NDB Cluster directory removed. NDB Cluster is not supported any more.
  • Deprecate alias v and replace by V.
  • Error unlink(/var/lock/subsys/mysql): Permission denied is caught more nicely now.
  • Unknown version in up guessing is improved.
  • MD5 checksum made portable for Darwin OS (Mac OSX).
MyEnv Installer
  • innodb_flush_log_at_trx_commit default in template changed.
  • Version guessing improved for installer.
  • Better download support for Percona Server and MariaDB added.
  • mkdir bug fixed.
  • Version check for RedHat made better.
  • Check for lsb_release and SELinux/AppArmor check added for faster finding problems during MySQL installation.
  • Template my.cnf from website is used for creating an intance.
  • Option hideschema is automatically added to the myenv.conf file now.
  • Check and warning implemented if non mysql user is used.
  • Error is caught when wrong user is used.
  • mysql_install_db output made more verbose in case of errors for debugging.
  • Default option changes from Add to Save after instance was changed.
  • Missing users HOME directory is caught now.
  • Question done? can be answered with y now.
  • Comment about waiting during instance installation added.
MyEnv Utilities
  • Table offline/online scripts integrated into MyEnv utilities.
  • does hide views from Primary Key check now.
MySQL Backup Manager

Get rid of wrongly deleted InnoDB tables

Shinguz - Wed, 2014-10-22 22:10
Taxonomy upgrade extras: BackupRestoreRecoveryinnodbtable

Precaution: Before you try this out on your production system do a BACKUP first! FromDual Backup Manager can help you with this.


A MySQL user has delete its InnoDB table files for example like this:

shell> rm -f $datadir/test/test.* Analysis

We do some analysis first:

mysql> DROP TABLE test; ERROR 1051 (42S02): Unknown table 'test' mysql> CREATE TABLE test (id INT) ENGINE = InnoDB; ERROR 1050 (42S01): Table '`test`.`test`' already exists

The MySQL error log shows us the following information:

141022 17:09:04 InnoDB: Operating system error number 2 in a file operation. InnoDB: The error means the system cannot find the path specified. InnoDB: If you are installing InnoDB, remember that you must create InnoDB: directories yourself, InnoDB does not create them. 141022 17:09:04 InnoDB: Error: trying to open a table, but could not InnoDB: open the tablespace file './test/test.ibd'! InnoDB: Have you moved InnoDB .ibd files around without using the InnoDB: commands DISCARD TABLESPACE and IMPORT TABLESPACE? InnoDB: It is also possible that this is a temporary table #sql..., InnoDB: and MySQL removed the .ibd file for this. InnoDB: Please refer to InnoDB: InnoDB: for how to resolve the issue.

User claims that he does NOT need the table and/or the data any more but wants to get rid of the error messages and/or create a new table with the same name.

mysql> CREATE SCHEMA recovery; mysql> use recovery mysql> CREATE TABLE test (id INT) ENGINE = InnoDB; mysql> \! cp $datadir/recovery/test.frm $datadir/test/ mysql> DROP SCHEMA recovery; mysql> use test mysql> DROP TABLE test; Prove

To prove it works we create a new table and fill in some records:


Migration between MySQL/Percona Server and MariaDB

Shinguz - Wed, 2014-10-08 21:40
Taxonomy upgrade extras: migrationsidegradeupgrademysqlmariadbpercona serverconversioncompatibility

This week we did some migrations from MariaDB 10.0 to Percona Server 5.6 at the IT department of a big German bank.

We were perfectly aware that since version 10.0 the MariaDB code base started to diverge slightly away from the MySQL and Percona Server code base which are still pretty close to each other.

Because of the Percona Server option enforce_storage_engine we wanted to do this migration.

We stopped the MariaDB 10.0 server replaced the binaries by the Percona Server 5.6 binaries and started the Percona 5.6 server again. After successfully starting the instance we found some error messages in the MySQL error log. By running the mysql_upgrade command some of the problems were fixed but not all of them. Still left problems were:

  • The MariaDB binary logs provoked some error messages for the Percona Server: [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 25, event_type: -93 [Warning] Error reading GTIDs from binary log: -1 [ERROR] Incorrect definition of table mysql.db: expected column 'User' at position 2 to have type char(16), found type char(80). [ERROR] Incorrect definition of table mysql.event: expected column 'definer' at position 3 to have type char(77), found type char(141). [ERROR] Incorrect definition of table mysql.event: expected column 'sql_mode' at position 14 to have type set... A purge of the binary logs solved this issue.
  • The tables mysql.event, mysql.innodb_table_stats and mysql.innodb_index_stats where not fixed by mysql_upgrade (a bug to fix for Percona and MySQL/Oracle?). We had to replace those tables manually by copying from an other already working Percona 5.6 Server.

Later in the FromDual technology labs we investigated further and tried the other way from Percona Server 5.6 to MariaDB 10.0. In this direction we found some other errors in the MySQL error log which also where not completely resolved by the mysql_upgrade utility:

  • The mysql.innodb_table_stats and mysql.innodb_index_stats tables where recreated manually (here a bug to fix for the MariaDB people?).
  • All error messages from tables affected by the following message: InnoDB: in InnoDB data dictionary has unknown flags 40/50/52. could be silenced by a run of the OPTIMIZE TABLE command (which can become quite expensive for very big tables).

Sidegrades from MySQL 5.6 to Percona Server 5.6 and back did not provoke any error message written to the MySQL log files. Sidegrades from MariaDB 10.0 to MySQL 5.6 and vice versa behaved exactly the same as MariaDB 10.0 to Percona Server 5.6 and back.

from/to: MySQL 5.6 MariaDB 10.0 Percona Server 5.6 MySQL 5.6 - 2 tables, OPTIMIZE OK MariaDB 10.0 binlog, 3 tables - binlog, 3 tables Percona Server 5.6 OK 2 tables, OPTIMIZE -

During our tests we got rid of the error messages. If they caused any technical harm to the tables or the data we cannot say so far. Further testing and experience from real life is needed. Any feedback is welcome!


It looks like MariaDB 10.0 understands MySQL/Percona Server replication but not the other way around. So replication from MariaDB 10.0 to MySQL 5.6 does probably not work (different implementation of GTID)?


To make sure a sigdegrade between these 3 MySQL branches/forks is seamlessly possible the best method seems to be to dump/restore (NOT xtrabackup!) the data. This can be an issue with huge databases (hundreds of Gbyte).

Further aid

Also have a look at our MySQL compatibility matrix and our MySQL upgrade check-list.

If you need any help to convert MySQL to MariaDB to Percona Server or the other way do not hesitate to contact the FromDual consultancy team. We will be pleased to assist you as a neutral and vendor independent consulting company.

MySQL Environment MyEnv 1.1.1 has been released

FromDual.en - Mon, 2014-09-08 08:29
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackupcatalog

FromDual has the pleasure to announce the release of the new version 1.1.1 of its popular MySQL, MariaDB and Percona Server multi-instance environment MyEnv.

The majority of improvements happened in the MySQL Backup Manager (mysql_bman) where we added a backup catalog.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to

Upgrade from 1.1.0 to 1.1.1 # cd ${HOME}/product # tar xf /download/myenv-1.1.1.tar.gz # rm -f myenv # ln -s myenv-1.1.1 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/
Upgrade from 1.0.5 to 1.1.1 # cd ${HOME}/product # tar xf /download/myenv-1.1.1.tar.gz # rm -f myenv # ln -s myenv-1.1.1 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/

Replace the following files:

  • cp ${HOME}/product/myenv/etc/variables.conf.template /etc/myenv/variables.conf
  • cp ${HOME}/product/myenv/bin/myenv.server /etc/init.d/myenv
Changes in MyEnv 1.1.1 MyEnv
  • Function to check upgrade requirements to v1.1 implemented.
MyEnv Installer
  • none
MyEnv Utilities
  • none
MySQL Backup Manager
  • End tag added to mysql_bman code in certain places where it was missing.
  • innodbackup bug fixed, pigz bug fixed.
  • Warning message more verbose explaining solution in case of missing xtrabackup binaries.
  • pigz as alternative compression mechanism implemented.
  • Log mysqldump error to stderr.
  • my_exec replaces exec/system calls.
  • Return code exits fixed.
  • Cleanup date is formated nicer and cleanup bug is fixed.
  • copy_preserve introduced to preserve timestamp and archive code cleaned up.
  • Retention time can be 0 or any value out of h, d, w, m.
  • Binary log begin and end ts is written to log file.
MySQL Backup Manager Catalog
  • Catalog DB handle was sometimes called without a catalog (ugly bug introduced in v1.1.0!).

MySQL Environment MyEnv 1.1.0 has been released

FromDual.en - Sat, 2014-08-23 19:46
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackupcatalog

FromDual has the pleasure to announce the release of the new version 1.1.0 of its popular MySQL, MariaDB and Percona Server multi-instance environment MyEnv.

The majority of improvements happened in the MySQL Backup Manager (mysql_bman) where we added a backup catalog.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to

Upgrade from 1.0.5 to 1.1.0 # cd ${HOME}/product # tar xf /download/myenv-1.1.0.tar.gz # rm -f myenv # ln -s myenv-1.1.0 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/

Replace the following files:

  • cp ${HOME}/product/myenv/etc/variables.conf.template /etc/myenv/variables.conf
  • cp ${HOME}/product/myenv/bin/myenv.server /etc/init.d/myenv
Changes in MyEnv 1.1.0 MyEnv
  • Error with !includedir fixed (Bug #143).
  • tput is hidden in interactive mode now. No more nasty warning messages.
  • Start/stop script did not pass error code correctly.
  • Timezone complain on myenv_start_stop.php fixed with temporary fix.
  • Init script should not through errors on SuSE any more.
  • Init script is improved. Myenv should only start if network is available (runlevel).
  • mysqladmin complains now if my.cnf has wrong parameters and does not show the schema down any more.
  • MYSQL_HOME was changed from basedir to datadir, consider this for upgrade, this is a significant change, impact is not clear!
  • Under some circumstances up did not show all schemata, fixed.
MyEnv Installer
  • No changes.
MyEnv Utilities
  • NUMA maps script from Jeremy Cole added.
  • fixed for more than 1 back-end.
MySQL Backup Manager
  • For many backup types cleanup, archive and compress functionality added.
  • Schema backup per-schema writes log pos now to log file and catalog.
  • Binlog policy added.
  • Backup structure is not created any more for cleanup and catalog operations.
  • Binlog backup should work now with and without default location.
  • Passing strange characters in password is handled more robust now.
  • Schema names more robust with back-ticks.
  • memory-table-check bug fixed.
  • Command line is displayed in output (without exposing password).
  • Cleanup of archivedir is possible now.
  • Backup for configuration file implemented.
  • Physical backup based on xtrabackup/mysqlbackup implemented.
  • All exit's replaced by returns. Proper error handling should be possible.
  • Backup logging implemented.
  • Password from command line is not exposed any more to log file. Back-port from v1.0.5.
  • Per schema privilege backup implemented.
MySQL Backup Manager Catalog
  • Catalog creation and catalog upgrade integrated.

FromDual invites MySQL community to company meeting 2014 in Barcelona

FromDual.en - Wed, 2014-08-06 14:29

FromDual holds its annual company meeting this year in Barcelona, Spain.

We are pleased to invite everybody interested in MySQL technologies (MySQL, Galera Cluster, FromDual Tools, Percona Cluster, MariaDB, etc.) to participate on Thursday evening September 11 at the HCC MONTBLANC, Via Laietana 61, to meet, exchange ideas and discuss MySQL related topics.

The event starts at 18:00, we will meet in the hotel lobby. The planned schedule is:

  • How to Implement GTID Replication in MySQL 5.6 (25') and 5' Questions and Answers.
  • MySQL backup/restore for anonymized exports (25') and 5' Questions and Answers.
  • Break 15 min
  • Quick presentation (15') of YOUR project or company.
  • Quick presentation (15') of YOUR project or company.
  • Break 15 min
  • Quick presentation (15') of YOUR project or company.
  • Quick presentation (15') of YOUR project or company.

Please feel free to send us your suggestion about your presentation. Any technical or non-technical MySQL related topic is welcome. For example how you use MySQL in your company or special problems you have faced and solved (or not solved yet), research work you have done on MySQL products, business cases you solve with MySQL products, evaluations or experience you have made, etc. For the proposal please send us a mail.

Please also send us a short notice if you plan to participate or join us with MeetUp.

This gives us the possibility to arrange and organize all the infrastructure with the Hotel.

The event is free of costs for all participants.

We are pleased if you can make it to the event,
Your FromDual Team

FromDual: Tools for MySQL and Galera - Backup - Monitoring - Operations

FromDual.en - Sun, 2014-07-27 10:46
Taxonomy upgrade extras: galeraBackupmanagermonitoringOperations

FromDual tools provide valuable additional functionality which facilitate and optimize daily operations of your MySQL databases. Since our last newsletter a lot of things have changed in the FromDual tools.

To the MySQL Environment (MyEnv) numerous improvements and suggestions of our customers were added. The most important changes were introduced to the MySQL Backup Manager (mysql_bman).

With the MySQL Ops Center we meet the wishes of our numerous customers, which desire a graphical user interface for operating complex MySQL environments.
These users are often little used in handling MySQL but want nevertheless operate more complex MySQL installations as Master/Slave- or Master/Master-replications.

In the MySQL Performance Monitor (mpm) numerous little bugs have been fixed which were reported to us by our customers.

Note: In our MySQL Service Contracts, Business Hour (5x9) and All around the Clock (7x24) the usage and support of our tools is included.
If you would like to know more about our service prices, we are pleased to send you an offer.

MyEnv v1.0.5

The MySQL Environment (MyEnv) gets more and more popularity in the MySQL eco-system. MyEnv is optimized for MySQL and mimics the popular TVD BasEnv which is popular with bigger Oracle database users.

With MyEnv you can easily consolidate several MySQL instances (mysqld) on one single machine. Thanks to MyEnv this complicated configuration is a piece of cake. Furthermore MyEnv is more and more common with customers testing their applications against different MySQL versions (5.5, 5.6 and 5.7) or different MySQL branches (Galera Cluster, MariaDB, Percona Server).

The most important improvements in MyEnv v1.0.5 are:

  • Old PHP functions were replaced to achieve better compatibility with PHP 5.4 and 5.5.
  • MyEnv overview (up) of installed MySQL instances was polished and numerous smaller bugs were fixed.
  • Extensions for active/passive fail-over clusters and Oracle Enterprise Monitor Agents for MySQL were integrated.
  • The user guidance of the MyEnv installer was made more user friendly.
  • Problems of MyEnv with SuSE Linux Enterprise Server (SLES) were removed.
  • The tools for MySQL Partitions were extended and improved.

All improvements in detail you can find in the Release Notes.

Here you can download MyEnv.

MySQL Backup Manager v1.0.5

The MySQL Backup Manager (mysql_bman) is actually getting most of interest from our customers. It significantly eases backups for MySQL for all different types of backups.

At this point we would like to quote a MySQL user:

"MySQL Backup Manager is a very nice tool! Congratulations for FromDual! I made my own shell script for catalog and maintained backups by xtrabackup, but mysql_bman is the best! Xtrabackup + mysql_bman!!!"

In mysql_bman version v1.0.5 the following improvements were integrated:

  • Security improvements (password is not exposed any more).
  • Every instance can be tagged with a name and uniquely identified.
  • The MySQL Backup Manager considers now the MySQL configuration file ~/.my.cnf.
  • The compression of backups can be disabled to support de-duplicating drives.
  • The option --no-memory-table-check was introduced to allow inconsistent backups with MEMORY tables.

Download (included in MyEnv).

MySQL Ops Center v0.2

Our MySQL customers have requested a simple user interface to operate and administer many MySQL databases. This is the reason why FromDual has launched the MySQL Ops Center.
The Ops Center can centrally operate and control complex MySQL configurations like Master/Slave or Master/Master set-ups, monitor, start, stop and reconfigure the replication.
With the MySQL Ops Center you can also start and stop easily virtual IP's and move them to an other host.

The most important features, which were added to the first public preview-release of the MySQL Ops Center v0.2. are:

  • Starting and stopping of MySQL databases on remote machines by a central management console.
  • Starting and stopping of the MySQL replication.
  • Starting and stopping of a virtual IP (VIP).
  • Fail-over of VIP from active master to slave (master/slave replication) or passive master (master/master replication).
  • Configuration of the master/slave replication.

The MySQL Ops Center can be downloaded here. Further information you can find at MySQL Ops Center.

MySQL Performance Monitor v0.9.3

The MySQL Performance Monitor (mpm) was optimized in many places. Further know bugs were fixed and the mpm agent was made ready for the newest Zabbix version v2.2:

  • Bugs related to sha/sha1 encryption were fixed.
  • A stopped database is better detected now.
  • DRBD informations were improved.
  • New behaviour of zabbix_senders in Zabbix v2.2 is handled correctly now.
  • New measuring points was added (Galera Cluster) and wrong ones fixed.

You can download the latest version of MySQL Performance Monitor from here here and for more information about the manual installation just follow up the steps on the installation guide. To check all changes and improvements of MySQL Performance Monitor check out the Release Notes.

FromDual Performance Monitor for MySQL 0.9.3 has been released

FromDual.en - Wed, 2014-07-09 12:25
Taxonomy upgrade extras: mysqlperformancemonitormonitoringmpmmaasperformance monitor

FromDual has the pleasure to announce the release of the new version 0.9.3 of its popular Database Performance Monitor for MySQL, Galera Cluster, MariaDB and Percona Server mpm.

This release contains various minor bug fixes and improvements.

You can download mpm from here.

In the inconceivable case that you find a bug in mpm please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to

New installation of mpm v0.9.3

Please follow our mpm installation guide.

Upgrade from 0.x to 0.9.3 # cd /download # tar xf mysql_performance_monitor-0.9.3.tar.gz # cd /opt # tar xf /download/mysql_performance_monitor_agent-0.9.3.tar.gz # rm -f mpm # ln -s mysql_performance_monitor_agent-0.9.3 mpm

No other upgrade requirements are known.

Changes in mpm v0.9.3 mpm agent
  • Typos fixed.
  • Kill trap reports to the log file as well now.
mpm agent and MaaS
  • Example for timeshift feature added to configuration template.
MySQL module
  • DB down not detected (bug #27/#138).
InnoDB module
  • InnoDB Status module: SHA fix (bug #139).
Master module
  • Missing values in cache file fixed.
mpm templates for Zabbix
  • No changes.

Replication channel fail-over with Galera Cluster for MySQL

Shinguz - Thu, 2014-06-19 07:05
Taxonomy upgrade extras: channelgaleraclusterfail-overreplicationmasterslave

Sometimes it could be desirable to replicate from a Galera Cluster to a single MySQL slave or to an other Galera Cluster. Reasons for this measure could be:

  • An unstable network between two Galera Cluster locations.
  • A separation of a reporting slave and the Galera Cluster so that heavy reports on the slave do not affect the Galera Cluster performance.
  • Mixing different sources in a slave or a Galera Cluster (fan-in replication).

This article is based on earlier research work (see MySQL Cluster - Cluster circular replication with 2 replication channels) and uses the old MySQL replication style (without MySQL GTID).

  • Enable the binary logs on 2 nodes of a Galera Cluster (we call them channel masters) with the log_bin variable.
  • Set log_slave_updates = 1 on ALL Galera nodes.
  • It is recommended to have small binary logs and relay logs in such a situation to reduce overhead of scanning the files (max_binlog_size = 100M).


Let us assume that for some reason the actual channel master of channel 1 breaks. As a consequence the slave of channel 1 does not receive any replication events any more. But we have to keep the replication stream up and running. So we have to switch the replication channel to channel master 2.

Switching replication channel

First for security reasons we should stop the slave of replication channel 1 first:

mysql> STOP SLAVE;

Then we have to find the actual relay log on the slave:

mysql> pager grep Relay_Log_File mysql> SHOW SLAVE STATUS\G mysql> nopager Relay_Log_File: slave-relay-bin.000019

Next we have to find the last applied transaction on the slave:

mysql> SHOW RELAYLOG EVENTS IN 'slave-relay-bin.000019'; | slave-relay-bin.000019 | 3386717 | Query | 5201 | 53745015 | BEGIN | | slave-relay-bin.000019 | 3386794 | Table_map | 5201 | 53745067 | table_id: 72 (test.test) | | slave-relay-bin.000019 | 3386846 | Write_rows | 5201 | 53745142 | table_id: 72 flags: STMT_END_F | | slave-relay-bin.000019 | 3386921 | Xid | 5201 | 53745173 | COMMIT /* xid=1457451 */ | +------------------------+---------+-------------+-----------+-------------+--------------------------------+

This is transaction 1457451 which is the same on all Galera nodes.

On the new channel master of channel 2 we have to find now the matching binary log. This can be done best by matching times between the relay log and the binary log of master of channel 2.

On slave:

shell> ll *relay-bin* -rw-rw---- 1 mysql mysql 336 Mai 22 20:32 slave-relay-bin.000018 -rw-rw---- 1 mysql mysql 3387029 Mai 22 20:37 slave-relay-bin.000019

On master of channel 2:

shell> ll *bin-log* -rw-rw---- 1 mysql mysql 2518737 Mai 22 19:57 bin-log.000072 -rw-rw---- 1 mysql mysql 143 Mai 22 19:57 bin-log.000073 -rw-rw---- 1 mysql mysql 165 Mai 22 20:01 bin-log.000074 -rw-rw---- 1 mysql mysql 62953648 Mai 22 20:40 bin-log.000075

It looks like binary log 75 of master 2 matches to relay log of our slave.

Now we have to find the same transaction on the master of channel 2:

mysql> pager grep -B 6 1457451 mysql> SHOW BINLOG EVENTS IN 'bin-log.000075'; mysql> nopager | bin-log.000075 | 53744832 | Write_rows | 5201 | 53744907 | table_id: 72 flags: STMT_END_F | | bin-log.000075 | 53744907 | Xid | 5201 | 53744938 | COMMIT /* xid=1457450 */ | | bin-log.000075 | 53744938 | Query | 5201 | 53745015 | BEGIN | | bin-log.000075 | 53745015 | Table_map | 5201 | 53745067 | table_id: 72 (test.test) | | bin-log.000075 | 53745067 | Write_rows | 5201 | 53745142 | table_id: 72 flags: STMT_END_F | | bin-log.000075 | 53745142 | Xid | 5201 | 53745173 | COMMIT /* xid=1457451 */ | +----------------+----------+-------------+-----------+-------------+---------------------------------------+

We successfully found the transaction and want the position of the next transaction 53745173 where we should continue replicating.

As a last step we have to set the slave to the master of replication channel 2:

mysql> CHANGE MASTER TO master_host='master2', master_port=3306, master_log_file='bin-log.000075', master_log_pos=53745173; mysql> START SLAVE;

After a while the slave has caught up and is ready for the next fail-over back.


We found during our experiments that an IST of a channel master does not lead to a gap or loss of events in the replication stream. So restarting a channel master does not require a channel fail-over as long as an IST can be used for resyncing the channel master with the Galera Cluster.

The increase of wsrep_cluster_conf_id is NOT an indication that a channel fail-over is required.

A SST resets the binary logs so after the SST a slave will not replicate any more. So using this method should be safe to use. If you find any situation where you experience troubles with channel fail-over please let us know.

MySQL Environment MyEnv 1.0.5 has been released

FromDual.en - Fri, 2014-06-13 18:29
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackup

FromDual has the pleasure to announce the release of the new version 1.0.5 of its popular MySQL, MariaDB and Percona Server multi-instance environment MyEnv.

The majority of improvements happened on the MySQL Backup Manager (mysql_bman) utility.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to

Upgrade from 1.0.x to 1.0.5 # cd ${HOME}/product # tar xf /download/myenv-1.0.5.tar.gz # rm -f myenv # ln -s myenv-1.0.5 myenv
Upgrade from 1.0.2 or older to 1.0.3 or newer

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/

Exchange the MyEnv section in ~/.bash_profile (make a backup of this file first?) by the following new one:

# BEGIN MyEnv # Written by the MyEnv installMyEnv.php script. . /etc/myenv/MYENV_BASE MYENV_PWD=`pwd` cd $MYENV_BASE/bin . myenv.profile cd $MYENV_BASE; $MYENV_BASE/bin/showMyEnvStatus.php; cd - > /dev/null cd $MYENV_PWD # END MyEnv Changes in MyEnv 1.0.5 MyEnv
  • Schema output in up was still ugly.
  • Instance output is split correctly similar to up/down display.
  • Instance list is now shorter when short instance names are used.
  • ignore-passive option added for myEnv to ignore passive databases in an active/passive fail-over cluster. Based on existence of datadir.
  • Upgrade instructions have been improved and denormalized.
  • Only display existing OEM agents, criteria is directory in oratab must exist.
  • Up instances are not reported with missing mysqladmin command (Galera binary tar balls) but it was not visible what is the reason. Reason is displayed as an error message now.
MyEnv Installer
  • Lists each basedir candidate in a separate line when adding a new instance. More conveniant for reading if many basedirs are available.
MyEnv Utilities
  • Insert instead of Append used for firewall rules. Only block load-balancer ports and not everything else.
  • made more flexible.
MySQL Backup Manager
  • Cleanup job errors with missing target. Fixed for MGB.
  • Password on command line is not exposed anymore to log file.
  • Instance name optionally added to binary-log backup file names.
  • Binary logs are not cleaned-up because they are not copied with bck_ prefix (Bug #143).
  • Config file example in --help output done more nicely.
  • More strict option checking implemented.
  • All schemas with non transactional tables are shown instead of just the first one.
  • Help typo fixed and example improved.
  • --ignore-memory-table-check implemented to avoid error exit with MEMORY tables.
  • Preparation work for blocking MyISAM backup done.

Backup Manager for MySQL, MariaDB and Percona Server (mysql_bman)

Shinguz - Tue, 2014-05-06 17:28
Taxonomy upgrade extras: mysqlBackupRestoreRecoverymysql_bmanpitrAbout

The MySQL Backup Manager (mysql_bman) is a wrapper script for standard MySQL backup tools. The Problem with MySQL backup tools is, that they have many options and thus are overcomplicated and errors are easy made.

mysql_bman has the intention to make backups for MySQL easier and technically correct. This means it should per default not allow non-consistent backups or complain if some functions or parameters are used in the wrong way to guarantee proper backups.

In addition it has added some nice features which are missing in standard MySQL backup tools or which are only known from Enterprise backup solutions.

Where to download mysql_bman

The Backup Manager for MySQL (mysql_bman) can be downloaded from our website.

What mysql_bman user say about

Mathias Brem DBA@DBAOnline on LinkedIn:

Ow! Nice!
mysql backup manager is a very nice tool! Congratulations for FromDual! I made a shell script for catalog and maintained backups by xtrabackup, but mysql_bman is the best!

Xtrabackup + mysql_bman!!!!

Where can mysql_bman help you

The intention of mysql_bman is to assist you in bigger MySQL set-ups where you have to follow some backup policies and where you need a serious backup concept.

mysql_bman example

To give you an impression of the power of the MySQL Backup Manager let us have a look at a little example:

shell> mysql_bman --target=bman:secret@ --type=full --mode=logical --policy=daily \ --no-compress --backupdir=/mnt/slowdisk \ --archive --archivedir=/mnt/nfsmount

With this backup method we do a logical full backup (mysqldump is triggered in the background). The backup is stored in the location for backups with the daily policy and is NOT compressed to speed up the backup by saving CPU power AND because the backup device is a de-duplicating drive. Then the backup is archived to and NFS mount.

Backup types

To achieve this we have defined different backup types:

TypeDescriptionfullfull logical backup (mysqldump) of all schemasbinlogbinary-log backupconfigconfiguration file backup (my.cnf)structurestructure backupcleanupclean-up of backup pieces older than n daysschemabackup of one or more schemasprivilegeprivilege dump (SHOW GRANTS FOR)

A backup type is specified with the option --type=<backup_type>.

Backup modes

A backup can either be logical or physical. A logical backup is typically what you do with mysqldump. A physical backup is typically a physical file copy without looking into the data. That is what for example xtrabackup does.

The backup mode is specified with the option --mode=<backup_mode>. The following backup modes are available:

ModeDescriptionlogicaldo a logical backup (mysqldump).physicaldo a physical backup (mysqlbackup/innobackup/xtrabackup)Backup policies

Further we have introduced different backup policies. Policies are there to distinguish how different backups should be treated.

The following backup policies exist:

PolicyDescriptiondailydirectory to store daily backupsweeklydirectory to store weekly backupsmonthlydirectory to store monthly backupsquarterlydirectory to store quarterly backupsyearlydirectory to store yearly backups

For example you could plan to do a daily MySQL backup with binary logs with a retention policy of 7 days. But once a week you want to do a weekly backup consisting of a full backup, a configuration backup and a structure dump. But this weekly backup you want to keep for 6 months. And because of legal reasons you want to do a yearly backup with a retention policy of 10 years.

A backup policy is specified with the --policy=<backup_policy> option. This leads us to the retention time:


The retention time which should be applied to a specific backup policy you can specify with the option --retention=<period_in_days>. The retention option means that a backup is not deleted before this amount of days when you run a clean-up job with mysql_bman.

Let us do an example:

shell> mysql_bman --type=cleanup --policy=daily --retention=30

This means that all backups in the daily policy should be deleted when they are older than 30 days.


With the --target option you specify the connect string to the database to backup. This database can be located either local (all backup types can be used) or remote (only client/server backup types can be used).

A target looks as follows: user/password@host:port (similar to URI specification) whereas you can omit password and port.

Backup location, archiving, compressing and clean-up

The --backupdir option is to control location of the backup files. The policy folders are automatically created under this --backupdir location.
If you have a second layer of backup stores (e.g. tapes or slow backup drives or deduplicated drives or NFS drives) you can use the --archive option to copy your backup files to this second layer storage which is specified with the --archivedir option. For restore performance reasons it is recommended to always keep one or two generations of backups on you fast local drive. If you want to remove the backuped files from the --backupdir destination after the archive job use the --cleanup option.
If you want to omit to compress backups, either to safe time or because your location uses deduplicated drives you can use the --no-compress option.

Per schema backup

Especially for hosting companies a full database backup is typically not the right backup strategy because a restore of one specific customer (= schema) is very complicated. For this case we have the --per-schema option. mysql_bman will do a backup of the whole database schema by schema. Keep in mind: This breaks consistency among schemas!

Sometimes you want to do a schema backup only for some specific schemas for this you can use the --schema option. This option allows you to specify schemas to backup or not to backup. --schema=+a,+b means backup schema a and b. --schema=-a,-b means backup all schemas except a and b.
The second variant is less error prone because you do not forget to backup a new database.

Instance name

MySQL does not know the concept of naming an instance (mysqld). But for bigger environments it could be useful to uniquely name each instance. For this purpose we have introduced the option --instance-name=<give_it_a_name>. This instance name should be unique within your whole company. But we do not enforce it atm. The instance name is used to name backup files and later to identify the backup history of an instance in our backup catalog and to allow us to track the files for restore.

mysql_bman configuration file

Specifying everything on the command line is cumbersome. Thus mysql_bman considers a configuration file specified with the --config=<config_file> option.
A mysql_bman configuration file looks for example as follows:

policy = daily target = root/secret@ type = schema schema = -mysql policy = daily archive = on archivedir = /mnt/tape per-schema = on no-compress = on
Simulate what happens

For the Sissies among us (as for example me) we have the --simulate option. This option simulates nearly all steps as far as possible without executing really anything. This option is either for testing some features or for debugging purposes.


If you want to track your backup history you can specify with the --log option where your mysql_bman log file should be located.

Using Catalog

It will be very useful when you can store your backups metadata in the database so you can check them in the future and to find out the backup criteria (type, mode, instance-name, ... etc) for specific backup processes. This could be achieved by using the catalog feature.

To activate this feature you have to create a database for the catalog "default name is bman_catalog" then create its tables by using the option --create in a special mysql_bman command (check examples below).
Finally, to store your backup metadata in the catalog what you only have to do is adding the option --catalog=catalog_connection_string to the normal mysql_bman command.
Check the examples below for using catalog in mysql_bman.

More help

A little more help you can get with the following command:

shell> mysql_bman --help

Do a full (logical = default) backup and store it in the daily policy folder:

shell> mysql_bman --target=root/secret@ --type=full --policy=daily

Do a full physical backup and store it in the weekly policy folder:

shell> mysql_bman --target=root/secret@ --type=full --mode=physical --policy=weekly

Do a binary-log backup omitting the password in the target and store it in the daily policy folder:

shell> mysql_bman --target=bman@ --type=binlog --policy=daily

Do a MySQL configuration backup and store it in the weekly policy folder:

shell> mysql_bman --target=root/secret@ --type=config --policy=weekly

Do a structure backup and store it in the monthly policy folder and name the file with the instance name:

shell> mysql_bman --target=root/secret@ --type=structure --policy=monthly --instance-name=prod-db

Do a weekly structure backup and archive it to an other backup location:

shell> mysql_bman --target=root/secret@ --type=structure --policy=weekly --archive --archivedir=/mnt/tape

Do a schema backup omitting the mysql schema:

shell> mysql_bman --target=root/secret@ --type=schema --schema=-mysql --policy=daily --archive --archivedir=/mnt/tape

Do a schema backup only of foodmart and world and write it to their own files. Omit compressing these backups because they are located for example on deduplicated drives:

shell> mysql_bman --target=root/secret@ --type=schema --schema=+foodmart,+world --per-schema --policy=daily --no-compress

Creation of a backup catalog (assuming you have created already a catalog database with the default name "bman_catalog"):

shell> mysql_bman --catalog=root/secret@ --create

Backups against catalog:

shell> mysql_bman --target=root/secret@ --catalog=root/secret@ --instance-name=test --type=full --policy=daily

Privilege backup:

shell> mysql_bman --target=root/secret@ --type=privilege --policy=daily --mode=logical

FromDual Performance Monitor for MySQL 0.9.2 has been released

FromDual.en - Tue, 2014-05-06 13:58
Taxonomy upgrade extras: mysqlperformancemonitormonitoringmpmmaasperformance monitor

FromDual has the pleasure to announce the release of the new version 0.9.2 of its popular Database Performance Monitor for MySQL, MariaDB and Percona Server mpm.

This release makes mpm compatible with Zabbix v2.2 and contains various minor bug fixes and improvements.

You can download mpm from here.

In the inconceivable case that you find a bug in mpm please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to

New installation of mpm v0.9.2

Please follow our mpm installation guide.

Upgrade from 0.x to 0.9.2 # cd /download # tar xf mysql_performance_monitor-0.9.2.tar.gz # cd /opt # tar xf /download/mysql_performance_monitor_agent-0.9.2.tar.gz # rm -f mpm # ln -s mysql_performance_monitor_agent-0.9.2 mpm

No other upgrade requirements are known.

Changes in mpm v0.9.2 mpm agent
  • Fix of mpm_mr_version newline at EOL which affects zabbix_sender.
  • zabbix_sender return code change from Zabbix v2.2 and v2.1.7 fixed (bug #124).
  • Log rotate problem caught.
  • 2 exit errors fixed.
  • Default file locations adapted to the newest standard.
  • Cache file is removed instead of shrinked now, and remove lock file message should be at the right place.
  • Agent locking problem fixed.
  • Check of upload file size and shrink introduced.
mpm agent and MaaS
  • Proxy settings should be considered now for MaaS solution.
  • Data upload switched from http to https.
  • Send data made more verbose related to http/s send method errors and prepared for https only behaviour.
DRBD module
  • DRBD information is now reported correctly (bug #133).
Galera module
  • wsrep_last_committed item added.
mpm templates for Zabbix
  • Some items added to templates according to customer needs.


Subscribe to MySQL, Galera Cluster and MariaDB support and services aggregator - FromDual all (en)