News

Taxonomy upgrade extras: logginggaleraclusterconflictdeadlockerror logerror

We typically suggest our customers to use our MySQL/Galera Cluster my.cnf configuration template to avoid MySQL configuration and performance problems.

And we are paranoid as well. Thus we enable all useful logging:

wsrep_log_conflicts = 1

But this has also some consequences of more visibility...

If you monitor carefully your Galera Cluster for example with the FromDual Performance Monitor for MySQL and MariaDB, you might probably see some strange values increasing from time to time:

mysql< SHOW GLOBAL STATUS LIKE 'wsrep_local_%r_s'; +---------------------------+-------+ | Variable_name | Value | +---------------------------+-------+ | wsrep_local_cert_failures | 42 | | wsrep_local_bf_aborts | 13 | +---------------------------+-------+

Those values are indicators that some transactions (Galera write sets) did to not succeed and were aborted by Galera. In this case the paranoid logging helps to find, what exactly was aborted and possibly helps to find out, if this can or should be fixed:

150410 1:44:18 [Note] WSREP: cluster conflict due to certification failure for threads: 150410 1:44:18 [Note] WSREP: Victim thread: THD: 151856, mode: local, state: executing, conflict: cert failure, seqno: 30399304 SQL: UPDATE login SET lTsexpire = UNIX_TIMESTAMP(NOW()) + lTimeout WHERE lSessionId = 'va3ta7besku82k56ncv3bnhlj5' *** Priority TRANSACTION: TRANSACTION 464359568, ACTIVE 0 sec starting index read mysql tables in use 1, locked 1 1 lock struct(s), heap size 360, 0 row lock(s) MySQL thread id 4, OS thread handle 0x7f1c0916c700, query id 8190690 Update_rows_log_event::find_row(30399302) *** Victim TRANSACTION: TRANSACTION 464359562, ACTIVE 0 sec mysql tables in use 1, locked 1 2 lock struct(s), heap size 360, 1 row lock(s), undo log entries 1 MySQL thread id 151856, OS thread handle 0x7f1c09091700, query id 8190614 172.20.100.11 sam_angiz query end UPDATE login SET lTsexpire = UNIX_TIMESTAMP(now()) + lTimeout WHERE lSessionId = 'va3ta7besku82k56ncv3bnhlj5' *** WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 835205 page no 3 n bits 72 index `PRIMARY` of table `fromdual`.`login` trx table locks 1 total table locks 2 trx id 464359562 lock_mode X locks rec but not gap lock hold time 0 wait time before grant 0 150410 1:44:18 [Note] WSREP: cluster conflict due to high priority abort for threads: 150410 1:44:18 [Note] WSREP: Winning thread: THD: 4, mode: applier, state: executing, conflict: no conflict, seqno: 30399302 SQL: (null) 150410 1:44:18 [Note] WSREP: Victim thread: THD: 151856, mode: local, state: committing, conflict: no conflict, seqno: -1 SQL: UPDATE login SET lTsexpire = UNIX_TIMESTAMP(now()) + lTimeout WHERE lSessionId = 'va3ta7besku82k56ncv3bnhlj5'

In the above Galera conflict 2 login transactions where running at the same time. They both come with the same Session ID and want to update the expiry timestamp. Now how to solve or fix this:

• First check, if this table has a Primary Key (tables without a PK causes full table scans which can last for long time, increasing the chance for conflicts).
• Second check, if there is a (UNIQUE?) index on lSessionId. A missing index leads to full table scans which increases the chance for conflicts.
• Third check WHY 2 logins from the same Session ID can arrive at the same time (within 1 second) on 2 different Galera nodes (Ajax requests, etc...). Try to avoid such situations.

Taxonomy upgrade extras: galeravmwaresnapshot

From time to time we see at Galera Cluster customer engagements the following, for me scary, warning in the MySQL error log:

[Warning] WSREP: last inactive check more than PT1.5S ago (PT7.06159S), skipping check

We mostly see this in VMware set-ups. Some further enquiry with the Galera developers did not give a satisfying answer:

This can be seen on bare metal as well - with poorly configured mysqld, O/S, or simply being overloaded. All it means is that this thread could not get CPU time for 7.1 seconds. You can imagine that access to resources in virtual machines is even harder (especially I/O) than on bare metal, so you will see this in virtual machines more often.

This is not a Galera specific issue (it just reports being stuck, other mysqld threads are equally stuck) so there is no configuration options for that. You simply must make sure that your system and mysqld are properly configured, that there is enough RAM (buffer pool not over provisioned), that there is swap, that there are proper I/O drivers installed on guest and so on.

Basically, Galera runs in virtual machines as well as well virtual machines approximates bare metal.

We were still suspecting that this is somehow VMware related. This week we had the chance to investigate... At 01:36 am node Galera2 lost connection to the Cluster and became NON-PRIMARY. This is basically a bad sign:

150401 1:36:15 [Warning] WSREP: last inactive check more than PT1.5S ago (PT5.08325S), skipping check 150401 1:36:15 [Note] WSREP: (09c6b2f2, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://192.168.42.2:4567 150401 1:36:16 [Note] WSREP: view(view_id(NON_PRIM,09c6b2f2,30) memb { 09c6b2f2,0 } joined { } left { } partitioned { ce6bf2e1,0 d1f9bee0,0 }) 150401 1:36:16 [Note] WSREP: view(view_id(NON_PRIM,09c6b2f2,31) memb { 09c6b2f2,0 } joined { } left { } partitioned { ce6bf2e1,0 d1f9bee0,0 }) 150401 1:36:16 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 150401 1:36:16 [Note] WSREP: Flow-control interval: [16, 16] 150401 1:36:16 [Note] WSREP: Received NON-PRIMARY. 150401 1:36:16 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 26304132) 150401 1:36:16 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 150401 1:36:16 [Note] WSREP: Flow-control interval: [16, 16] 150401 1:36:16 [Note] WSREP: Received NON-PRIMARY. 150401 1:36:16 [Warning] WSREP: Send action {(nil), 328, TORDERED} returned -107 (Transport endpoint is not connected) 150401 1:36:16 [Note] WSREP: New cluster view: global state: dcca768c-b5ad-11e3-bbc0-fb576fb3c451:26304132, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3 150401 1:36:17 [Note] WSREP: (09c6b2f2, 'tcp://0.0.0.0:4567') reconnecting to d1f9bee0 (tcp://192.168.42.1:4567), attempt 0

I suspected, after some investigation with the FromDual Performance Monitor for MySQL and MariaDB, that the database backup (mysqldump) could be the reason. It was not. But the customer explained, that after the database backup they do a VMware snapshot.

And when we compared our problem with the backup log file:

2015/04/01 01:35:08 [3] backup.fromdual.com: Creating a snapshot of galera3 2015/04/01 01:35:16 [3] backup.fromdual.com: Created a snapshot of galera3 2015/04/01 01:35:23 [3] backup.fromdual.com: galera3: backup the changed blocks of disk 'Festplatte 1' using NBD transport 2015/04/01 01:36:10 [3] backup.fromdual.com: galera3: saving the Change Block Tracking's reference for disk 'Festplatte 1' 2015/04/01 01:36:10 [3] backup.fromdual.com: Removing Arkeia's snapshot of galera3

we can see that our problem pretty much started with the end of the WMware snapshot (01:36:10 + 5.08 = 1:36:15). By the way: For such kind of investigations it is always good to have a ntp daemon for time synchronization running. Otherwise problem investigation becomes much harder...

Some more and deeper investigation shows that we loose from time to time nodes during VMware snapshots (galera3). But they recover quickly because they can do an IST. In worst case we can loose 2 nodes and then the whole Galera Cluster has gone.

192.168.42.3 / node Galera3 2015-04-10 01:44:00 [3] backup.fromdual.com: Creating a snapshot of galera3 2015-04-10 01:44:08 [3] backup.fromdual.com: Created a snapshot of galera3 2015-04-10 01:44:15 [3] backup.fromdual.com: galera3: backup the changed blocks of disk 'Festplatte 1' using NBD transport 2015-04-10 01:45:39 [3] backup.fromdual.com: galera3: saving the Change Block Tracking's reference for disk 'Festplatte 1' 2015-04-10 01:45:39 [3] backup.fromdual.com: Removing Arkeia's snapshot of galera3
150410 1:44:07 [Note] WSREP: (158f71de, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://galera1:4567 tcp://galera2:4567 150410 1:44:07 [Warning] WSREP: last inactive check more than PT1.5S ago (PT7.06159S), skipping check 150410 1:44:08 [Note] WSREP: Received NON-PRIMARY. 150410 1:44:10 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 30399299) 150410 1:44:11 [Warning] WSREP: Gap in state sequence. Need state transfer. 150410 1:44:11 [Note] WSREP: Prepared IST receiver, listening at: tcp://galera3:4568 150410 1:44:11 [Note] WSREP: Member 0.0 (galera3) requested state transfer from '*any*'. Selected 2.0 (galera2)(SYNCED) as donor. 150410 1:44:11 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 30399309) 150410 1:44:11 [Note] WSREP: Requesting state transfer: success, donor: 2 150410 1:44:11 [Note] WSREP: 2.0 (galera2): State transfer to 0.0 (galera3) complete. 150410 1:44:11 [Note] WSREP: Member 2.0 (galera2) synced with group. 150410 1:44:11 [Note] WSREP: Receiving IST: 8 writesets, seqnos 30399291-30399299 150410 1:44:11 [Note] WSREP: IST received: dcca768c-b5ad-11e3-bbc0-fb576fb3c451:30399299 150410 1:44:11 [Note] WSREP: 0.0 (galera3): State transfer from 2.0 (galera2) complete. 150410 1:44:11 [Note] WSREP: Shifting JOINER -> JOINED (TO: 30399309) 150410 1:44:11 [Note] WSREP: Member 0.0 (galera3) synced with group. 150410 1:44:11 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 30399309) 150410 1:44:11 [Note] WSREP: Synchronized with group, ready for connections 150410 1:44:13 [Note] WSREP: (158f71de, 'tcp://0.0.0.0:4567') turning message relay requesting off 150410 1:45:42 [Warning] WSREP: last inactive check more than PT1.5S ago (PT2.47388S), skipping check 150410 1:45:43 [Note] WSREP: (158f71de, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://galera1:4567 tcp://galera2:4567 150410 1:45:44 [Note] WSREP: (158f71de, 'tcp://0.0.0.0:4567') reconnecting to 54de92f8 (tcp://galera1:4567), attempt 0 150410 1:45:44 [Note] WSREP: (158f71de, 'tcp://0.0.0.0:4567') reconnecting to c9d964d3 (tcp://galera2:4567), attempt 0 150410 1:45:48 [Note] WSREP: (158f71de, 'tcp://0.0.0.0:4567') turning message relay requesting off 150410 1:47:26 [Note] WSREP: (158f71de, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://galera1:4567 150410 1:47:27 [Note] WSREP: (158f71de, 'tcp://0.0.0.0:4567') reconnecting to 54de92f8 (tcp://galera1:4567), attempt 0 150410 1:47:31 [Note] WSREP: (158f71de, 'tcp://0.0.0.0:4567') turning message relay requesting off
192.168.42.1 / node Galera1 2015-04-10 01:47:24 [3] backup.fromdual.com: Creating a snapshot of galera1 2015-04-10 01:47:29 [3] backup.fromdual.com: Created a snapshot of galera1 2015-04-10 01:47:40 [3] backup.fromdual.com: galera1: backup the changed blocks of disk 'Festplatte 1' using NBD transport 2015-04-10 01:48:43 [3] backup.fromdual.com: galera1: saving the Change Block Tracking's reference for disk 'Festplatte 1' 2015-04-10 01:48:44 [3] backup.fromdual.com: Removing Arkeia's snapshot of galera1 150410 1:44:02 [Note] WSREP: (54de92f8, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://galera3:4567 150410 1:44:04 [Note] WSREP: (54de92f8, 'tcp://0.0.0.0:4567') reconnecting to 158f71de (tcp://galera3:4567), attempt 0 150410 1:44:12 [Note] WSREP: Member 0.0 (galera3) requested state transfer from '*any*'. Selected 2.0 (galera2)(SYNCED) as donor. 150410 1:45:43 [Note] WSREP: (54de92f8, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://galera3:4567 150410 1:45:44 [Note] WSREP: (54de92f8, 'tcp://0.0.0.0:4567') reconnecting to 158f71de (tcp://galera3:4567), attempt 0 150410 1:45:48 [Note] WSREP: (54de92f8, 'tcp://0.0.0.0:4567') turning message relay requesting off 150410 1:47:27 [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.66452S), skipping check 150410 1:47:27 [Note] WSREP: (54de92f8, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://galera3:4567 150410 1:47:30 [Note] WSREP: (54de92f8, 'tcp://0.0.0.0:4567') turning message relay requesting off
192.168.42.2 / node Galera2 2015-04-10 02:09:55 [3] backup.fromdual.com: Creating a snapshot of galera2 2015-04-10 02:09:58 [3] backup.fromdual.com: Created a snapshot of galera2 2015-04-10 02:10:05 [3] backup.fromdual.com: galera2: backup the changed blocks of disk 'Festplatte 1' using NBD transport 2015-04-10 02:10:53 [3] backup.fromdual.com: galera2: saving the Change Block Tracking's reference for disk 'Festplatte 1' 2015-04-10 02:10:54 [3] backup.fromdual.com: Removing Arkeia's snapshot of galera2
150410 1:44:02 [Note] WSREP: (c9d964d3, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://galera3:4567 150410 1:44:03 [Note] WSREP: (c9d964d3, 'tcp://0.0.0.0:4567') reconnecting to 158f71de (tcp://galera3:4567), attempt 0 150410 1:44:08 [Warning] WSREP: discarding established (time wait) 158f71de (tcp://192.168.42.3:4567) 150410 1:44:11 [Note] WSREP: Member 0.0 (galera3) requested state transfer from '*any*'. Selected 2.0 (galera2)(SYNCED) as donor. 150410 1:44:13 [Note] WSREP: (c9d964d3, 'tcp://0.0.0.0:4567') turning message relay requesting off 150410 1:45:43 [Note] WSREP: (c9d964d3, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://galera3:4567 150410 1:45:44 [Note] WSREP: (c9d964d3, 'tcp://0.0.0.0:4567') reconnecting to 158f71de (tcp://galera3:4567), attempt 0 150410 1:45:48 [Note] WSREP: (c9d964d3, 'tcp://0.0.0.0:4567') turning message relay requesting off 150410 1:47:26 [Note] WSREP: (c9d964d3, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://galera1:4567 150410 1:47:27 [Note] WSREP: (c9d964d3, 'tcp://0.0.0.0:4567') reconnecting to 54de92f8 (tcp://galera1:4567), attempt 0 150410 1:47:30 [Note] WSREP: (c9d964d3, 'tcp://0.0.0.0:4567') turning message relay requesting off 150410 2:09:57 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.83618S), skipping check

The backups are done with the 2 options:

• Change Block Tracking (CBT) (Arkeia, VMware) and
• Quiesce Guest File System (Arkeia, VMware)

enabled.

Possibly this is the reason and one should disable those features in combination with Galera. Further investigation is going on. In worst case VMware snapshotting with Galera should be avoided.

Taxonomy upgrade extras: partitionrenameDDL

Before I forget it and have to search again here a short note about how to rename a MySQL Partition:

My dream:

ALTER TABLE history RENAME PARTITION p2015_kw10 INTO p2015_kw09;
In reality: ALTER TABLE history REORGANIZE PARTITION p2015_kw10 INTO ( PARTITION p2015_kw09 VALUES LESS THAN (UNIX_TIMESTAMP('2015-03-02 00:00:00')) );

Caution: REORGANIZE PARTITION causes a full copy of the whole partition!

Hint: I assume it would be very easy for MySQL or MariaDB to make this DDL command an in-place operation...

MySQL Partitioning was introduced in MySQL 5.1.

FromDual is pleased to announce the release of a new version 1.0.0 of the Nagios and Icinga plug-ins for MySQL, MariaDB, Percona Server and Galera Cluster.

Any information about the changes and the new functions of the Nagios and Icinga plug-ins you can find here.

If you have any problems with the new version you can reach us by e-mail at: contact@fromdual.com or by phone on +41 44 500 58 20.

Your FromDual Team

Taxonomy upgrade extras: nagiosicingaplug-inmysqlmariadbpercona serverGalera Clusterrelease

FromDual is pleased to announce the release of a new version 1.0.0 of the Nagios and Icinga plug-ins for MySQL, MariaDB, Percona Server and Galera Cluster.

Any information about the changes and the new functions of the Nagios and Icinga plug-ins you can find here.

If you have any problems with the new version you can reach us by e-mail at: contact@fromdual.com or by phone on +41 44 500 58 20.

Your FromDual Team

Taxonomy upgrade extras: nagiosicingaplug-inmysqlmariadbpercona serverGalera Clusterrelease

Taxonomy upgrade extras: nagiosicingaplug-inmysqlmariadbpercona serverGalera Clusterrelease

FromDual is pleased to announce the release of a new version 1.0.0 of the Nagios and Icinga plug-ins for MySQL, MariaDB, Percona Server and Galera Cluster.

Any information about the changes and the new functions of the Nagios and Icinga plug-ins you can find here.

If you have any problems with the new version you can reach us by e-mail at: contact@fromdual.com or by phone on +41 44 940 24 82.

Your FromDual Team

Taxonomy upgrade extras: mebMySQL Enterprise BackupenterpriseBackupincremental backup

MySQL Enterprise Backup (MEB) has the capability to make real incremental (differential and cumulative?) backups. The actual releases are quite cool and you should really look at it...

Unfortunately the original MySQL documentation is much too complicated for my simple mind. So I did some testing and simplified it a bit for our customers...

If you want to dive into the original documentation please look here: Making an Incremental Backup .

If you want to use MySQL Enterprise Backup please let us know and we send you a quote...

Prepare MySQL Backup infrastructure mkdir /backup/full /backup/incremental1 /backup/incremental2
Full MySQL Backup mysqlbackup --defaults-file=/etc/my.cnf --user=root --backup-dir=/backup/full backup mysqlbackup --defaults-file=/etc/my.cnf --user=root --backup-dir=/backup/full apply-log
First MySQL Incremental Backup mysqlbackup --defaults-file=/etc/my.cnf --user=root --incremental --incremental-base=dir:/backup/full --incremental-backup-dir=/backup/incremental1 backup mysqlbackup --defaults-file=/etc/my.cnf --user=root --backup-dir=/backup/full --incremental-backup-dir=/backup/incremental1 apply-incremental-backup
Second MySQL Incremental Backup mysqlbackup --defaults-file=/etc/my.cnf --user=root --incremental --incremental-base=dir:/backup/full --incremental-backup-dir=/backup/incremental2 backup mysqlbackup --defaults-file=/etc/my.cnf --user=root --backup-dir=/backup/full --incremental-backup-dir=/backup/incremental2 apply-incremental-backup

and so on...

MySQL Restore mysqlbackup --defaults-file=/etc/my.cnf --user=root --backup-dir=/backup/full copy-back

Have fun with MySQL Enterprise Backup. If you need any help with your MySQL Backup concept, please let us know.

Taxonomy upgrade extras: MySQL Enterprise Monitormonitoringeventhandlermpmperformance monitor

MySQL Enterprise Monitor (MEM) has by default no Event Handlers created and activated. These Event Handlers you have to define yourself according to your needs.

In this article we discuss how to create MySQL Enterprise Monitor Event Handlers with MEM v.3.0.18. For other (older) versions the steps may vary...

Task: Event Handler for maximum Connections reached

We would like to be notified by MySQL Enterprise Monitor when the number of connections is near to max_connections.

For this we search first which Advisors are available at all: Configuration -> Advisors -> Availability.

Here we can see that we have an Advisor called Maximum Connection Limit Nearing Or Reached which is scheduled for every 5 minutes and has thresholds at 75, 85, 95 and 100%:

Now we know which Advisor should create and Event. As a next step we have to create and Event which should be triggered: Configuration -> Event Handling -> Create Event Handler.

Here we can create and Event with all its needed configuration: Events -> All -> server.

If we look at the Events we can even see the detailed description and how the values for the Event are collected:

Task: Event Handler for used disk space

For this Event Handler we need the Advisor Filesystem Free Space under Operating System:

In this advisor we can configure the Threshold as well:

In the Event Handler we can define which Assets shall be monitored. For example the mountpoint: /.

Local disks can only be monitored, if a local MySQL Enterprise Monitor Agent is installed. An agent-less MySQL Enterprise Monitor cannot monitor local disk resources...

Have fun using the MySQL Enterprise Monitor. If you need any help in installing or configuring MEM do not hesitate to contact us.

All these functions are also implemented in the FromDual Performance Monitor for MySQL. If you want to relay on Open Source technology only you should consider our Performance Monitor.

FromDual has the pleasure to announce the release of the new version 1.1.4 of its popular MySQL, Galera Cluster, MariaDB and Percona Server multi-instance environment MyEnv.

The new MyEnv can be downloaded here.

In the inconceivable case that you find a bug in the MyEnv please report it to our bug tracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.1.x to 1.1.4 # cd ${HOME}/product # tar xf /download/myenv-1.1.4.tar.gz # rm -f myenv # ln -s myenv-1.1.4 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/
Changes in MyEnv 1.1.4

Because an ugly but non-dangerous bugs we skipped the announcement of version 1.1.3.

All changes since MyEnv 1.1.2 are listed here. We recommend to skip version 1.1.3

MyEnv

• Error messages are sent to stderr instead of stdout, start error messages are tested after up test.
• Error message if wrong user wants to stop database.
• MyEnv init script slightly improved.
• False positive error message in start fixed.
• Variable parsing in my.cnf made more robust.
• Error messages and warnings made more verbose/clear.
• Bug during stopping database with dummy instance fixed (MGB).
• Cgroups functionality added.
• PHP debug message is now just displayed in debug mode.

MyEnv Installer

• installMyEnv cleans up my.cnf in basedir if it did not exists before to avoid nasty warning message.
• Move [client] and [mysqldump] section to top to not forget them later.
• Bug with empty myenv.conf fixed (KN).
• Install advice for SuSE was wrong for runlevels for MyEnv.
• my.cnf template updated.
• /tmp as location for socket removed because of RedHat tmpcleaner.
• datadir redundancy removed.
• lost+found folder was added to hideschema.
• installMyEnv.sh replaced by installMyEnv.

MyEnv Utilities

• Partitioning tools error output is now written to stderr correctly.
• log_maintenance.sh script contribution added (TS).
• HAproxy check added.
• purge_binary_log.py added.
• insert_test.sh improved.
• lb.monitor skript added.

For subscriptions of commercial use of MyEnv please get in contact with us.

Taxonomy upgrade extras: MyEnvoperationMySQL Operationsmulti instanceconsolidationtestingrelease

FromDual has the pleasure to announce the release of the new version 1.1.4 of its popular MySQL, Galera Cluster, MariaDB and Percona Server multi-instance environment MyEnv.

The new MyEnv can be downloaded here.

In the inconceivable case that you find a bug in the MyEnv please report it to our bug tracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.1.x to 1.1.4 # cd ${HOME}/product # tar xf /download/myenv-1.1.4.tar.gz # rm -f myenv # ln -s myenv-1.1.4 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/
Changes in MyEnv 1.1.4

Because an ugly but non-dangerous bugs we skipped the announcement of version 1.1.3.

All changes since MyEnv 1.1.2 are listed here. We recommend to skip version 1.1.3

MyEnv

• Error messages are sent to stderr instead of stdout, start error messages are tested after up test.
• Error message if wrong user wants to stop database.
• MyEnv init script slightly improved.
• False positive error message in start fixed.
• Variable parsing in my.cnf made more robust.
• Error messages and warnings made more verbose/clear.
• Bug during stopping database with dummy instance fixed (MGB).
• Cgroups functionality added.
• PHP debug message is now just displayed in debug mode.

MyEnv Installer

• installMyEnv cleans up my.cnf in basedir if it did not exists before to avoid nasty warning message.
• Move [client] and [mysqldump] section to top to not forget them later.
• Bug with empty myenv.conf fixed (KN).
• Install advice for SuSE was wrong for runlevels for MyEnv.
• my.cnf template updated.
• /tmp as location for socket removed because of RedHat tmpcleaner.
• datadir redundancy removed.
• lost+found folder was added to hideschema.
• installMyEnv.sh replaced by installMyEnv.

MyEnv Utilities

• Partitioning tools error output is now written to stderr correctly.
• log_maintenance.sh script contribution added (TS).
• HAproxy check added.
• purge_binary_log.py added.
• insert_test.sh improved.
• lb.monitor skript added.

For subscriptions of commercial use of MyEnv please get in contact with us.

Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationtestingupgraderelease

Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationtestingupgrade

FromDual has the pleasure to announce the release of the new version 1.1.4 of its popular MySQL, Galera Cluster, MariaDB and Percona Server multi-instance environment MyEnv.

The new MyEnv can be downloaded here.

In the inconceivable case that you find a bug in the MyEnv please report it to our bug tracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.1.x to 1.1.4 # cd ${HOME}/product # tar xf /download/myenv-1.1.4.tar.gz # rm -f myenv # ln -s myenv-1.1.4 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/
Changes in MyEnv 1.1.4

Because an ugly but non-dangerous bugs we skipped the announcement of version 1.1.3.

All changes since MyEnv 1.1.2 are listed here. We recommend to skip version 1.1.3

MyEnv

• Error messages are sent to stderr instead of stdout, start error messages are tested after up test.
• Error message if wrong user wants to stop database.
• MyEnv init script slightly improved.
• False positive error message in start fixed.
• Variable parsing in my.cnf made more robust.
• Error messages and warnings made more verbose/clear.
• Bug during stopping database with dummy instance fixed (MGB).
• Cgroups functionality added.
• PHP debug message is now just displayed in debug mode.

MyEnv Installer

• installMyEnv cleans up my.cnf in basedir if it did not exists before to avoid nasty warning message.
• Move [client] and [mysqldump] section to top to not forget them later.
• Bug with empty myenv.conf fixed (KN).
• Install advice for SuSE was wrong for runlevels for MyEnv.
• my.cnf template updated.
• /tmp as location for socket removed because of RedHat tmpcleaner.
• datadir redundancy removed.
• lost+found folder was added to hideschema.
• installMyEnv.sh replaced by installMyEnv.

MyEnv Utilities

• Partitioning tools error output is now written to stderr correctly.
• log_maintenance.sh script contribution added (TS).
• HAproxy check added.
• purge_binary_log.py added.
• insert_test.sh improved.
• lb.monitor skript added.

For subscriptions of commercial use of MyEnv please get in contact with us.

Taxonomy upgrade extras: nagiosicingaplug-inmonitorperformancealert

FromDual has the pleasure to announce the release of the new version 1.0.0 of its widely used Nagios and Icinga plug-ins for MySQL, Galera Cluster, MariaDB and Percona Server.

All plug-ins are basically renewed and should now work all correctly.

The new Nagios/Icinga plug-ins can be downloaded here.

In the inconceivable case that you find a bug in the Nagios/Icinga plug-ins please report it to our bug tracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Description of the current functionality

Details about the functionality and the usage of each plug-in you get with the option: --help.

The following Nagios/Icinga plug-in for MySQL and MariaDB are currently available:

check_db_mysql.pl

This Nagios/Icinga plug-in alerts you if your MySQL database is not up and running.

check_errorlog_mysql.pl and errorLogFilterRules.pm

This Nagios/Icinga plug-in alerts you if it finds some suspicious messages in the MySQL error log.
The rules which messages should be ignored can be found in the file errorLogFilterRules.pm. If you want to add your own filter rules please add them in this file as well.

check_galera_nodes.pl

This Nagios/Icinga plug-in alerts you if the actual number of nodes in your a Galera Cluster is not the expected one.

check_repl_mysql_cnt_slave_hosts.pl

This Nagios/Icinga plug-in alerts you if your MySQL Slaves have not reported to their Master properly their existence with the report_host variable.

check_repl_mysql_heartbeat.pl

This Nagios/Icinga plug-in alerts you if your MySQL Slave is too many heartbeats behind its Master.

check_repl_mysql_io_thread.pl

This Nagios/Icinga plug-in alerts you if your MySQL Slaves IO thread is not up an running.

check_repl_mysql_read_exec_pos.pl

This Nagios/Icinga plug-in alerts you if your MySQL Slaves read and execution positions differ too much.

check_repl_mysql_readonly.pl

This Nagios/Icinga plug-in alerts you if your MySQL Slave is NOT set to readonly.

check_repl_mysql_seconds_behind_master.pl

This Nagios/Icinga plug-in alerts you if your MySQL Slave falls too many seconds behind its Master.

check_repl_mysql_sql_thread.pl

This Nagios/Icinga plug-in alerts you if your Slaves SQL thread is not up an running.

perf_mysql.pl

This Nagios/Icinga plug-in gathers MySQL and MariaDB performance data.

Changes in FromDual Nagios/Icinga plug-ins 1.0.0 All plug-ins

• Usage was improved. The usage can be shown with the --help option.
• Usage states now which GRANT privileges are needed for a specific plug-in.
• Examples added how to use each plug-in.
• Default socket location moved from /tmp/mysql.sock to /var/run/mysqld/mysqld.sock.
• New host/socket convention implemented in all scripts similar to MySQL client tools.
• -epn tag added for Icinga.

check_errorlog_mysql.pl

• Some bugs fixed.
• More filtering rules added.
• Filtering rules separated into own file.
• Entry point finding problem fixed.

check_repl_mysql_heartbeat.pl

• Script name fixed.

check_db.pl

• Unknown command problem with Galera Cluster caught.
• mysqladmin ping removed and implemented in Perl.

Support and Subscription for commercial use

For subscriptions for commercial use of this software please get in contact with us.

At FromDual, we are currently preparing for our participation in the "Chemnitzer Linux-Tage" in March.
While we don't yet know whether the programme committee accepted our proposed talks, we will have a booth and hope for interesting exchanges with others from the MySQL, database, Linux, ... world. Of course, we will also mention that we are looking for additional colleagues - there are so many tasks that we need more people to handle them all. (In case you got curious, look here: http://www.fromdual.com/mysql-dba-2014-12-de )

While I attended the Chemnitzer Linux-Tage already last year (thank you, Frank Hofmann, for recommending them!), it will be the first time there for FromDual as a company. I enjoyed the informal atmosphere, which made it easy to get into contact with new people, to learn, and to discuss both facts and opinions.

To those of you who didn't yet attend such events, I can only say: You are missing some great experiences! It is good to get stimulated by others, by their problems and their experiences, so that one's thoughts are broadened and start considering new fields. Often, they lead to insights that are helpful for one's own daily work.

In the opposite direction, we at FromDual hope that during the Linux-Tage we can help others to make (better) use of MySQL in all its variants, with its surrounding tools and products, so that everybody will profit.

The Chemnitzer Linux-Tage will take place on March 21 and 22, 2015, in Chemnitz, Germany. Parts of the programme will probably be delivered in English, so even those of you who don't understand German might enjoy coming there. If you now consider a visit, you will find further information here:
https://chemnitzer.linux-tage.de/2015/de

We will be happy to meet you in Chemnitz - to talk about whatever interests both you and us, maybe even about you joining us?

At FromDual, we are currently preparing for our participation in the "Chemnitzer Linux-Tage" in March.
While we don't yet know whether the programme committee accepted our proposed talks, we will have a booth and hope for interesting exchanges with others from the MySQL, database, Linux, ... world. Of course, we will also mention that we are looking for additional colleagues - there are so many tasks that we need more people to handle them all. (In case you got curious, look here: http://www.fromdual.com/mysql-dba-2014-12-de )

While I attended the Chemnitzer Linux-Tage already last year (thank you, Frank Hofmann, for recommending them!), it will be the first time there for FromDual as a company. I enjoyed the informal atmosphere, which made it easy to get into contact with new people, to learn, and to discuss both facts and opinions.

To those of you who didn't yet attend such events, I can only say: You are missing some great experiences! It is good to get stimulated by others, by their problems and their experiences, so that one's thoughts are broadened and start considering new fields. Often, they lead to insights that are helpful for one's own daily work.

In the opposite direction, we at FromDual hope that during the Linux-Tage we can help others to make (better) use of MySQL in all its variants, with its surrounding tools and products, so that everybody will profit.

The Chemnitzer Linux-Tage will take place on March 21 and 22, 2015, in Chemnitz, Germany. Parts of the programme will probably be delivered in English, so even those of you who don't understand German might enjoy coming there. If you now consider a visit, you will find further information here:
https://chemnitzer.linux-tage.de/2015/de

We will be happy to meet you in Chemnitz - to talk about whatever interests both you and us, maybe even about you joining us?

At FromDual, we are currently preparing for our participation in the "Chemnitzer Linux-Tage" in March.
While we don't yet know whether the programme committee accepted our proposed talks, we will have a booth and hope for interesting exchanges with others from the MySQL, database, Linux, ... world. Of course, we will also mention that we are looking for additional colleagues - there are so many tasks that we need more people to handle them all. (In case you got curious, look here: http://www.fromdual.com/mysql-dba-2014-12-de )

While I attended the Chemnitzer Linux-Tage already last year (thank you, Frank Hofmann, for recommending them!), it will be the first time there for FromDual as a company. I enjoyed the informal atmosphere, which made it easy to get into contact with new people, to learn, and to discuss both facts and opinions.

To those of you who didn't yet attend such events, I can only say: You are missing some great experiences! It is good to get stimulated by others, by their problems and their experiences, so that one's thoughts are broadened and start considering new fields. Often, they lead to insights that are helpful for one's own daily work.

In the opposite direction, we at FromDual hope that during the Linux-Tage we can help others to make (better) use of MySQL in all its variants, with its surrounding tools and products, so that everybody will profit.

The Chemnitzer Linux-Tage will take place on March 21 and 22, 2015, in Chemnitz, Germany. Parts of the programme will probably be delivered in English, so even those of you who don't understand German might enjoy coming there. If you now consider a visit, you will find further information here:
https://chemnitzer.linux-tage.de/2015/de

We will be happy to meet you in Chemnitz - to talk about whatever interests both you and us, maybe even about you joining us?

Taxonomy upgrade extras: enterprise monitorBackupworkbenchenterprise

MySQL provides some great enterprise features beside the MySQL Server. The ones we are asked the most at customers are:

• MySQL Enterprise Backup (MEB)
• MySQL Enterprise Monitor (MEM) and
• MySQL Enterprise Workbench (MWB)

MySQL Enterprise Backup (MEB)

MySQL Enterprise Backup (MEB) is an alternative to the mysqldump backup utility. Its big advantage is its fast backup but even faster restore performance. This is a must for all MySQL users having bigger databases than let's say 10 to 20 Gigabytes and/or having hard requirements for restore times (MTTR).

Last implementation tests we did with a customer for an about 30 Gbyte database were:

 MEBmysqldumpBackup10 minutes18 minutesRestore12 minutes80 minutes

If you need our help implementing MySQL Enterprise Backup into your backup infrastructure please get in contact with us. MySQL Enterprise Backup also seamlessly integrates into the FromDual Backup Manager for MySQL.

MySQL Enterprise Monitor (MEM)

MySQL Enterprise Monitor (MEM) is an Enterprise Monitoring Solution for MySQL which Monitors your business critical MySQL databases. Various predefined advisors rise an alert if something with your precious MySQL database does not work as expected.

Our alternative competitive product is the FromDual Performance Monitor for MySQL.

MySQL Enterprise Workbench (MWB)

MySQL Enterprise Workbench (MWB) is the tool modern MySQL Database administrators use to operate their MySQL databases. Old fashion ones still use the CLI... MySQL developers can easily write and test database queries and develop their data model with the ER diagram modeller.

Download Enterprise tools

But how can we get now to these precious tools? This is quite easy following the screen shots below:

As a fist step you go to www.mysql.com/downloads:

Here you can find a link to Oracle eDelivery which is the MySQL/Oracle download facility. Then you get to the welcome screen:

Before you get access to the software you have to Sign In with your Oracle/MySQL customer account if you have one. If you do not have an account yet you can Create an Account to get to the software:

Then you have to agree (2 times) to the Terms & Restrictions (this is what Oracle is really good in). Once to the Oracle Trial License Agreement and once to the Export Restrictions:

Then you get to the Media Pack Search. Here you can define what product you are interested in and on which platform you are using it. Unfortunately a sub-product filter cannot be chosen. So you get a long list to pick your final package from:

An last you can download your product of desire:

Unfortunately the packages get some silly names like V59684-01.zip instead of meaningful names. But with the following command you get some information what is included in the package:

unzip -l V59684-01.zip Archive: V59684-01.zip Length Date Time Name --------- ---------- ----- ---- 2958631 2014-11-04 13:29 meb-3.11.1-linux-glibc2.5-x86-64bit.tar.gz 185 2014-11-05 08:30 meb-3.11.1-linux-glibc2.5-x86-64bit.tar.gz.asc 77 2014-11-04 13:29 meb-3.11.1-linux-glibc2.5-x86-64bit.tar.gz.md5 2130 2014-11-05 15:06 README.txt --------- ------- 2961023 4 files

Have fun trying the MySQL Enterprise Features. If you need any help installing or integrating them into your infrastructure, do not hesitate to contact FromDual.

Taxonomy upgrade extras: BackupRestoreRecoverymysqldumppoint-in-time-recoverypitr

Sometimes we face the situation where we have a full MySQL database backup done with mysqldump and then we have to restore and recover just one single table out of our huge mysqldump file.
Further our mysqldump backup was taken hours ago so we want to recover all the changes on that table since our backup was taken up to the end.

In this blog article we cover all the steps needed to achieve this goal for MySQL and MariaDB.

Recommendation: It is recommended to do theses steps on a testing system and then dump and restore your table back to the production system. If you do it directly on your production system you have to know exactly what you are doing...
Further this process should be tested carefully and regularly to get familiar with it and to assure your backup/restore/recovery procedure works properly.

The table we want to recover is called test.test from our backup full_dump.sql.gz. As a first step we have to do the recovery with the following command to our test database:

shell> zcat full_dump.sql.gz | extract_table.py --database=test --table=test | mysql -u root

The script extract_table.py is part of the FromDual Recovery Manager to extract one single table from a mysqldump backup.

As a next step we have to extract the binary log file and its position where to start recovery from out of our dump:

shell> zcat full_dump.sql.gz | head -n 25 | grep CHANGE CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000216', MASTER_LOG_POS=1300976;

Then we have to find out where we want to stop our Point-in-Time-Recovery. The need for recover is possibly due to a TRUCATE TABLE command or similar operation executed on the wrong system or it is just a time somebody has indicated us to recover to. The position to stop we can find as follows:

shell> mysqlbinlog -v mysql-bin.000216 | grep -B5 TRUNCATE --color #150123 19:53:14 server id 35622 end_log_pos 1302950 CRC32 0x24471494 Xid = 3803 COMMIT/*!*/; # at 1302950 #150123 19:53:14 server id 35622 end_log_pos 1303036 CRC32 0xf9ac63a6 Query thread_id=54 exec_time=0 error_code=0 SET TIMESTAMP=1422039194/*!*/; TRUNCATE TABLE test

And as a last step we have to apply all the changes from the binary log to our testing database:

shell> mysqlbinlog --disable-log-bin --database=test --start-position=1300976 --stop-position=1302950 mysql-bin.000216 | mysql -u root --force

Now the table test.test is recovered to the wanted point in time and we can dump and restore it to its final location back to the production database.

shell> mysqldump --user=root --host=testing test test | mysql --user=root --host=production test

This process has been tested on MySQL 5.1.73, 5.5.38, 5.6.22 and 5.7.5 and also on MariaDB 10.0.10 and 10.1.0.

FromDual has the pleasure to announce the release of the new version 1.2.1 of the popular Backup and Recovery Manager for MySQL and MariaDB (fromdual_bman).

You can download the FromDual Backup and Recovery Manager from here.

In the inconceivable case that you find a bug in the Backup and Recovery Manager please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.2.0 to 1.2.1 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.1.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.1 fromdual_brman
Changes in FromDual Backup Manager 1.2.1 FromDual Backup Manager

It contains mainly one very critical bug fix, one bug fix for using FromDual Backup Manger in combination with MariaDB and xtrabackup and several minor bug fixes and small improvements.

We STRONGLY RECOMMEND to upgrade immediately to v1.2.1 if you are using FromDual Backup Manager v1.2.0-RC1 and newer. In these versions errors of mysqldump in combination with the --direct-compress option are not handled correctly and a return code of 0 is returned which can be wrong. This leads to incomplete backups and loss of data during the restore.

You can verify your current FromDual Backup Manager version with the following command:

fromdual_bman --version

• Merges of myEnv.inc with the one of MyEnv.
• Bug with MariaDB using xtrabackup fixed (PM).
• Lock file handling improved.
• Very important: max_allowed_packet bug fixed which was introduced in v1.2.0-RC1.
• Schema names with special characters are now handled properly.
• Log directory is automatically created if it does not exist.
• Error comment for --blocking-backup improved.

Taxonomy upgrade extras: BackupRestoreRecoverypitrfromdual_brmanreleasebrmanbmanrman

FromDual has the pleasure to announce the release of the new version 1.2.1 of the popular Backup and Recovery Manager for MySQL and MariaDB (fromdual_bman).

You can download the FromDual Backup and Recovery Manager from here.

In the inconceivable case that you find a bug in the Backup and Recovery Manager please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.2.0 to 1.2.1 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.1.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.1 fromdual_brman
Changes in FromDual Backup Manager 1.2.1 FromDual Backup Manager

It contains mainly one very critical bug fix, one bug fix for using FromDual Backup Manger in combination with MariaDB and xtrabackup and several minor bug fixes and small improvements.

We STRONGLY RECOMMEND to upgrade immediately to v1.2.1 if you are using FromDual Backup Manager v1.2.0-RC1 and newer. In these versions errors of mysqldump in combination with the --direct-compress option are not handled correctly and a return code of 0 is returned which can be wrong. This leads to incomplete backups and loss of data during the restore.

You can verify your current FromDual Backup Manager version with the following command:

fromdual_bman --version

• Merges of myEnv.inc with the one of MyEnv.
• Bug with MariaDB using xtrabackup fixed (PM).
• Lock file handling improved.
• Very important: max_allowed_packet bug fixed which was introduced in v1.2.0-RC1.
• Schema names with special characters are now handled properly.
• Log directory is automatically created if it does not exist.
• Error comment for --blocking-backup improved.

Taxonomy upgrade extras: BackupRestoreRecoverypitrfromdual_brmanreleasebrmanbmanrman

Taxonomy upgrade extras: BackupRestoreRecoveryfromdual_brmanmysql_bman

FromDual has the pleasure to announce the release of the new version 1.2.1 of the popular Backup Manager for MySQL and MariaDB (fromdual_bman).

You can download the FromDual Backup Manager from here.

In the inconceivable case that you find a bug in the Backup Manager please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.2.0 to 1.2.1 # cd ${HOME}/product # tar xf /download/fromdual_brman-1.2.1.tar.gz # rm -f fromdual_brman # ln -s fromdual_brman-1.2.1 fromdual_brman
Changes in FromDual Backup Manager 1.2.1 FromDual Backup Manager

It contains mainly one very critical bug fix, one bug fix for using FromDual Backup Manger in combination with MariaDB and xtrabackup and several minor bug fixes and small improvements.

We STRONGLY RECOMMEND to upgrade immediately to v1.2.1 if you are using FromDual Backup Manager v1.2.0-RC1 and newer. In these versions errors of mysqldump in combination with the --direct-compress option are not handled correctly and a return code of 0 is returned which can be wrong. This leads to incomplete backups and loss of data during the restore.

You can verify your current FromDual Backup Manager version with the following command:

fromdual_bman --version

• Merges of myEnv.inc with the one of MyEnv.
• Bug with MariaDB using xtrabackup fixed (PM).
• Lock file handling improved.
• Very important: max_allowed_packet bug fixed which was introduced in v1.2.0-RC1.
• Schema names with special characters are now handled properly.
• Log directory is automatically created if it does not exist.
• Error comment for --blocking-backup improved.