This manual section 17.3.6. Switching Masters During Failover got me confused.
In the 2nd para below th 1st diagram it describes switching the web clients from the failed master to one of the slaves. It says:
Very clever! The new master (former slave 1) was configured all along with log-bin but not log-slave-updates. So its bin log accumulates every update after the web clients switch.
The next but one para gives instructions for the process:
What I don't understand is the last 3 words "and RESET MASTER." This deletes the binary log on the new master, i.e. on the server that became the new master with the switch of the web clients. I understand this simplifies reconfiguring the other slaves because RESET MASTER also resets we can use the default bin log coordinates (1st file, offset 4), but all its updates since we switched the web clients are lost!
It seems to me that if we're going to use RESET MASTER on the new master than we need to do that before we promote it, i.e. before we switch the web clients.
Alternatively we could dispense with the RESET MASTER and instead use SHOW MASTER STATUS at any time (perhaps periodically) before the web client switch, record the file and offset, and use those coordinates when we reconfigure the other slaves.
Back to the manual... The next 2 paras seems to imply that we should switch the web clients only after we've done all this reconfiguration of the replication system. OK. So perhaps that other para, the 1st of my quotes, is merely descriptive, not imperative, and the imperative instructions of the failover process are the three paras beginning with the one that is my 2nd quote. So the manual is implying, from the order of these instructions, that the web client switch must happen after the RESET MASTER.
That would explain my concern about deleting vital bin log data with the RESET MASTER.
But it seems like a daft procedure, one that wastes a brilliant feature of MySQL. The trick of configuring all servers with log-bin but not log-slave-updates is very clever. It means that failover to a new master can be done by processes entirely external to all of the MySQL servers. All that needs to happen is: choose the new master and switch the clients. Immediately the new master is doing the right thing. We can then proceed to carefully reconfigure however we like.
The procedure in the manual, on the other hand, inserts a whole bunch of server reconfiguration work between the detection of the former master server's failure and the resumption of service. That takes time, of course, but much worse: it has a significant probability of failure. The scripts would be distributed and be difficult to test.
In the 2nd para below th 1st diagram it describes switching the web clients from the failed master to one of the slaves. It says:
Quote:Each MySQL Slave (Slave 1, Slave 2, and Slave 3) is a slave running with --log-bin and without --log-slave-updates. Because updates received by a slave from the master are not logged in the binary log unless --log-slave-updates is specified, the binary log on each slave is empty initially. If for some reason MySQL Master becomes unavailable, you can pick one of the slaves to become the new master. For example, if you pick Slave 1, all Web Clients should be redirected to Slave 1, which will log updates to its binary log. Slave 2 and Slave 3 should then replicate from Slave 1.
Very clever! The new master (former slave 1) was configured all along with log-bin but not log-slave-updates. So its bin log accumulates every update after the web clients switch.
The next but one para gives instructions for the process:
Quote:Make sure that all slaves have processed any statements in their relay log. On each slave, issue STOP SLAVE IO_THREAD, then check the output of SHOW PROCESSLIST until you see Has read all relay log. When this is true for all slaves, they can be reconfigured to the new setup. On the slave Slave 1 being promoted to become the master, issue STOP SLAVE and RESET MASTER.
What I don't understand is the last 3 words "and RESET MASTER." This deletes the binary log on the new master, i.e. on the server that became the new master with the switch of the web clients. I understand this simplifies reconfiguring the other slaves because RESET MASTER also resets we can use the default bin log coordinates (1st file, offset 4), but all its updates since we switched the web clients are lost!
It seems to me that if we're going to use RESET MASTER on the new master than we need to do that before we promote it, i.e. before we switch the web clients.
Alternatively we could dispense with the RESET MASTER and instead use SHOW MASTER STATUS at any time (perhaps periodically) before the web client switch, record the file and offset, and use those coordinates when we reconfigure the other slaves.
Back to the manual... The next 2 paras seems to imply that we should switch the web clients only after we've done all this reconfiguration of the replication system. OK. So perhaps that other para, the 1st of my quotes, is merely descriptive, not imperative, and the imperative instructions of the failover process are the three paras beginning with the one that is my 2nd quote. So the manual is implying, from the order of these instructions, that the web client switch must happen after the RESET MASTER.
That would explain my concern about deleting vital bin log data with the RESET MASTER.
But it seems like a daft procedure, one that wastes a brilliant feature of MySQL. The trick of configuring all servers with log-bin but not log-slave-updates is very clever. It means that failover to a new master can be done by processes entirely external to all of the MySQL servers. All that needs to happen is: choose the new master and switch the clients. Immediately the new master is doing the right thing. We can then proceed to carefully reconfigure however we like.
The procedure in the manual, on the other hand, inserts a whole bunch of server reconfiguration work between the detection of the former master server's failure and the resumption of service. That takes time, of course, but much worse: it has a significant probability of failure. The scripts would be distributed and be difficult to test.