• Jan
  • 10

Step-by-step: how to setup MySQL Database replication

This tutorial will go through the setup of MySQL database replication. I will also talk about how to get everything working smoothly again after a server crash, or if you wish to switch databases. I will try to explain what is going on behind the scenes for every step (something I've found missing from other tutorials). This is written specifically for MySQL 5.0 on Centos 4, but should be very similar on other Linux distributions. It should also work this way with MySQL 4.x.

The theory

We have 2 servers, one of which is a Master and the other which is a Slave. We tell the Master that it should keep a log of every action performed on it. We tell the slave server that it should look at this log on the Master and whenever something new happens, it should do the same thing.

You should follow the instructions below with two console windows open - one for the Master and one for the Slave. Also note that I will capitalise the first letters of Master and Slave to indicate I am talking about the servers.

Configuring the Master

First of all, we need to create a user on the Master server that the Slave will connect as. I call mine 'slave_user'. Log into mysql as root and create the user:

mysql -u root -p (log into MySQL)

CODE:
GRANT REPLICATION SLAVE ON *.* TO 'slave_user'@'%' IDENTIFIED BY 'your_password';
FLUSH PRIVILEGES;

Now, we should edit the my.cnf file (usually in /etc/my.cnf), in the [mysqld] section and tell MySQL that it's going to be a Master:

CODE:
log-bin = /home/mysql/logs/mysql-bin.log
binlog-do-db=my_database
server-id=1

The first line tells MySQL to start writing a log, and tells it where to write the log. Make sure this directory is empty of all replication logs, especially if you're starting again after replication has already been used.

The second line chooses the database to write the log for. You should change this to your database. The third line gives the server an ID (to distinguish it from the Slave).

You should also make sure skip-networking has not been enabled.

You should now restart the Master:

CODE:
/etc/rc.d/init.d/mysqld restart

(MySQL restart commands may vary)

Configuring the Slave

Again, we should change the /etc/my.cnf of the Slave server, in the [mysqld] section:

CODE:
server-id=2
master-host=128.0.0.1
master-connect-retry=60
master-user=slave_user
master-password=slave_password
replicate-do-db=my_database

relay-log = /var/lib/mysql/slave-relay.log
relay-log-index = /var/lib/mysql/slave-relay-log.index

Line 1 gives the Slave its unique ID. Line 2, tells the Slave the I.P address of the Master server - so you need to change the I.P here.

The remaining lines set a retry limit, and tell the Slave the user, password and database it needs to replicate. We also tell the slave what to use as its relay log. It's best to set this directly, or MySQL will create the name from the hostname and should you change hostname, replication will fail.

You should also make sure skip-networking has not been enabled.

You should now restart the Slave:

CODE:
/etc/rc.d/init.d/mysqld restart

Getting the data onto the Slave

On the Master...

I'm assuming you have a live Master server, and an as yet empty Slave server. This stage depends on whether data is constantly being added to the Master. If so, we will have to prevent all database access on the Master so nothing can be added. This means your server will hang during the next step. If no data is being added to the server, you can skip this step. On the Master server, log into MySQL and do the following:

mysql -u root -p (log into MySQL)

CODE:
FLUSH TABLES WITH READ LOCK;

Now we will use mysqldump to get the data out. So, still on the Master server:

CODE:
mysqldump my_database -u root -p > /home/my_home_dir/database.sql;
gzip /home/my_home_dir/database.sql;

Make sure you change my_database to your database name, and my_home_dir to the name of your home directory (or another directory of your choosing). You wll now have a file called database.sql.gz in your home directory. This is a gziped copy of your database.

On the Slave...

Now we need to copy over the gzipped file. On the Slave run the following:

CODE:
scp root@128.0.0.1:/home/my_home_dir/database.sql.gz /home/my_home_dir/

Make sure 128.0.0.1 is the I.P of the Master. This will copy the file from the Master and put it in your home directory on the Slave. Now we just need to import into MySQL:

mysql -u root -p (log into MySQL)

CODE:
CREATE DATABASE `my_database`;

CODE:
gunzip /home/my_home_dir/database.sql.gz
mysql -u root -p my_database  </home/my_home_dir/database.sql

Ready to rumble...

On the Master...

Now we're ready to kick things off. We need to find the position the Master is at in the logs. So, log into MySQL and run the following:

mysql -u root -p (log into MySQL)

CODE:
SHOW MASTER STATUS;

This should give you an output along these lines:

CODE:
+---------------------+----------+-------------------------------+------------------+
| File                | Position | Binlog_Do_DB                  | Binlog_Ignore_DB |
+---------------------+----------+-------------------------------+------------------+
| mysql-bin.000001    | 21197930 | my_database,my_database       |                  |
+---------------------+----------+-------------------------------+------------------+

Keep that on-screen.

On the Slave...

Log into MySQL and do the following:

mysql -u root -p (log into MySQL)

CODE:
slave stop;
CHANGE MASTER TO MASTER_HOST='128.0.0.1', MASTER_USER='slave_user', MASTER_PASSWORD='slave_password', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=21197930;
slave start;

First we stop the Slave. Then we tell it exactly where to look in the Master log file. We use the values for our previous SHOW MASTER STATUS; command on the Master. You should change 128.0.0.1 to the I.P of the Master, and change the user and password accordingly.

The Slave will now be waiting. So all that's left is to...

Back on the Master...

We shoud already be logged into MySQL, so all you have to do is:

CODE:
unlock tables;

To release the tables from lock. Note you only have to do this if you previously ran FLUSH TABLES WITH READ LOCK;

And the recovery part?

Several times it's happened to me that a server has crashed, or a hostname changed or whatever, and I've had a real trouble getting replication to work again. The solution has been to clear out the logs.

On the Slave

Clear out any replication logs from /var/lib/mysql or whever the logs are being stored, as stated in my.cnf. This usually does the trick:

CODE:
rm *relay*
rm master.info

On the Master

Again, get rid of the logs, as per where they are stored in my.cnf. For me it's the following:

CODE:
cd /home/mysql/logs/
rm -f *

This should give you a fresh start on things. You can now start again from the beginning...

Final Notes

My database doesn't use InnoDB tables - it's all MyISAM. However, the MySQL manual recommends adding this to my.cnf for InnoDB databases:

CODE:
innodb_flush_log_at_trx_commit=1
sync_binlog=1

See here for more info: http://dev.mysql.com/doc/refman/5.1/en/replication-howto-masterbaseconfig.html

References

Thanks to the following for their help:
http://dev.mysql.com/doc/refman/5.1/en/replication-howto.html
http://www.howtoforge.com/mysql_database_replication
http://www.gra2.com/article.php/setting-up-database-replication-on-mysql


Related posts

Posted by admin in mysql, tutorials
trackback
 

  • Jan
  • 11
Aaron
Aaron

Excellent!
Thanks.

  • Jan
  • 22
louis w
louis w

great tut.

  • Jan
  • 23
Marcus Bointon
Marcus Bointon

You can improve this process quite a lot, especially if you're using InnoDB tables. Pass these extra params to mysqldump:

CODE:
mysqldump -u root -p --opt --master-data --single-transaction my_database>somefile.sql

This will give you a clean snapshot without a read lock even while the server is running (InnoDB only); it includes the vital master info so you don't need to bother setting it on the slave because the import will include it; the exported data file will be smaller and faster to import (important if it's several gigs).

Those settings for InnoDB are incredibly conservative and will cause you major disk i/o overhead, try these instead:

innodb_flush_method=O_DIRECT
innodb_file_per_table
innodb-flush-log-at-trx-commit = 2

Don't set sync-binlog at all. innodb_file_per_table means you will recover disk space more efficiently when data is deleted.

  • Jan
  • 23
admin
Leon

Thanks Marcus. As I mentioned I've never used this for InnoDB tables, but looks like some useful tips there, --single-transaction especially.

  • Jan
  • 24
Antoine
Antoine

The only tutorial on the internet that actually works !

  • Feb
  • 7
jviola
jviola

binlog-do-db=my_database

If I want to replicate all my databases on my Master should I use an "*" after the "=" sign? Or do I list them separated by commas?

  • Feb
  • 7
jviola
jviola

innodb_flush_method=O_DIRECT
innodb_file_per_table
innodb-flush-log-at-trx-commit = 2

Does this go in the my.cnf under [mysqld] of the Master or the slave?

  • Feb
  • 7
admin
Leon

jviola - you should use multiple lines like so:
binlog-do-db=my_database_1
binlog-do-db=my_database_2

and add the innodb info to the Master.

  • Feb
  • 8
Jeff Viola
Jeff Viola

Thanks,

Should I also have multiple entries in the Master with
binlog-do-db=database_1
binlog-do-db=database_2

  • Feb
  • 12
admin
Leon

Those lines (binlog-do-db=database_1 etc) should only be in the Master.

  • Mar
  • 4
fab
fab

And what happen if my slave server is down, when it is up again, will it get automatically missed data ????

Thanks

  • Mar
  • 4
admin
Leon

fab - yes, it should automatically retrieve all the missing data.

  • Mar
  • 5
fab
fab

Instead of "skip-networking"
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 127.0.0.1

On my debian i have this ;)
I guess i have to comment also bind-adresse putting a "#"

at beginning of "Configuring the slave" , u've written : /etc/my.cnf/ instead of /etc/mysql/my.cnf

  • Mar
  • 5
fab
fab

Sorry for double comments

Your post is great :D Thanks

  • Mar
  • 20
fab
fab

How can i manage if i want my slave server to replicate many databases ??

server-id=2
master-host=128.0.0.1
master-connect-retry=60
master-user=slave_user
master-password=slave_password
replicate-do-db=my_database

relay-log = /var/lib/mysql/slave-relay.log
relay-log-index = /var/lib/mysql/slave-relay-log.index

I would like to set several masterhosts, each one linked to a various database in my slave server

Do i have to add something to my.cnf to do this ??

  • Mar
  • 23
admin
Leon

fab - I'm afraid I've never had to do that so I don't know for sure. Please let us know if you find out!



* Required.
Comments need to be approved and may take a while to appear.
  • © 2008 Aciddrop.com
  • Top

Categories

Archive

Links

Feeds