HOME TAGS ARCHIVE ABOUT

Mysql Replication in Swarm Mode

Swarm is like a Charm when it comes to simplicity !!

I’ve been trying for 4 days to setup a Mysql cluster in Kubernetes, the usual master slaves architecture. But till now it’s been all failures, one after another. So I decided to take a break, and try instead swarm, and believe it or not, within 15 minutes, ALL is setup & working.

Here is the source here

Let’s start with the Mysql 5.7 Dockerfile :

FROM mysql:5.7

# default replication user and pass
ENV REPLICATION_USER replication
ENV REPLICATION_PASSWORD replication_pass

COPY core/replication-entrypoint.sh /usr/local/bin/
COPY core/init-slave.sh /

RUN chmod +x /usr/local/bin/replication-entrypoint.sh
ENTRYPOINT ["/usr/local/bin/replication-entrypoint.sh"]
CMD ["mysqld"]

For the replication part, we just use need to configure mysql and let it handle the rest :

the replication entrypoint script :

#!/bin/bash
set -eo pipefail

cat > /etc/mysql/mysql.conf.d/repl.cnf << EOF
[mysqld]
log-bin=mysql-bin
relay-log=mysql-relay
#bind-address=0.0.0.0
#skip-name-resolve
innodb_file_per_table = 1
innodb_flush_method = O_DIRECT
innodb-flush-log-at-trx-commit = 0
transaction-isolation = READ-COMMITTED
max_allowed_packet = 128M
EOF

# If there is a linked master use linked container information
if [ -n "$MASTER_PORT_3306_TCP_ADDR" ]; then
  export MASTER_HOST=$MASTER_PORT_3306_TCP_ADDR
  export MASTER_PORT=$MASTER_PORT_3306_TCP_PORT
fi

if [ -z "$MASTER_HOST" ]; then
  export SERVER_ID=1
  cat >/docker-entrypoint-initdb.d/init-master.sh  <<'EOF'
#!/bin/bash

echo Creating replication user ...
mysql -uroot -p$MYSQL_ROOT_PASSWORD -e "\
  GRANT \
    FILE, \
    SELECT, \
    SHOW VIEW, \
    LOCK TABLES, \
    RELOAD, \
    REPLICATION SLAVE, \
    REPLICATION CLIENT \
  ON *.* \
  TO '$REPLICATION_USER'@'%' \
  IDENTIFIED BY '$REPLICATION_PASSWORD'; \
  FLUSH PRIVILEGES; \
"
EOF
else
  # TODO: make server-id discoverable
  export SERVER_ID=2
  cp -v /init-slave.sh /docker-entrypoint-initdb.d/
  cat > /etc/mysql/mysql.conf.d/repl-slave.cnf << EOF
[mysqld]
log-slave-updates
master-info-repository=TABLE
relay-log-info-repository=TABLE
relay-log-recovery=1
EOF
fi

cat > /etc/mysql/mysql.conf.d/server-id.cnf << EOF
[mysqld]
server-id=$SERVER_ID
EOF

exec docker-entrypoint.sh "$@"

The init slave script :

#!/bin/bash
# TODO: cover slave side selection for replication entities:
# * replicate-do-db=db_name only if we want to store and replicate certain DBs
# * replicate-ignore-db=db_name used when we don't want to replicate certain DBs
# * replicate_wild_do_table used to replicate tables based on wildcard patterns
# * replicate_wild_ignore_table used to ignore tables in replication based on wildcard patterns

REPLICATION_HEALTH_GRACE_PERIOD=${REPLICATION_HEALTH_GRACE_PERIOD:-3}
REPLICATION_HEALTH_TIMEOUT=${REPLICATION_HEALTH_TIMEOUT:-10}

check_slave_health () {
  echo Checking replication health:
  status=$(mysql -uroot -p$MYSQL_ROOT_PASSWORD -e "SHOW SLAVE STATUS\G")
  echo "$status" | egrep 'Slave_(IO|SQL)_Running:|Seconds_Behind_Master:|Last_.*_Error:' | grep -v "Error: $"
  if ! echo "$status" | grep -qs "Slave_IO_Running: Yes"    ||
     ! echo "$status" | grep -qs "Slave_SQL_Running: Yes"   ||
     ! echo "$status" | grep -qs "Seconds_Behind_Master: 0" ; then
	echo WARNING: Replication is not healthy.
    return 1
  fi
  return 0
}


echo Updating master connetion info in slave.

mysql -uroot -p$MYSQL_ROOT_PASSWORD -e "RESET MASTER; \
  CHANGE MASTER TO \
  MASTER_HOST='$MASTER_HOST', \
  MASTER_PORT=$MASTER_PORT, \
  MASTER_USER='$REPLICATION_USER', \
  MASTER_PASSWORD='$REPLICATION_PASSWORD';"

mysqldump \
  --protocol=tcp \
  --user=$REPLICATION_USER \
  --password=$REPLICATION_PASSWORD \
  --host=$MASTER_HOST \
  --port=$MASTER_PORT \
  --hex-blob \
  --all-databases \
  --add-drop-database \
  --master-data \
  --flush-logs \
  --flush-privileges \
  | mysql -uroot -p$MYSQL_ROOT_PASSWORD

echo mysqldump completed.

echo Starting slave ...
mysql -uroot -p$MYSQL_ROOT_PASSWORD -e "START SLAVE;"

echo Initial health check:
check_slave_health

echo Waiting for health grace period and slave to be still healthy:
sleep $REPLICATION_HEALTH_GRACE_PERIOD

counter=0
while ! check_slave_health; do
  if (( counter >= $REPLICATION_HEALTH_TIMEOUT )); then
    echo ERROR: Replication not healthy, health timeout reached, failing.
	break
    exit 1
  fi
  let counter=counter+1
  sleep 1
done

And the swarm stack

---
version: "3.3"

services:
  master:
    hostname: 'master'
    image: actency/docker-mysql-replication:5.7
    environment:
    - MYSQL_ROOT_PASSWORD=password
    - MYSQL_USER=testuser
    - MYSQL_PASSWORD=password
    - MYSQL_DATABASE=testdb
    - REPLICATION_USER=repl_user
    - REPLICATION_PASSWORD=repl_pass
    volumes:
    - mysql_master:/var/lib/mysql
    ports:
    - "3306"
    networks:
    - mysqlnet
    deploy:
      placement:
        constraints:
        - node.role == manager
  replica:
    image: actency/docker-mysql-replication:5.7
    environment:
    - MYSQL_ROOT_PASSWORD=password
    - MYSQL_USER=testuser
    - MYSQL_PASSWORD=password
    - MYSQL_DATABASE=testdb
    - REPLICATION_USER=repl_user
    - REPLICATION_PASSWORD=repl_pass
    volumes:
    - mysql_replica:/var/lib/mysql
    ports:
    - "3306"
    networks:
    - mysqlnet
    deploy:
      placement:
        constraints:
        - node.role == worker
networks:
  mysqlnet:

volumes:
  mysql_master:
  mysql_replica:

Now you can add more replicas, and test by adding data to the database, and check if all nodes are synchronized.