dbdeployer Tutorial on Mac

Not very long ago (well, maybe a little longer, this post is in draft for more than a year), in the spawn of less than 5 days, I suggested many colleagues to reproduce a problem they had with MySQL in a "more simple environment".  Such more simple environment can be created with dbdeployer.  dbdeployer is a tool to create "MySQL Sandboxes" on a Mac (laptop or desktop) or on Linux (vm, laptop or desktop).  It is relatively simple to use, but if you do not know what to do, getting things set up the first time can be challenging.  The goal of this post is to ease this first setup.  In this tutorial, I describe how to install dbdeployer on a Mac and how to create your first sandboxes.  I might do a Linux tutorial in the future.

A quick note: Giuseppe Maxia, alias datacharmer, announced The end of dbdeployer in October 2023.  So the tool will not see a new release any time soon, but it is still useful even if bugs are not fixed and incompatibility with new MySQL versions might happen.  I know of at least one thing not working well: MySQL 8.3.0 with GTIDs (works without GTIDs).  I created a channel in the MySQL Community Slack to discuss such issues: #dbdeployer-for-life (the workaround for GTIDs with 8.3.0 is in there).

Important things about dbdeployer on a Mac:

• For now, we are not able to run MariaDB Server because neither the MariaDB Foundation nor the MariaDB Corporation distribute binaries for Mac (it might be possible to hack the Homebrew MariaDB packages and create a binary package, but I am leaving this for someone else to explore and document);
• For now, we are not able to run Percona Server because Percona does not distribute binaries for Mac (it might be possible to compile Percona Server for MacOS and create a binary package, but I am leaving this for someone else to explore and document);
• We are not able to run MySQL 5.7.32+ (8.0 ok, 5.6.42+ and 5.5.54+ also non-ok) in dbdeployer on Mac because Oracle does not distribute Mac binaries for these versions (it looks like end-of-life happened earlier on Mac and that this is not a bug);
• All of the above work with dbdeployer on Linux.

For installing dbdeployer on a Mac, we can run the commands below in a terminal.  These are inspired from the instructions found in the dbdeployer wiki.

os=osx
version=1.73.0
origin=https://github.com/datacharmer/dbdeployer/releases/download/v${version}
file=dbdeployer-${version}.${os}
cd $(mktemp -d)      # Let's go in a temp directory.
wget ${origin}/${file}.tar.gz &&
  tar -xzf ${file}.tar.gz     &&
  chmod +x ${file}            &&
  sudo cp ${file} /usr/local/bin/dbdeployer
dbdeployer -v        # Final testing.

Now that we have installed and tested dbdeployer, we can download MySQL binaries and unpack them with the commands below (adapt the fs variables to the binaries you want).  I tested on a M2 with MacOS 14: both x86_64 and arm64 binaries works.  If you want other binaries, you might have to change macosxxx with macosyyy (see osx10.9, macos10.13, ..., macos10.15, macos11, ..., and macos14 in the file names below), look for exact file names on the MySQL Download Page.

mkdir -p ~/mysql_bin ~/opt/mysql
cd ~/mysql_bin

a=x86_64    # or arm64 which started with 8.0.26.
mdd=https://dev.mysql.com/get/Downloads
fs=""
fs="$fs $(echo mysql-5.5.{41,53}-osx10.9-${a}.tar.gz)"
fs="$fs $(echo mysql-5.6.{39,41}-macos10.13-${a}.tar.gz)"
fs="$fs $(echo mysql-5.7.{24,31}-macos10.14-${a}.tar.gz)"
fs="$fs $(echo mysql-8.0.{11,12}-macos10.13-${a}.tar.gz)"
fs="$fs $(echo mysql-8.0.{13,18}-macos10.14-${a}.tar.gz)"
fs="$fs $(echo mysql-8.0.{19,23}-macos10.15-${a}.tar.gz)"
fs="$fs $(echo mysql-8.0.{24,28}-macos11-${a}.tar.gz)"
fs="$fs $(echo mysql-8.0.{30,31}-macos12-${a}.tar.gz)"
fs="$fs $(echo mysql-8.0.{32,35}-macos13-${a}.tar.gz)"
fs="$fs $(echo mysql-8.0.36-macos14-${a}.tar.gz)"
fs="$fs $(echo mysql-8.{1,2}.0-macos13-${a}.tar.gz)"
fs="$fs $(echo mysql-8.3.0-macos14-${a}.tar.gz)"
for f in $fs; do wget $mdd/$f && dbdeployer unpack $f; done
ls -l ~/opt/mysql

Now that everything is set up, we can create our first standalone sandbox with the commands below.  And we are using the 5.7.31 version downloaded above, but we can easily use another version (left as an exercise to the reader).

dbdeployer deploy single 5.7.31
cd ~/sandboxes/msb_5_7_31
./use -e "CREATE DATABASE my_test"
./use my_test <<< "CREATE TABLE t(id int)"
./use my_test                  # and then do whatever you want in MySQL.
# Play with the sandbox as much as you want.
./stop                         # Stop MySQL.
rm -rf ~/sandboxes/msb_5_7_31  # Optional cleanup.

Now that we have played with a standalone sandbox, we can create our first replication sandbox with the commands below.  And we are NOT using the latest 8.0 version downloaded above (maybe this is what we are running in production), but we can easily use another version (left as an exercise to the reader).

dbdeployer deploy replication 8.0.30
cd ~/sandboxes/rsandbox_8_0_30
./m -e "CREATE DATABASE my_test"    # m for master / primary.
echo "CREATE TABLE t(id int)" | ./m my_test 
./s2 <<< "STOP REPLICA"             # s2 for slave #2 / replica #2.
./m  my_test -e "CREATE TABLE t2(id int)"
./s1 my_test -e "SHOW TABLES"
./s2 my_test -e "SHOW TABLES"
# Play with m, s1 and s2 as much as you want.
./stop_all                          # Stop all nodes.
rm -rf ~/sandboxes/rsandbox_8_0_30  # Optional cleanup.

To save disk space, we can delete unused binaries from ~/opt/mysql (they can be unpacked again when needed from ~/mysql_bin).

If we need help, we can run below or look in the dbdeployer wiki.

dbdeployer --help
dbdeployer deploy --help             | less
dbdeployer deploy single --help      | less
dbdeployer deploy replication --help | less

Enjoy dbdeployer on Mac !