Monday, November 2, 2026

MariaDB Performance Claim : Technically Correct, but Misleading

A recent post on the MariaDB Foundation Blog has the impressive title How MariaDB 11.8 became about 3× faster on some in-memory workloads, fixing a regression dating back to 10.5.  The claim is technically correct when viewed in isolation, but I believe it becomes misleading when the broader context is taken into account.  The details are below.

The post mentions a benchmark on 11.8 before the fix showing 20,326 qps.  With the fix, the same benchmark shows 57,052 qps, which is 2.8× faster.  So, from this myopic perspective, 3× faster is technically correct.

But the post also mentions the same benchmark showing 52,410 qps with 10.4.34.  So, compared to the performance before the regression, 11.8 with the fix is only about 9% faster.  This is a welcome improvement, but it is far from 3× faster compared to the original baseline.

The 3× improvement is real, but it mostly comes from fixing a regression.  The original title does mention the regression, but that detail can easily be overlooked when skimming a newsletter or scrolling through social media feeds.

A less misleading title would have been How MariaDB 11.8 is now about 9% faster than 10.4 on some in-memory workloads, recovering from a regression dating back to 10.5.  It is obviously less impressive, but much closer to reality.

And speaking of newsletters, the MariaDB Foundation Newsletter of November 1, 2026 also emphasizes the impressive performance improvement, while describing the regression only in technical terms and omitting the relevant historical performance baseline.  This newsletter also links to the original post using the shortened title "How MariaDB 11.8 became about 3× faster", omitting the regression.  This makes the presentation even more misleading.

Excerpt from the MariaDB Foundation Newsletter, November 1, 2026.

Why am I calling this out ?

I consider myself part of the MySQL Ecosystem (also recently called the M-Ecosystem).  I want MySQL to be successful.  This success will come, amongst other things, from fair competition between the Vendors.  If one vendor starts making misleading presentations or exaggerated claims, or more broadly resorts to questionable tactics, others might start doing the same, and then the whole Ecosystem suffers.

I believe we can all work together to elevate the MySQL Ecosystem.  This is why I joined the OurSQL Foundation as a Board Member.  Such collaboration requires honesty in how we communicate, mutual respect between Ecosystem participants, and fair representation of all Vendors and projects, regardless of their size or influence.  This means no misleading presentations, no FUD or propaganda, no requests for preferential treatment or a bigger seat at the table, and no attempts to present oneself as more important than others.

Let us work together to promote MySQL, compete fairly to better serve our users, and call out behavior that could undermine the OurSQL Community / Ecosystem.

These are my personal views, not an official position of the OurSQL Foundation.

No comments:

Post a Comment