Dear Maintainer,

This is related to #1141976: on this host, mariadb.socket was enabled
by the 1:10.11.18-0+deb12u1 upgrade (2026-10-08), not by me. However,
the problem below is independent of how the socket got enabled: the
maintainer scripts stop mariadb.service while leaving mariadb.socket
active, so any client can re-activate the server mid-upgrade. This
affects anyone who enables socket activation deliberately, too.

On Debian 12 (bookworm), upgrading mariadb-server from
1:10.11.18-0+deb12u1 to 1:10.11.19-0+deb12u1 (from bookworm-security)
left the database unavailable and ultimately led to a system-wide outage.
Since security updates are commonly applied by unattended-upgrades, this
can happen without an administrator present.

mariadb.socket listens on /run/mysqld/mysqld.sock, @mariadb and
0.0.0.0:3306. The host runs FreePBX/Asterisk with per-minute cron jobs
that connect to the database, and is a replication peer of another
server.

What happens:

1. The maintainer script stops mariadb.service; dpkg warns:
     Warning: Stopping mariadb.service, but it can still be activated by:
       mariadb.socket
2. The old 10.11.18 server shuts down cleanly in about one second.
3. In the same second, a client connection on mariadb.socket triggers
   socket activation, and systemd starts mariadb.service again, still
   with the old 10.11.18 binary, while the upgrade is in progress.
4. This new instance stalls during startup (after buffer pool load,
   before "ready for connections") and never becomes ready.
5. The maintainer script stops the service again. The instance ignores
   SIGTERM; because the unit has SendSIGKILL=no, systemd waits the full
   stop-sigterm and final-sigterm timeouts (2 x 15 min) and then leaves
   the process running.
6. All later start attempts fail ("Found left-over process ... Will not
   start SendSIGKILL=no service ... Device or resource busy"), and the
   postinst restart fails ("Could not execute systemctl: at
   /usr/bin/deb-systemd-invoke line 145").
7. Meanwhile mariadb.socket keeps accepting client connections that are
   never served, so clients hang indefinitely instead of failing. In my
   case cron jobs piled up overnight (~1700 cron, ~1400 php, ~250 mysql
   client processes) until the OOM killer fired and the machine became
   unreachable; only a power cycle recovered it.

Relevant journal (trimmed, times in WEST):

22:02:59 mariadbd[3904780]: /usr/sbin/mariadbd (initiated by: unknown): Normal shutdown
22:02:59 systemd[1]: Stopping mariadb.service - MariaDB 10.11.18 database server...
22:03:00 mariadbd[3904780]: InnoDB: Shutdown completed; log sequence number 83086382283
22:03:00 mariadbd[3904780]: /usr/sbin/mariadbd: Shutdown complete
22:03:00 systemd[1]: Stopped mariadb.service - MariaDB 10.11.18 database server.
22:03:00 systemd[1]: Starting mariadb.service - MariaDB 10.11.18 database server...
22:03:00 mariadbd[869784]: Starting MariaDB 10.11.18-MariaDB-0+deb12u1-log ... as process 869784
22:03:01 mariadbd[869784]: Using systemd activated socket host 0.0.0.0 port 3306
22:03:01 mariadbd[869784]: Using systemd activated unix socket /run/mysqld/mysqld.sock
22:03:01 mariadbd[869784]: Using systemd activated unix socket @mariadb
22:03:01 mariadbd[869784]: InnoDB: Buffer pool(s) load completed at 261015 22:03:01
   (no further output from 869784)
22:18:00 systemd[1]: mariadb.service: State 'stop-sigterm' timed out. Skipping SIGKILL.
22:33:00 systemd[1]: mariadb.service: State 'final-sigterm' timed out. Skipping SIGKILL. Entering failed mode.
22:33:00 systemd[1]: mariadb.service: Found left-over process 869784 (mariadbd) in control group while starting unit. Ignoring.
22:33:00 systemd[1]: mariadb.service: Will not start SendSIGKILL=no service of type KillMode=control-group or mixed while processes exist
22:33:00 systemd[1]: mariadb.service: Failed to run 'start-pre' task: Device or resource busy

I could not determine why the socket-activated instance stalled during
startup; no stack trace was captured before the power cycle. However,
the instance should never have been started in the first place.

I have only tested this on bookworm (10.11.x); I have not verified
whether the 11.8 packaging in trixie/sid still stops mariadb.service
without stopping mariadb.socket.

Expected behaviour:

The maintainer scripts should stop (or otherwise hold) mariadb.socket
together with mariadb.service before unpacking, and start both again
in postinst, e.g.

  deb-systemd-invoke stop mariadb.socket mariadb.service

so that no client can re-activate the server while the package is being
replaced.

Workaround:

Stop both units manually before upgrading:

  systemctl stop mariadb.socket mariadb.service
  apt upgrade
  systemctl start mariadb.socket mariadb.service

Regards,
José Gonçalves

Hi!

Thanks for reporting
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1151058 and
following up on
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1141976.

The regression was in 1:10.11.18+deb12u1 that included upstream commit
https://github.com/MariaDB/server/commit/9fa3f51f59f5b7a6d84c68a45e6a6579f55847b3
that lead to Debhelper (version 10) injecting extra lines as listed in
https://salsa.debian.org/mariadb-team/mariadb-server/-/merge_requests/171#note_817952.

Initially 1:10.11.18-0+deb12u1 was only in bookworm-proposed-updates
and wasn't let into Bookworm itself due to the regression. However,
once Bookworm went into LTS, all pending proposed updates were
released, so unfortunately all users who upgraded .17->.18 potentially
hit this.

The fix in .19 does not have this regression anymore. Upgrades from
.17 to .19 skip this issue.

The fix does however not include any workaround to shut down
potentially existing socket activated instances. If you want to spend
time developing such a fix, you are welcome to do so and submit your
contribution in the open source spirit at
https://salsa.debian.org/mariadb-team/mariadb-server. Alternatively,
if you have a support contract with some provider, you can request
them to engineer the fix for you (and potentially also submit it to
Debian).

For anyone who ran into this issue and wanted to manually fix it,
simply run `pgrep -ef mariadbd` and/or `pgrep -ef mysqld` and look at
your process listings to see that all instances have shut down. After
that you can restart your system or just restart the systemd service
to let systemd manage starting the service.

Hi Otto,

Thanks for the reply and for explaining the .18 regression.

I'd like to clarify one point, because I think it is separate from
#1141976. The incident happened during the .18 -> .19 upgrade. The
socket being enabled by .18 is what made it possible, but the failure
itself comes from the maintainer scripts stopping mariadb.service while
mariadb.socket stays active. dpkg warned about exactly that:

   Warning: Stopping mariadb.service, but it can still be activated by:
     mariadb.socket

As I understand it, this leaves two open issues:

1. Hosts that went through .18 still have mariadb.socket enabled after
upgrading to .19, unless the admin noticed and disabled it. On those
hosts, the next security update can hit the same race. Since these
updates are usually applied by unattended-upgrades, that may happen with
nobody watching.

2. The same race applies to anyone who enables socket activation
deliberately, regardless of .18.

SendSIGKILL=no makes the outcome much worse. The stalled instance cannot
be stopped, systemd refuses to start the service while it exists, and
the socket keeps accepting connections that are never served. Clients
hang instead of failing.

Could you confirm whether the .19 maintainer scripts stop mariadb.socket
during upgrades? If they don't, would you accept a change that stops
mariadb.socket together with mariadb.service before unpacking and starts
both again in postinst (only if the socket was active before)?

Regards,
José Gonçalves

I can confirm experiencing the same issue on a Debian 12 production server
during a MariaDB upgrade today (October 21, 2026).

The package upgrade became stuck in dpkg, and I had to manually kill the
MariaDB process to allow it to proceed. The APT transaction subsequently
failed, requiring an additional repair run to restore the package
management state.

As in the original report, the reason MariaDB was started again during the
upgrade was that several scripts running from cron.d attempted to connect
to the database, triggering systemd socket activation. These scripts also
became stuck, and I had to manually kill all of them.

After the failed upgrade, I discovered that mariadb.socket had been
enabled, causing systemd to listen on all interfaces on TCP port 3306,
despite MariaDB being configured with bind-address=127.0.0.1. Fortunately,
existing firewall rules prevented external access.

I have since disabled mariadb.socket and restored normal service operation.

This appears to confirm that the socket activation regression can cause
both unexpected network exposure and serious package upgrade failures on
existing Debian 12 installations.

Unfortunately, I did not preserve diagnostic logs, as restoring service as
quickly as possible was the priority on this production server.