#1002738 redis-server: Default systemd unit file system protection settings prevent writing of logfiles, crashing redis

Package:
redis-server
Source:
redis
Description:
Persistent key-value database with network interface
Submitter:
Johannes Bülow
Date:
2022-01-28 08:15:03 UTC
Severity:
normal
#1002738#5
Date:
2021-12-28 16:02:17 UTC
From:
To:
Dear Maintainer,

I installed openvas on my kali vm, but redis broke, so I tried to
install redis-server on a clean debian instance, with the same result.

If I disable System Protection and comment out all Read/Write stuff in
the systemd service unit file, redis
boots up just fine. I therefore suspect the issue being with the write
restrictions in the default Systemd unit file for redis-server.

The message in my syslog:

Dec 28 15:19:05 debian redis-server[521]: *** FATAL CONFIG FILE ERROR
(Redis 6.0.15) ***
Dec 28 15:19:05 debian redis-server[521]: Reading the configuration
file, at line 260
Dec 28 15:19:05 debian redis-server[521]: >>> 'logfile
/var/log/redis/redis-server.log'
Dec 28 15:19:05 debian redis-server[521]: Can't open the log file:
Read-only file system
Dec 28 15:19:05 debian systemd[1]: redis-server.service: Main process
exited, code=exited, status=1/FAILURE

#1002738#10
Date:
2021-12-28 17:13:03 UTC
From:
To:
Hi Johannes,

It sounds like that to me too. However:

That's interesting, because there is a specific allowance made so that
Redis can write to its own logfiles. Specifically:

  ReadWriteDirectories=-/var/log/redis

Ah, perhaps your version of systemd is newer? Can you reset or
otherwise reinstate the "ReadWriteDirectories" lines along with the
rest of the file and replace the start with "ReadWritePaths=" and let
me know how you get on?


Regards,

#1002738#15
Date:
2021-12-29 07:55:45 UTC
From:
To:
On my systems they are already ReadWritePaths, not ReadWriteDirectories
(except for /etc/redis and / )
I am running systemd 247.3-6 on the affected systems, but Kernel
5.15.8-1-default. On Kernel 5.14 and older it seems to work fine.
That sadly didn't help much, but setting ProtectSystem to "strict",
following the systemd.exec manpage, and removing
"ReadOnlyDirectories=/" solved the problem for me.
My only guess is that it's some issue with (Kernel) namespaces either
on my System specifically or with Kernel 5.15 in general.

       ProtectSystem=
           Takes a boolean argument or the special values "full" or "strict". If true, mounts the /usr/ and the boot loader directories (/boot and /efi) read-only for processes invoked by this unit. If set to "full", the /etc/
           directory is mounted read-only, too. If set to "strict" the entire file system hierarchy is mounted read-only, except for the API file system subtrees /dev/, /proc/ and /sys/ (protect these directories using
           PrivateDevices=, ProtectKernelTunables=, ProtectControlGroups=). This setting ensures that any modification of the vendor-supplied operating system (and optionally its configuration, and local mounts) is prohibited for
           the service. It is recommended to enable this setting for all long-running services, unless they are involved with system updates or need to modify the operating system in other ways. If this option is used,
           ReadWritePaths= may be used to exclude specific directories from being made read-only. This setting is implied if DynamicUser= is set. This setting cannot ensure protection in all cases. In general it has the same
           limitations as ReadOnlyPaths=, see below. Defaults to off.




Regards,

Johannes Bülow

#1002738#20
Date:
2021-12-29 17:19:02 UTC
From:
To:
Hi,
[..]

Hm, which makes me think the bug is elsewhere and not in the Redis
systemd unit. Unless I'm missing something, the configuration
directives used here should 'just work'. Or rather: you are likely to
encounter (or are already encountering!) other breakages on your system
if this is not working.

Thoughts?

Chris

ps. I assume you are using bullseye? (redis 6.0.15-1 is the version
    available there)

#1002738#25
Date:
2022-01-02 09:33:25 UTC
From:
To:
Hello,

These Problems could theoretically come from other settings, yes.

I did use Bullseye for testing this, I don't have any other Debian
systems currently (if you want, I can test it in bookworm or buster
through).

If you want, you can close this Bug Report if you feel this is a weird
edge-case, I can still reopen it if investigation downstream comes to a
different conclusion.

My reasoning for opening a Bug Report for this was because I had this
issue on multiple Debian LXC containers and VMs, but on my SuSE and
Rocky systems I didn't encounter this issue, and they do have different
unit files. When compairing the files for redis in Debian and SuSE, I
found those differences.

TLDR, if you want, feel free to close this ticket, I'll reopen it if
something changes downstream.

Greetings,

Johannes

#1002738#30
Date:
2022-01-02 17:17:03 UTC
From:
To:
Hi Johannes,

Thanks for your mail. I'm happy to keep this bug open in case
something comes up, but I'm not sure what I would do if we could
definitively demonstrate a bug in Redis' unit file.

That is to say, the only solution (whatever the cause might turn out
to be) would seem to be to remove all of the hardening features from
the unit file... hardly the right solution here. :)


Regards,

#1002738#35
Date:
2022-01-28 08:11:44 UTC
From:
To:
Hello Chris,

Downstream didn't really do anything so far on this topic.
I also can only reproduce this issue on Kernel 5.15 and higher.

I don't know whether setting / as ReadOnly directly or using
ProtectSystem=strict along with PrivateDevices=true,
ProtectKernelTunables=true and ProtectControlGroups=true is the more
secure configuration.

as for reproducing, I can reproduce it pretty consistently on LXC
containers on new kernels, but not on 5.10 or 5.12.

This is probably going to lead to a dead end

Best Regards,

Johannes