#976190 systemd: LXC memory limits messed up after some time

Package:
systemd
Source:
systemd
Description:
system and service manager
Submitter:
Paul Slootman
Date:
2022-07-05 18:33:03 UTC
Severity:
normal
Tags:
#976190#5
Date:
2020-12-01 08:27:46 UTC
From:
To:
I have libvirt configured to run LXC by default.
I also have memory limits in place. Directly after starting the LXC
containers, they are correct; e.g.:

root@dns:~# free
              total        used        free      shared  buff/cache   available
Mem:        1048576       41660     1006784       48124         132     1006784
Swap:             0           0           0


After some time, I have noticed that they are no longer correct; in fact
they are wildly wrong:

root@dns:~# free
              total        used        free      shared  buff/cache   available
Mem:    9007199254740991      598372 9007199254108535       85652       34084 9007199254108535
Swap:             0        1692       -1692


I noticed that this seems to happen when something in systemd is
reconfigured, e.g. enabling a service. I saw this in the log:

Dec 01 09:02:41 tix systemd[1]: Reloading.
Dec 01 09:02:41 tix systemd[1]: /lib/systemd/system/virtlogd.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-sock → /run/libvirt/virtlogd-sock; please update the unit file accordingly.
Dec 01 09:02:41 tix systemd[1]: /lib/systemd/system/virtlogd-admin.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlogd-admin-sock → /run/libvirt/virtlogd-admin-sock; please update the unit file accordingly.
Dec 01 09:02:41 tix systemd[1]: /lib/systemd/system/virtlockd.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-sock → /run/libvirt/virtlockd-sock; please update the unit file accordingly.
Dec 01 09:02:41 tix systemd[1]: /lib/systemd/system/virtlockd-admin.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/libvirt/virtlockd-admin-sock → /run/libvirt/virtlockd-admin-sock; please update the unit file accordingly.
Dec 01 09:02:41 tix systemd[1]: /lib/systemd/system/dbus.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/dbus/system_bus_socket → /run/dbus/system_bus_socket; please update the unit file accordingly.
Dec 01 09:02:41 tix systemd[1]: /lib/systemd/system/lxc.service:16: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
Dec 01 09:02:41 tix systemd[1]: /lib/systemd/system/lxc.service:17: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
Dec 01 09:02:41 tix systemd[1]: /lib/systemd/system/smartd.service:9: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
Dec 01 09:02:41 tix systemd[1]: /lib/systemd/system/apcupsd.service:13: PIDFile= references a path below legacy directory /var/run/, updating /var/run/apcupsd.pid → /run/apcupsd.pid; please update the unit file accordingly.
Dec 01 09:02:41 tix systemd[1]: machine-lxc\x2d30315\x2ddns.scope: Failed to set 'memory.limit_in_bytes' attribute on '/machine.slice/machine-lxc\x2d30315\x2ddns.scope' to '-1': Invalid argument

Note the last line:

	Failed to set 'memory.limit_in_bytes' attribute ...  to '-1'

Wht is it trying to do that? This seems to be the moment that the memory
settings in LXC is screwed up.

#976190#10
Date:
2020-12-01 13:25:22 UTC
From:
To:
Am 01.12.20 um 09:27 schrieb Paul Slootman:

I suppose, this is from within the container?

How do you apply/setup the limits?

#976190#17
Date:
2020-12-01 13:29:18 UTC
From:
To:
Hi,

Yes, correct

This is in the libvirt container definition:

<domain type='lxc'>
  <name>dns</name>
  <uuid>32bd6117-0082-49eb-b622-87e1de716c88</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <memtune>
    <hard_limit unit='KiB'>1048576</hard_limit>
    <soft_limit unit='KiB'>1048576</soft_limit>
    <swap_hard_limit unit='KiB'>1048576</swap_hard_limit>
  </memtune>
  <vcpu placement='static'>1</vcpu>


Paul

#976190#22
Date:
2020-12-01 13:54:38 UTC
From:
To:
Am 01.12.20 um 14:29 schrieb Paul Slootman:

Let's loop in the the libvirt maintainers.
If I understand Paul correctly (and read the journal log correctly), the
memory limits applied by libvirt are lost once a `systemctl
daemon-reload` is happening. I'm not really familiar how libvirt
interacts with lxc/systemd, so your input would be welcome here.

Michael

#976190#27
Date:
2020-12-01 14:03:33 UTC
From:
To:
Am 01.12.20 um 14:54 schrieb Michael Biebl:

As well as the lxc maintainers.

#976190#32
Date:
2022-07-05 18:27:23 UTC
From:
To:
Please let us know, if this is still reproducible with a recent version
of systemd. If so, please file this issue upstream at
https://github.com/systemd/systemd/issues

#976190#37
Date:
2022-07-05 18:30:35 UTC
From:
To:
Am 05.07.22 um 20:27 schrieb Mail Delivery System:


It appears the original email address is no longer reachable.
As we don't have enough information to reproduce the issue, I'm closing
the bug report.

Regards,
Michael