Dear maintainer,

after an update kea-dhcp4 refuses to start due to an apparmor
missconfiguration. To track down the problem I started the server
manualy. No luck. Same error(s) - Therefore further step backs.
Here to reproduce the problem:

1) Install kea-dhcp4-server
2) Start the server manualy:

# kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile
Unable to use interprocess sync lockfile (Permission denied): /var/run/kea/logger_lockfile

After adopting /etc/apparmor.d/usr.sbin.kea-dhcp4 by adding
"owner /run/kea/logger_lockfile rwk,":

# kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
2023-03-08 04:49:34.880 INFO  [kea-dhcp4.dhcp4/26720.140090514692544] DHCP4_STARTING Kea DHCPv4 server version 2.2.0 (stable) starting
2023-03-08 04:49:34.881 WARN  [kea-dhcp4.dhcp4/26720.140090514692544] DHCP4_CONFIG_SYNTAX_WARNING configuration syntax warning: /etc/kea/kea-dhcp4.conf:436.39: Extraneous comma. A piece of configuration may have been omitted.
INFO  HOSTS_BACKENDS_REGISTERED the following host backend types are available: mysql postgresql
INFO  DHCPSRV_CFGMGR_SOCKET_TYPE_DEFAULT "dhcp-socket-type" not specified , using default socket type raw
INFO  DHCPSRV_CFGMGR_NEW_SUBNET4 a new subnet has been added to configuration: 192.0.2.0/24 with params: t1=900, t2=1800, valid-lifetime=3600
INFO  COMMAND_ACCEPTOR_START Starting to accept connections via unix domain socket bound to /run/kea/kea4-ctrl-socket
INFO  DHCP4_CONFIG_COMPLETE DHCPv4 server has completed configuration: added IPv4 subnets: 1; DDNS: disabled
INFO  DHCPSRV_MEMFILE_DB opening memory file lease database: lfc-interval=3600 type=memfile universe=4
INFO  DHCPSRV_MEMFILE_LEASE_FILE_LOAD loading leases from file /var/lib/kea/kea-leases4.csv
2023-03-08 04:49:34.884 ERROR [kea-dhcp4.dhcp4/26720.140090514692544] DHCP4_CONFIG_LOAD_FAIL configuration error using file: /etc/kea/kea-dhcp4.conf, reason: Unable to open database: unable to open '/var/lib/kea/kea-leases4.csv'
2023-03-08 04:49:34.885 ERROR [kea-dhcp4.dhcp4/26720.140090514692544]
DHCP4_INIT_FAIL failed to initialize Kea server: configuration error
using file '/etc/kea/kea-dhcp4.conf': Unable to open database: unable to
open '/var/lib/kea/kea-leases4.csv'

I was unable to fix that issue exept by removing the entiere apparmor
profile.

Regards
    Benedikt Spranger

Hi,
This rule exists in the apparmor profile already:
https://salsa.debian.org/debian/isc-kea/-/blob/debian/unstable/debian/usr.sbin.kea-dhcp4#L36


DHCP4_INIT_FAIL failed to initialize Kea server: configuration error
There is a rule for that too:

https://salsa.debian.org/debian/isc-kea/-/blob/debian/unstable/debian/usr.sbin.kea-dhcp4#L45

Maybe you have some other apparmor profile installed, and when you upgraded
the package, it wasn't replaced?

What are the contents of your /etc/apparmor.d/usr.sbin.kea-dhcp4 file?

Do you have some dpkg backup file perhaps? Check
/etc/apparmor.d/usr.sbin.kea-dhcp4*

Benedikt Spranger wrote on 08/03/2023:
service files, like this:

Environment="KEA_LOCKFILE_DIR=/run/lock/kea"

which is different from the default /var/run/kea/, which got used in
your manual attempt.

The issue you're seeing is likely not with the lockfile. Running:

# KEA_LOCKFILE_DIR=/run/lock/kea kea-dhcp4 -c /etc/kea/kea-dhcp4.conf

may show the actual issue, but I suggest using e.g.

journalctl -u kea-dhcp4-server.service

Please do follow up to this bug if you figure out something more about
this issue: if there's a bug in the apparmor profile we want to fix is
sooner than later.

Thanks!

1)  Purge kea-dhcp4-server from the system to ensure a clean install
# apt-get purge kea-dhcp4-server

2) Ensure ther is no apparmor profile left:
# ls -l /etc/apparmor.d/
insgesamt 88
drwxr-xr-x 2 root root   95 15. Feb 08:03 abi
drwxr-xr-x 4 root root 4096 27. Feb 07:51 abstractions
drwxr-xr-x 2 root root    6 18. Mär 2018  force-complain
drwxr-xr-x 2 root root 4096 27. Feb 07:51 libvirt
drwxr-xr-x 3 root root 4096  8. Mär 19:40 local
-rw-r--r-- 1 root root 1379 14. Feb 12:49 lsb_release
-rw-r--r-- 1 root root 1189  3. Sep 2021  nvidia_modprobe
drwxr-xr-x 2 root root    6 26. Feb 2019  samba
-rw-r--r-- 1 root root 3461  9. Jan 09:25 sbin.dhclient
drwxr-xr-x 5 root root  266 15. Feb 08:03 tunables
-rw-r--r-- 1 root root 3448  5. Jul 2020  usr.bin.man
-rw-r--r-- 1 root root 2255 11. Nov 2020  usr.lib.ipsec.charon
-rw-r--r-- 1 root root  872 11. Nov 2020  usr.lib.ipsec.stroke
-rw-r--r-- 1 root root 1871 19. Aug 2021  usr.lib.libvirt.virt-aa-helper
-rw-r--r-- 1 root root 2628  1. Feb 2022  usr.sbin.chronyd
-rw-r--r-- 1 root root  761  5. Feb 00:25 usr.sbin.cups-browsed
-rw-r--r-- 1 root root 6027  6. Sep 2021  usr.sbin.cupsd
-rw-r--r-- 1 root root  621 25. Nov 2020  usr.sbin.haveged
-rw-r--r-- 1 root root  744 17. Feb 19:20 usr.sbin.kea-dhcp-ddns
-rw-r--r-- 1 root root  855 17. Feb 19:20 usr.sbin.kea-lfc
-rw-r--r-- 1 root root 4732 28. Jan 17:03 usr.sbin.libvirtd
-rw-r--r-- 1 root root  730 15. Okt 2020  usr.sbin.mariadbd
-rw-r--r-- 1 root root 2654 26. Jan 21:13 usr.sbin.named
-rw-r--r-- 1 root root 1196 11. Nov 2020  usr.sbin.swanctl

# aa-status
apparmor module is loaded.
25 profiles are loaded.
25 profiles are in enforce mode.
   /usr/bin/man
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/lib/cups/backend/cups-pdf
   /usr/lib/ipsec/charon
   /usr/lib/ipsec/stroke
   /usr/sbin/chronyd
   /usr/sbin/cups-browsed
   /usr/sbin/cupsd
   /usr/sbin/cupsd//third_party
   /usr/sbin/haveged
   /usr/sbin/swanctl
   /{,usr/}sbin/dhclient
   kea-dhcp-ddns
   kea-lfc
   libvirtd
   libvirtd//qemu_bridge_helper
   lsb_release
   man_filter
   man_groff
   named
   nvidia_modprobe
   nvidia_modprobe//kmod
   virt-aa-helper
0 profiles are in complain mode.
0 profiles are in kill mode.
0 profiles are in unconfined mode.
7 processes have profiles defined.
2 processes are in enforce mode.
   /usr/sbin/cupsd (6782)
   /usr/lib/cups/notifier/dbus (6785) /usr/sbin/cupsd
0 processes are in complain mode.
5 processes are unconfined but have a profile defined.
   /usr/lib/ipsec/charon (1820)
   /usr/sbin/chronyd (2268)
   /usr/sbin/chronyd (2317)
   /usr/sbin/cups-browsed (2199)
   /usr/sbin/haveged (1858)
0 processes are in mixed mode.
0 processes are in kill mode.

3) install kea-dhcp4-server
# apt-get install kea-dhcp4-server

4) Start manually:
# KEA_LOCKFILE_DIR=/run/lock/kea kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
2023-03-08 19:43:47.887 INFO  [kea-dhcp4.dhcp4/7774.139648314530240] DHCP4_STARTING Kea DHCPv4 server version 2.2.0 (stable) starting
2023-03-08 19:43:47.888 WARN  [kea-dhcp4.dhcp4/7774.139648314530240] DHCP4_CONFIG_SYNTAX_WARNING configuration syntax warning: /etc/kea/kea-dhcp4.conf:436.39: Extraneous comma. A piece of configuration may have been omitted.
INFO  HOSTS_BACKENDS_REGISTERED the following host backend types are available: mysql postgresql
INFO  DHCPSRV_CFGMGR_SOCKET_TYPE_DEFAULT "dhcp-socket-type" not specified , using default socket type raw
INFO  DHCPSRV_CFGMGR_NEW_SUBNET4 a new subnet has been added to configuration: 192.0.2.0/24 with params: t1=900, t2=1800, valid-lifetime=3600
INFO  COMMAND_ACCEPTOR_START Starting to accept connections via unix domain socket bound to /run/kea/kea4-ctrl-socket
INFO  DHCP4_CONFIG_COMPLETE DHCPv4 server has completed configuration: added IPv4 subnets: 1; DDNS: disabled
INFO  DHCPSRV_MEMFILE_DB opening memory file lease database: lfc-interval=3600 type=memfile universe=4
INFO  DHCPSRV_MEMFILE_LEASE_FILE_LOAD loading leases from file /var/lib/kea/kea-leases4.csv
2023-03-08 19:43:47.891 ERROR [kea-dhcp4.dhcp4/7774.139648314530240] DHCP4_CONFIG_LOAD_FAIL configuration error using file: /etc/kea/kea-dhcp4.conf, reason: Unable to open database: unable to open '/var/lib/kea/kea-leases4.csv'
2023-03-08 19:43:47.891 ERROR [kea-dhcp4.dhcp4/7774.139648314530240] DHCP4_INIT_FAIL failed to initialize Kea server: configuration error using file '/etc/kea/kea-dhcp4.conf': Unable to open database: unable to open '/var/lib/kea/kea-leases4.csv'

QED: Same apparmor error I could not fix...

# ls /etc/apparmor.d/usr.sbin.kea-dhcp4*
/etc/apparmor.d/usr.sbin.kea-dhcp4

The content of /etc/apparmor.d/usr.sbin.kea-dhcp4:
--- 8< ---
abi <abi/3.0>,

include <tunables/global>

profile kea-dhcp4 /usr/sbin/kea-dhcp4 {
  include <abstractions/base>

  # for MySQL access, localhost
  include <abstractions/mysql>
  include <abstractions/openssl>

  capability net_bind_service,
  capability net_raw,

  network inet dgram,
  network inet stream,
  network netlink raw,
  network packet raw,

  /etc/nsswitch.conf r,
  /etc/services r,
  /etc/hosts r,
  /etc/host.conf r,
  /etc/host.conf r,
  /run/systemd/resolve/stub-resolv.conf r,

  /etc/gss/mech.d/ r,
  /etc/gss/mech.d/* r,

  /etc/kea/ r,
  /etc/kea/** r,
  /usr/sbin/kea-dhcp4 mr,
  /usr/sbin/kea-lfc Px,

  owner /run/kea/kea-dhcp4.kea-dhcp4.pid w,
  owner /run/lock/kea/logger_lockfile rwk,

  # Control sockets
  # Before LP: #1863100, these were in /tmp. For compatibility, let's keep both
  # locations
  owner /{tmp,run/kea}/kea4-ctrl-socket w,
  owner /{tmp,run/kea}/kea4-ctrl-socket.lock rwk,

  # this includes .completed, .output, .pid, .[0-9]
  owner /var/lib/kea/kea-leases4.csv* rw,

  owner /var/log/kea/kea-dhcp4.log rw,
  owner /var/log/kea/kea-dhcp4.log.[0-9]* rw,
  owner /var/log/kea/kea-dhcp4.log.lock rwk,
--- 8< ---

Regards
    Benedikt Spranger

PS: The sysv init scripts are broken. Patch follows when the apparmor issue is fixed.

Hi,

what's the actual apparmor DENIED message you get in the logs? Check
`dmesg`.

I see you are not using the systemd unit, so I suspect you are running kea
as root directly, instead of as the unprivileged `_kea` user, and you are
probably tripping over the "owner" flag of the apparmor rules.

Thanks for the hint... (\me buys some big brown paperbag...)

It is working now with the following patch to /etc/init.d/kea-dhcp4-server.

Regards
    Benedikt Spranger

--- /etc/init.d/kea-dhcp4-server.orig       2023-03-08 22:00:35.249600025 +0100
+++ /etc/init.d/kea-dhcp4-server    2023-03-08 22:12:11.803979999 +0100
@@ -16,7 +16,7 @@
 # PATH should only include /usr/* if it runs after the mountnfs.sh script
 PATH=/sbin:/usr/sbin:/bin:/usr/bin
 DESC="kea-dhcp4"
-NAME=kea-dhcp4-server
+NAME=kea-dhcp4
 DAEMON=/usr/sbin/kea-dhcp4
 DAEMON_ARGS="-c /etc/kea/kea-dhcp4.conf"
 PIDFILE=/run/$NAME.pid
@@ -24,6 +24,7 @@
 KEA_PIDFILE_DIR=/run/
 KEA_LOCKFILE_DIR=/run/lock/kea/
 KEA_LOGGER_DESTINATION=/var/log/kea/
+KEA_USER=_kea

 # Exit if the package is not installed
 [ -x "$DAEMON" ] || exit 0
@@ -52,13 +53,14 @@
 do_start()
 {
        create_lockfile_dir
+       export KEA_LOCKFILE_DIR
        # Return
        #   0 if daemon has been started
        #   1 if daemon was already running
        #   2 if daemon could not be started
-       start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
+       start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -u $KEA_USER --test > /dev/null \
                || return 1
-       start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -b -- \
+       start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -c $KEA_USER -b -- \
                $DAEMON_ARGS \
                || return 2
 }

bene wrote on 08/03/2023:

[...]

Thanks for the patch. However I have a couple of questions:

Are you actually using Bookworm with sysv, having removed systemd, or
are you using the init.d scripts for some other reason (integration with
other software, habit, ...)?

If your init system is systemd, then I strongly advise using systemctl
to start/stop/... the daemons. I don't think the init scripts are
actively maintained at the moment, as you noticed (FIXME kea team, Cc:).
Plus QA on the package (e.g. DEP8 tests) is done assuming systemd.

If you are a sysv init user, are you willing to test packages with a
candidate fix, before an upload is done? I am not running sysv systems;
The case looks simple enough for me to attempt a fix, but I need
validation from an actual sysv user. Even better if you can submit a
salsa MR, which will also speed up the process of landing a fix:

https://salsa.debian.org/debian/isc-kea/

Cheers,

Paride

On Fri, 10 Mar 2023 15:04:44 +0100 Paride Legovini <paride@debian.org> wrote:

[...]
I am using bookworm/sid with sysv, having systemd not installed/purged.
systemd simply does not fit *my* needs, while sysv does. Therefore sysv.
See above: no systemd
I am aware of that. And I fully understand the rationale behind that
decision.
If you need help here, do not hesitate to ask. I can test the package.

Can do that next week. ATM I am busy to prepare stuff for a trade fair
starting next week...

Regards
    Bene

Benedikt Spranger wrote on 10/03/2023:

Sound good, thanks! Keep in mind that Debian will be in hard freeze.
Given that isc-kea is a non-key package with autopkgtests we'll still be
able to upload a "small, targeted fix" [1] for this issue, but the
sooner the better.

Cheers,

Paride

[1] https://release.debian.org/testing/freeze_policy.html#full

Same deal here, but on bookworm using systemd and the installation is some 10 days old.

Regards

Luigi Baldoni wrote on 15/10/2023:

Hello Luigi, that is likely a different issue. Can you please file a ne
bug report, describing the problem you are facing in more detail, possibly
providing steps to reproduce from a clean Bookworm system?

Thank you,

Paride

Paride Legovini wronte on 19/10/2023:

For future reference, the error was:
audit[7919]: AVC apparmor="DENIED" operation="mknod" profile="kea-dhcp4" name="/run/kea/logger_lockfile" pid=7919 comm="kea-dhcp4" requested_mask="c" denied_mask="c" fsuid=107 ouid=107

which was fixed by this patch:

diff -udpr etc/apparmor.d.orig/usr.sbin.kea-dhcp4 etc/apparmor.d/usr.sbin.kea-dhcp4
--- etc/apparmor.d.orig/usr.sbin.kea-dhcp4      2023-10-20 13:17:59.724793546 +0200
+++ etc/apparmor.d/usr.sbin.kea-dhcp4   2023-10-20 13:18:54.981066011 +0200
@@ -27,7 +27,7 @@ profile kea-dhcp4 /usr/sbin/kea-dhcp4 {
   /usr/sbin/kea-lfc Px,

   owner /run/kea/kea-dhcp4.kea-dhcp4.pid w,
-  owner /run/lock/kea/logger_lockfile rwk,
+  owner /run/{lock,}/kea/logger_lockfile rwk,

   # Control sockets
   # Before LP: #1863100, these were in /tmp. For compatibility, let's keep both
diff -udpr etc/apparmor.d.orig/usr.sbin.kea-lfc etc/apparmor.d/usr.sbin.kea-lfc
--- etc/apparmor.d.orig/usr.sbin.kea-lfc        2023-10-20 13:17:59.724793546 +0200
+++ etc/apparmor.d/usr.sbin.kea-lfc     2023-10-20 13:19:04.205111517 +0200
@@ -11,7 +11,7 @@ profile kea-lfc /usr/sbin/kea-lfc {
   /usr/sbin/kea-lfc mr,

   owner /run/kea/logger_lockfile rwk,
-  owner /run/lock/kea/logger_lockfile rw,
+  owner /run/{lock,}/kea/logger_lockfile rwk,

   # Control sockets
   # Before LP: #1863100, these were in /tmp. For compatibility, let's keep both

Hi, I hope I am not misunderstanding this, but I think I've got the same
problem with

[2949072.463008] audit: type=1400 audit(1704633046.887:50): apparmor="DENIED" operation="open" profile="kea-dhcp4" name="/run/kea/kea-dhcp4.kea-dhcp4.pid" pid=3589658 comm="kea-dhcp4" requested_mask="r" denied_mask="r" fsuid=124 ouid=124

ii  kea-dhcp4-server 2.2.0-6      arm64        IPv4 DHCP server

/ralph