Hi. Earlier this year I was asked [#1072021] to remove Recommends: ... system-log-daemon from one of my packages. There are some explanations here: [0]: https://lists.debian.org/debian-devel/2024/05/msg00425.html https://lists.debian.org/debian-devel/2024/05/msg00436.html The effect of making this change is that on non-systemd systems, nothing pulls in a syslog daemon and no logging takes place. This seems wrong. Also, I notice that this MBF proposal appears to have had no involvement with the Policy process even though it, effectively, amounts to the abolition of the system-log-daemon virtual package, which is, of course, established by Policy [1]. It seems to me that the semantics of the "system-log-daemon" virtual package must mean "there is a syslog service". Since systemd is capable of being a syslog service, that means that it should Provide that virtual package. Because the syslog daemon is a singleton, implementors of the virtual package should Conflict/Provide. Therefore, to avoid apt trying to deinstall systemd, the parts of the systemd.deb that provide this functionality (or enable it) should be split out into a separate package - ie systemd should own and listen on the standard syslog socket iff that package is installed. That is how multiple mutually-exclusive provisions of of the same facility are done in Debian. This option was rejected by the MBF proposal on the grounds that [0] This doesn't seem to make sense since packages can be installed by default, so it would be possible to split the package and and retain the intended behaviour in the default install. So it seems to me that there is no reason why systemd's syslog functionality couldn't follow Policy, use the virtual package as intended. It should do so, and all changes made as a result of the MBF should be reverted. Perhaps there are some other reasons, why this is not feasible or desirable. In that case, we should still arrange that systems which do *not* have systemd, still end up with a syslog daemon when required. Simon McVitte had a suggestion how this might be achieved [2] but this was not embodied in the MBF. In any case, Policy should be updated rather than bypassed. Please would the TC reaffirm policy, and give appropriate advice. Alternatively, if policy needs to change please would the TC assist this process, and help ensure that the resulting policy (i) suits the needs of all Debian configurations (ii) is properly documented (iii) is implemented. On a previous occasion when I brought a matter to the TC, I was subjected to a sustained campaign of harassment, on the TC mailing list, which Debian's authorities collectively allowed to continue. THe harassers included full Debian members and one of the proponents of the present MBF. Therefore: please do not email me any further about this subject, until a TC decision is made. When Policy is updated, I will change my packages accordingly. In the meantime, I'll adjust my mail filters to discard messages to this bug. Ian. [1] https://www.debian.org/doc/debian-policy/ch-relationships.html#relationships-between-source-and-binary-packages-build-depends-build-depends-indep-build-depends-arch-build-conflicts-build-conflicts-indep-build-conflicts-arch https://www.debian.org/doc/packaging-manuals/virtual-package-names-list.yaml [2] https://lists.debian.org/debian-devel/2024/05/msg00426.html
Hi, I agree that the proposed solution is less than ideal. As far as I understand things, this characterization is no longer accurate. The system-log-daemon facility used to be singleton, but the way systemd approaches it is no longer singleton. You can have: * No logging facility. * systemd-journald logging to /var/log/journal * A traditional system-log-daemon such as rsyslog running on sysvinit. * systemd-journald forwarding logs to a traditional system-log-daemon such as rsyslog. The tricky part here is that journald kinda implements what packages expect from the system-log-daemon virtual package but at the same time, it does not conflict with other system-log-daemons due to the message forwarding. I can relate to this view in the sense that journald is tightly coupled into systemd and that operating a systemd-as-pid-1 without journald is not something that can be expected to work. In such a split, systemd-sysv would likely depend on journald and as such exclude any other system-log-daemon, but they are meant to be able to coexist. systemd-journald would like to provide system-log-daemon without excluding another system-log-daemon. Our mechanism for multiple providers very much assumes the singleton approach that systemd would to not follow as systemd maintainers consider coexistence of journald with another system-log-daemon an important feature. I'm not sure what the solution is, but the current system-log-daemon implementation doesn't cut it. Either the reference is wrong or the attribution should be Simon Richter. The proposal is to update system-log-daemon dependencies to "systemd-sysv | system-log-daemon" and that is something that cover the relevant use cases of running systemd-only, sysvinit with system-log-daemon and systemd with forwarding to a system-log-daemon. It feels a bit annoying to have to encode systemd-sysv here, but on a technical level, that's what we want to express here. I fear what is required here is design work. In principle, we could add another virtual package dev-log-socket. It would be provided by systemd-sysv and any system-log-daemon, but without declaring Conflicts. Then all dependencies on system-log-daemon that merely require /dev/log to be captured into storage (which I expect most to be) can switch their dependency to dev-log-socket. I also feel like the problem at hand is a bit academic in nature. The vast majority of Debian installations are running systemd as pid 1 and journald with a persistent journal (by default). The system-log-daemon facility is therefore available by default in the vast majority of installations. In the container case, neither installing systemd-sysv nor a system-log-daemon provider tends to help as no services are being started (Simon Richter pointed this out). In the non-systemd case, an administrator will manually install a system-log-daemon in all but unusual situations. The ability to not install a system-log-daemon (or systemd-sysv) even when formally required (e.g. by forwarding the /dev/log socket from a container to an external journald process) seems like a benefit to me as it adds flexibility while not impacting the common case. I would like to agree, but Policy only mentions system-log-daemon in virtual-package-names-list.yaml: | - name: system-log-daemon | description: a daemon that provides a logging facility for other applications This description still feels accurate except that journald cannot provide for the sake of supporting coexistence. As a result, I am not sure what change could be effected here to resolve Ian's concern. I see how Ian had a bad experience earlier. His refusal to interact with opponents vaguely makes sense on those ground, but doesn't help the matter. His refusal to interact with CTTE members removes our ability to solicit feedback in order to resolve the matter. Therefore I suggest that we completely turn down the request on procedural grounds: Either he is willing to discuss the matter with us (not with systemd people) or we cannot help resolve it. I am intentionally not taking ownership (in the sense of our moderation procedure) of this bug as my availability is limited during the next weeks. Helmut
Helmut> I see how Ian had a bad experience earlier. His refusal to
Helmut> interact with opponents vaguely makes sense on those ground,
Helmut> but doesn't help the matter. His refusal to interact with
Helmut> CTTE members removes our ability to solicit feedback in
Helmut> order to resolve the matter. Therefore I suggest that we
Helmut> completely turn down the request on procedural grounds:
Helmut> Either he is willing to discuss the matter with us (not with
Helmut> systemd people) or we cannot help resolve it.
I think this sounds like a great general principle. In order to bring
something to the TC, you need to be willing to work with the TC at least
to work on a solution. I would encourage the TC in drafting language to
turn down requests under these grounds to point out that if the
proponent of the request is uncomfortable working with the TC, they can
find another proponent. If they cannot find another proponent, it
strongly suggests that the issue might not have enough support to
justify a TC intervention.
Hello, I think we should put some effort into Ian's bug, and not reject it on purely procedural grounds. We have had several bugs where key parties have simply ignored our requests for engagement, and we have had difficulty resolving those bugs without that engagement, but we have nevertheless been able to make progress. Ian is explicitly limiting his involvement in order to protect himself. That's at least as good as, if not preferable to, implicitly limiting one's involvement by simply ignoring TC requests for engagements. It seems, therefore, unfair to summarily dismiss Ian's bug if we didn't do that for those other cases.
Hi, On Sat, 2024-10-12 at 17:29 +0800, Sean Whitton wrote: Okay, then let me ask some questions: 1. What should decide whether system-wide logging facilities exist? Some central defaults or random packages (say foobard) shipping a daemon? 1a. If not random packages, should policy be updated to recommend packages not doing that? 2. Does a patch for the proposed changes to the systemd packaging exist? 3. Why should use of systemd-journald together with rsyslogd (or other syslog servies) be made impossible? (Ian demands these to conflict.) This is a widely used configuration as far as I know. What is the alternative for our users? Ansgar
Likewise (and I see some strength in the argument that stopping anything from depending on system-log-daemon is a policy change even if the system-log-daemon remains as only a thing that things Provide: and Conflicts:). objection is that doing so means that container images may end up pulling in an unnecessary syslog package. I don't think that's the end of the world (it's a couple of MB at most), but I can see that a solution that avoided this would have merit. Hippotat Recommends: system-log-daemon, however, so presumably if one were putting it into a minimal image, one would not be pulling in Recommends: willy-nilly in any case. My feeling is, I think, therefore, that packages should still be able to Recommend (et al) system-log-daemon (since we do have sysvinit users of Debian), and that Simon Richter's suggestion is a sensible way of achieving that absent a neater solution that helps with the container-building question. Regards, Matthew
Hello,
I think I see a way to distinguish these four cases in a way that gets
everyone what they want.
systemd adds an *empty* binary package
Package: systemd-journald-is-syslog
Provides/Conflicts: system-log-daemon
We install systemd-journald-is-syslog by default, probably using a
Recommends from one of systemd's other binary packages.
It's an empty binary package, so journald is still installed and running
if systemd is PID 1, no matter what. Then:
systemctl disable systemd-journald
systemd-journald-is-syslog can remain or installed or be removed
depending on whether the sysadmin wants to assert to other packages that
a logging facility if available, even though one isn't actually running.
This is the default. systemd-journald-is-syslog is installed.
The sysadmin installs rsyslog, which deinstalls
systemd-journald-is-syslog, in the usual way with virtual packages.
Nothing else about the systemd installation changes.
Well, in this case, systemd and systemd-* are not installed, and rsyslog
Provides system-log-daemon.
* Sean Whitton <spwhitton@spwhitton.name> [241028 09:39]: It seems like there are some words missing or incorrect in this sentence, could you please clarify? Today, this is the setup where both rsyslog and systemd are installed, is common on servers, and "just works". Making this setup harder than necessary seems wrong and probably not what was intended to say? Chris
Hello, Ooops, thanks. I got the last two the wrong way around. Here are the four cases again, corrected: systemctl disable systemd-journald systemd-journald-is-syslog can remain or installed or be removed depending on whether the sysadmin wants to assert to other packages that a logging facility if available, even though one isn't actually running. This is the default. systemd-journald-is-syslog is installed. Well, in this case, systemd and systemd-* are not installed, and rsyslog Provides system-log-daemon. The sysadmin installs rsyslog, which deinstalls systemd-journald-is-syslog, in the usual way with virtual packages. Nothing else about the systemd installation changes.
Hi Sean, Thank you for bringing this up. Despite the little confusion in the end that Chris remarked, I think this practically covers the four cases. However, I think there is a fifth case that is becoming more and more practically relevant. Both docker and podman now have a logging driver called journald. https://docs.docker.com/engine/logging/drivers/journald/ I'm not yet sure exactly how this works, but the context is "slim" containers (i.e. those that do not run systemd as pid 1) and I very much expect them to not run a journald from the container environment either. Rather the container runtime essentially provides /dev/log and a journald socket to the container in some (unspecified?) way. This kinda is similar to dbconfig where we have dbconfig-no-thanks. We'd need another package syslog-no-thanks that would have the same provides and conflicts but no systemd dependency now. Of course just adding such a package would result in apt selecting it whenever systemd is difficult to install. In effect, adding such a package would render dependencies on system-log-daemon meaningless and we could just drop them - which is what has been happening and has resulted in this bug. So now we're running in circles. Effectively, we must decide whether the container use case is more important than the non-systemd case or the other way round. I do not see a way to make both just work in a sane way. Helmut
Hello, I think I understand what you mean about why a syslog-no-thanks virtual package would not be helpful, but I don't understand how the journald driver option interacts with my four options. How do systemd and these journald drivers interact? Aren't we in case #1?
Wouldn't we expect that such containers have systemd (and thus journald) installed, but nothing will start them (because there is no init). If that is the case (I think it currently is), then nothing special needs to happen. If someone were to define how Debian should *generally* behave in slim containers, maybe the /dev/log topic should be part of that definition. But so far I don't think a definition exists. Policy is probably silent about slim containers, too? Chris
Hi Sean,
Admittedly, my understanding of this is quite shallow as well. Roughly
speaking when we talk about docker/podman we most often imply an
application container (i.e. one not running systemd as pid 1) rather
than a system container (where there are systemd and jounrnald proceses
inside the container). More specifically, if we were talking about
system containers, we could easily install your
systemd-journald-is-syslog package and no problem would arise.
Therefore, I assume we are talking about application containers only. An
application container will not have systemd-sysv and probably not even
systemd installed. As a consequence, systemd-journald-is-syslog cannot
be installed as it would have an unsatisfied dependency. Still the
docker/podman journald driver would ensure that /dev/log and
/run/systemd/journal/socket are present and usable. As such the syslog()
function will work in the sense that the log message will be recorded
somewhere. This is different from your #1 ("No logging facility"). No
installed package provides system-log-daemon even though the semantics
of system-log-daemon are provided by the container runtime. From a
satisfiability point of view, we certainly are in your #1 case, but we'd
like to assert to other packages, that system-log-daemon is available
and we cannot without a syslog-no-thanks package.
To put this another way, we need to gain more clarity about what
system-log-daemon actually means beyond what is recorded in policy:
| a daemon that provides a logging facility for other applications
If the focus of this description is on providing /dev/log as a means to
record log message, we can no longer express this in terms of package
relations, because we can run the same container image with or without
the journald driver and thus have this property differ. This is vaguely
similar to how we cannot depend on the availability of Linux kernel
features or CPU instruction set extensions (even though we do via
isa-support).
If the focus is on the exclusion of alternative logging mechanisms
rather than about the availability of a /dev/log listener, then removing
dependencies on this virtual package (as has been happening) is the
right way forward.
To me, this is a strong clue that we should be updating Debian policy in
some way as the current ambiguity is evidently resulting in incompatible
interpretations. Do you agree with this? If yes, would you clone a
policy bug asking to clarify the meaning of system-log-daemon?
Helmut
Chris Hofstaedtler wrote: Sometimes, but not always. Many application containers will not have an ordinary init system present at all; they'll be launched entirely by the container system, which bind-mounts things from outside the container into the container's filesystem, unshares, and then either serves as a minimal init itself (if the application might do non-trivial things like fork/exec and might need init as a reaper) or directly execs the application (especially if the entire container is mostly going to have only 1 process in it). Debian can't serve *all* of these use cases, in part because of Essential. And there's a limit to the degree that Debian should cater to cases that force-remove essential packages to make small containers, in the absence of a distro-wide effort to reduce Essential. But the use case of slim containers does not inherently require having any init system installed within the container. That said, I think it'd be perfectly reasonable to defer that use case, and focus primarily on the use case of letting packages generally request that /dev/log be provided by something, without requiring users of systemd-journald to install a separate syslog daemon they don't want. A Debian-provided package that's *effectively* an equivs stub saying "install this to assert that something will provide /dev/log" seems pretty reasonable for that. There's no way to assert that the init system providing that is actually *running*, because dependencies can only say what is installed and not what is running, but it's not obvious how to do any *better* than that within the framework of dependencies.
Hello Helmut, Josh, Chris, Thank you for your replies. Helmut, I think my conversation with you is somewhat verging into detailed design work. We don't want the TC to be trying to decide exactly what sort of containers we want to support; we just want to be sure we're not definitely blocking anything we don't want to definitively block. So, I think my proposal (to the extent it too is not design work) still stands as a resolution to the bug. I'll write to Ian about it off-list to see if he agrees.
I wonder why we then not adopt the systemd ways of seeing this. The system is expected to provide the logging facility or their lack of. So sysvinit just needs to pull in a syslog daemon. Then we don't longer have that burden on the lower packages. Bastian
Hi Sean, I respectfully disagree with your characterization. I used the podman example to demonstrate actual use of the underlying concept that a container runtime would be responsible for providing logging services and concluded that a dependency on logging services would no longer be expressible in our dependency system. To me, this is not yet design work. Rather we would be shifting to a state where logging services are always assumed available and all that we would continue to maintain is the exclusion mechanism. I have little doubts that Ian would agree with your proposal. I do have severe doubts that systemd maintainers would agree with it though. So the ones to talk to first would be systemd maintainers in my view. Given my current understanding of the matter, I'd vote your proposal below NOTA, because I think that it leaves an important use case unaddressed. Indeed, we can lift Bastian's mail into a proper proposal. Logging services are generally assumed to be available. It becomes the responsibility of the init system or container runtime. The system-log-daemon virtual package mainly serves as an exclusion mechanism. Alternative init systems such as sysvinit-core should issue a dependency or recommendation on it. Helmut
Hi, This seems like the wrong shape of solution to me; we've not previously assumed this, and I don't think the case for such a policy change has yet been made. Regards, Matthew
Hi, Random packages not installing system facilities like system-wide log services was already suggested as a solution earlier: 1. What should decide whether system-wide logging facilities exist? Some central defaults or random packages (say foobard) shipping a daemon? 1a. If not random packages, should policy be updated to recommend packages not doing that? There was no explicit answer to either of these questions (not others) so far. (FWIW I consider a popcon=1 custom VPN protocol server package like the package this tech-ctte bug is about a "random package" in the context of question #1) Could I ask why the ctte considers this the wrong solution? As it was suggested previously, I assume it was at least taken into consideration in discussions. Ansgar
Hello, The Debian way is to use virtual package dependencies for this sort of thing.
runtimes consider logging as a system facility. So it would be the task of the equivalent package: sysvinit or initscripts. Just think who is responsible for filesystem mounts: systemd, container runtime, initscripts. Add what? I don't even see anything about syslog in it right now. Bastian
Hello, I read your previous message again and I think I understand the example a bit better now. So, these super-slim containers don't have systemd installed at all, so systemd can't provide the virtual package inside them, right? Also, sorry -- what exactly do you mean when you say "the exclusion mechanism"? Thanks.
system-log-daemon On a typical install, that will install a log daemon; but a user who wants more control can arrange that it not do so. The maintainer is saying that "in all but unusual installations" a system-log-daemon would be found installed alongside hippotat-server. That doesn't seem on the face of it to be an unreasonable thing for a maintainer to say; if it were wrong in the case of this particular package, then that would be a bug against that package. Recommends: not a Depends: Although, even then, it's not entirely clear to me that a package that would only work if there was a system-log-daemon available shouldn't Depends: upon it. Again, if in fact it would work just fine without, then that would be a bug against the particular package, but that doesn't mean that it's in principle wrong to depend on system-log-daemon. To address Bastian's point in a follow-up to yours, policy currently contains the system-log-daemon virtual package "a daemon that provides a logging facility for other applications". In the past, people have understood this as a thing that they might reasonably declare dependencies upon, as well as a thing that they might Provides: and Conflicts:. I.e., the historical understanding of the system-log-daemon virtual package was as an optional facility that might be needed on an installation, and that might be provided by a number of different packages. As I understand the position of the systemd maintainers, they want to change the interpretation of the system-log-daemon package to being only something one can Provides: and/or Conflicts:. They want to change policy such that every Debian system can be assumed to have a logging facility available. I'm not entirely clear on the problem with Sean's proposal (systemd-journald-is-syslog). To be clear, that is my position, not the committees. Regards, Matthew
Hi Sean,
I am relieved to see that our disagreement is rooted in
misunderstanding.
distinction I've seen is into "fat" or "system" containers where there
is a process with pid 1 (commonly systemd) doing service supervision
inside the container and "application" container where the actual
application is the only process (group) inside the container (either as
pid 1 or with a minimalistic supervisor provided by the container
runtime). Essentially, any kind of bubblewrap, flatpak, snap, as well as
most use of docker/podman fall into this latter category. We don't
expect the systemd binary package to be installed in such containers and
if it happens to be, we don't expect systemd to run as pid 1.
Does this add clarity?
I would also like to observer that Build-Depends: system-log-daemon will
commonly get you an environment that lack logging services.
In my understanding, system-log-daemon virtual package originally served
two distinct purposes:
1. If you depend on it, then you can assume that messages emitted using
syslog(3) or written to /dev/log are handled in some means.
2. The dependency system prevents two implementations of a /dev/log
handling service from being installed at the same time.
When I say "exclusion mechanism" I refer to the second purpose. My
understanding of the conversation thus far is that we have broad
consensus for continued use of the virtual package for preventing
coinstallation of two logging services (2) while we have some
disagreement about how to express a requirement on logging services to
be available (1).
Helmut
Re: Matthew Vernon I'm inclined to say logging should be a system facility and nothing that a "normal" package should depend on. Then if I *don't* want logging, I could just remove that facility and even installing packages that would usually use logging wouldn't pull it back in. The expectation would be that debian-installer would set up logging in normal cases. Same for containers etc. Packages should only depend on logging (system-log-daemon or whatever incantation) if they specifically do something with logging, like fail2ban and the like. Christoph
Hello, Thanks, I see what you mean now.
Hello, I struggle to see why logging is special, in this case. Why isn't an MTA a system facility? Well, because many many systems don't need an MTA at all. And similarly, others might not want a standard logging facility, because they do something else to record their work, or specifically don't want to (some simple appliance). I think *reading* logs is something else entirely. That's highly log daemon-dependent: depending on system-log-daemon doesn't enable fail2ban to know where to look for the logs.
Hi Sean, system-log-daemon" so it is easier to not have it installed at all. We spent quite some time to have packages *NOT* "Recommend: mta" so it doesn't get installed by default. (Though I think there are still left- overs outside the default install...) Having "system facilities" that the admins decide to install or not install makes keeping optional stuff optional easier than random "Recommends:" pulling in packages that might or might not be useful for individual installations. As far as I understand there are often concerns that "Recommends:" is sometimes far too inclusive in Debian. On the other hand some people argue that a "proper" UNIX-like system always needs a MTA (and a running syslog and a compiler and so on) because that is what some systems looked like in the past... Ansgar
Re: Ansgar 🙀 That's what I was trying to say, yes. Ack. Right, but times are changing. In the past, fail2ban could depend on system-log-daemon and expect files in /var/log/ to appear, but systemd/journalctl keep things elsewhere. Perhaps that is the issue we should be discussing? Christoph
Hi Sean, This is asking a very sensible question that helps us getting closer to the core of the matter. Thank you! To me there are multiple aspects that indeed make it special. If logging fails, applications typically ignore the error and move on. There isn't much they can do if logging fails. I mean for many other kinds of failures you might log a message or abort execution. Neither of these makes sense when logging fails. It is a facility whose absence tends to be handled gracefully. Some container runtimes provide logging. This is a relatively recent development, but it makes logging somewhat similar to the Linux kernel. You don't depend on a new kernel package when you need new syscalls and even if you were, that would be no guarantuee that you'd actually be running a recent kernel. I'm only aware of one container deployment where smtp service is part of the runtime and that's fairly idiosyncratic and the way of doing it is providing 127.0.0.1:25. Commonly though, the expectation of a local MTA is to also provide /usr/bin/sendmail. If you happen to usr /usr/bin/logger on the other hand, what you need is bsdutils and not the logging service. And then the amount of installations that have logging services by default also is an aspect to consider. We basically take for granted that every system that is running systemd as pid 1 (container or not) is providing logging services. This group of systems is now a strong majority. On the flip side, all kinds of build environments are not providing logging services even if you depend on system-log-daemon. As a result, the benefit of expressing this dependency is relatively low. These may not be the strong reasons that you were looking for and each of them in isolation would not be convincing me. It is more the sum of them and I recognize that your conclusion may differ from mine here. Helmut
the systemd authors want it to be something that the authors of system
services can take for granted as "part of the API", so that authors of
system services aren't all expected to implement their own equivalent of
e.g. /var/log/apache2/, with its own configuration.
Specifically, systemd-as-pid-1 implies that /dev/log, syslog(3), the
sd_journal family of APIs, the sockets in /run/systemd/journal, and even
the stderr of a service with no special configuration are all reasonable
ways to log diagnostic messages, and messages sent to any of those places
will end up in the Journal somehow.
Precisely where Journal messages will end up is a decision for the
sysadmin (a limited-size transient database in /run/log, a larger
database in /var/log/journal that persists across reboots, flat text
files in /var/log, and/or forwarding to a remote log server), but the
only components that need to know which choice the sysadmin has made
here, other than journald itself, are those that directly interact with
the logs behind systemd's back (for example by reading /var/log/syslog).
I think it could be reasonable to say that any fully-bootable
Debian system with an init system should be expected to provide the
non-systemd-specific subset of this interface: a writeable /dev/log and
therefore a working syslog(3), and perhaps also providing some sort of
reasonable stderr to system services by default. Precisely where logs
sent to those places end up is a quality-of-implementation issue, with
answers that could include an in-memory ring buffer, flat text files, a
remote log server, or in the worst case scenario, /dev/null (in which case
the services will still work but their diagnostic messages will be lost).
For Debian systems without an init system - meaning chroots, and "thin"
application containers with no init system, which are essentially the same
concept implemented differently - it is already the case that there is no
dependency that you can add to a package that will guarantee that there is
a working /dev/log. If the chroot/container manager implementation happens
to provide a working /dev/log, then it will continue to work whether
your package has a dependency on system-log-daemon or not. Otherwise,
it will not work, and adding a dependency on system-log-daemon will not
solve that (because a system-log-daemon like rsyslog will be *installed*,
but that doesn't mean anything for whether it is *running*).
smcv
Hello, What issue exactly do you mean? I'm asking because what you describe does seem to be the issue we are discussing, to me.
Hello, But for maximum flexibility, we try to express as much as possible in our dependency system. If someone doesn't want a logging facility, they should probably blacklist it for instllation in /etc/apt/apt.conf.d. It's still reasonable for a package maintainer to assert that in all but unusual installations, this particular daemon would want to log. I don't believe this is relevant.
Helmut, Simon, Thank you for your arguments. I myself find them compelling. I think that I now understand better why Ian made reference to the Debian Policy process. If there was a bug called "abolish the system-log-daemon virtual package" against bin:debian-policy, then your messages could be copied verbatim to that bug. So, I think we could pass a resolution saying that the Policy Changes Process should be followed before anyone is asked to remove reference to the system-log-daemon virtual package from d/control. It's always true that Policy discussions that cannot reach consensus can be referred to the TC, but we could also explicitly state in our resolution that we would like it to come back to us if that process fails to establish consensus. I'm suggesting this because I think the Policy process will either resolve the issue, or get clearer on the stakes, such that the TC can then make a decision about what Debian will prioritise. We would also be encouraging a more appropriate escalation: try other decision-making procedures in the project, and use the TC as a last resort. How about it?
Re: Sean Whitton What exactly can a package expect to get when depending on system-log-daemon? 1) /dev/log exists and messages can be posted 2) messages appear in /var/log/something (with some amount of standardized filenames) 3) there is some way to emit log messages that the admin can read Afaict the OP's use case is actually the generic 3) case. (vpns should be able to log messages) fail2ban would want 2) (it doesn't care how messages get there). Everyone in this thread seems to assume that we are talking about 1). The classic sysloggers implement 1) 2) 3). journald implements 1) and 3), and we consider that ok. So loosely speaking, the only package in the archive that has a legitimate use case to depend on logging (fail2ban) depends on something that journald does not provide. Christoph
Hi Christoph, I think fail2ban is a red herring in this discussion. It has multiple backends one of which is called "systemd" and that happens to read the journal. The default backend is auto. fail2ban rightfully suggests system-log-daemon to prevent an installed daemon from being automatically removed. Other than that, it's default is to just work with journald. Helmut
Hi, there is also an alternative solution which I think hasn't been mentioned: 1. Add "Provides: system-log-daemon" to systemd 2. Drop the "Conflicts: system-log-daemon" from providers of system- log-daemon (1.) is technically correct: systemd provides the system-log-daemon system facility. (2.) is also technically correct: different implementations of system- log-daemon can coexists (even in useful ways) as shown with, for example, the common choice systemd+rsyslog. (Depending on the set of packages coinstalled this might require manual configuration by the admin; dropping the "Conflicts" also allows maximum flexibility as desired by some people.) In particular (2.) also works out of the box for common choices which is different from the time when having the "Conflicts" by default looked like a good choice, but reality has changed since then. (That said, I still think packages shouldn't depend on system facilities like a system log, but leave that to the admin / default setup to allow for flexibility; though technically "Depends/Recommends: system-log-daemon" and "Conflicts: system-log-daemon" are independent.) Ansgar
I think systemd is unusual in that it can co-exist with other system-log-daemon providers, so I don't think making this change would be wise. [FTR, the most recent TC meeting resolved we were at the point of voting on options for this bug, I've just been away recently and haven't yet got to drafting it. Soon, hopefully!] Regards, Matthew
Hi, I would expect most can. The admin might have to configure them to use a different socket. As a random example I checked syslog-ng. It can be configured to listen anywhere: https://syslog-ng.github.io/admin-guide/060_Sources/220_unix-stream_unix-dgram/README#example-using-the-unix-stream-and-unix-dgram-drivers That the default configuration might not allow multiple service to start by default is not that much of a problem. Note that apache2 and nginx don't conflict with each other even though both provide httpd and bind to port 80 in their default configuration (thus installing both fails to start at least one of them). system-log- daemon does the same, but with /dev/log instead of a TCP port. Ansgar
Hi, At the last TC meeting we concluded that we'd heard sufficient argument on this issue to clarify the issues, and that we should therefore proceed to a vote. Here's a draft ballot; please suggest changes and/or say you're happy with it in the next couple of days, I (or someone else!) can call for votes on it. ===8<=== In Bug #1084924, the Technical Committee was asked about a mass bug filing that aimed to remove all dependencies (except Provides: and Conflicts:) upon the system-log-daemon virtual package. Whilst the wording of policy in this area is unclear, the Technical Committee notes that long-standing practice in this area as reflected by policy was that packages could declare appropriate dependencies upon the system-log-daemon virtual package. The Technical Committee also acknowledges that on systemd systems, journald can serve the purpose of system-log-daemon, but that systemd also supports installing a separate system-log-daemon. A) The Technical Committee affirms that it is reasonable for a package to declare any suitable dependency upon the system-log-daemon virtual package. The Technical Committee suggests that Policy be updated to clarify this, and that maintainers who removed such dependencies as a result of the mass bug filing consider restoring them. B) The Technical Committee agrees that packages should now only declare Provides: and Conflicts: relationships with the system-log-daemon virtual package, and that Debian systems may be assumed to run a syslog logger. The Technical Committee suggests that Policy be update to reflect this change. C) The Technical Committee resolves that this is a de facto attempt to change Policy, and that the Policy process should be used to consider whether to change Policy relating to system-log-daemon from the status quo of packages being able to declare any reasonable dependency upon system-log-daemon to the state where only Provides: and Conflicts: may be used. Until that process is concluded, dependencies upon the system-log-daemon should not be removed (unless they are incorrect on the merits of an individual case). N) None of the above / Further Discussion. ===8<=== Regards, Matthew
Hi, Does this mean that systemd should (accourding to the technical committee) have "Provides: system-log-daemon" as it can serve the purpose of that virtual package? How should this dependency look like? If systemd can fulfill the dependency, but cannot use "Provides: system-log-daemon" (see question above), should other packages use "Recommends: systemd | system-log- daemon"? I think it would be helpful to be explicit on these questions. Ansgar
Hi Matthew,
Thanks for kicking of the drafting.
I like your introduction on many accounts. It does a good compromise on
mentioning much of the relevant context without being very long.
This resolution does not make sense on its own to me, because the most
common case - using journald as your only logging daemon - is not
covered by this resolution. In the IRC meeting we considered augmenting
it with a systemd-journald-is-syslog dummy package that Provides:
system-log-daemon and Depends: systemd-sysv. Is there a reason of not
including this in the resolution? Separately, did we connect with
systemd developers about this package and whether we'd have to overturn
them?
Whilst this resolution sounds reasonable to me. I don't think it can be
understood in a good way without adding more context. As such, I
recommend adding a rationale section to it. I propose the following
text:
Whether logging is available no longer is a boolean. A container
runtime can provide a /dev/log service by forwarding to an external
logging service. Many systems now use journald as their only logging
daemon, but journald can also cooperate with another logging daemon.
Removing dependencies on system-log-daemon should be seen as a
recognition that the availability of a logging daemon can no longer
reasonably be expressed using the dependency system. Additionally,
most log message producers fail gracefully in the absence of a log
consumer by skipping logging.
I think Ansgar's option is worth adding.
D) The Technical Committee recognizes that logging daemons can now
coexist. As a result, logging daemons should stop conflicting with
one another and systemd-sysv should gain Provides:
system-log-daemon. As the original motivation for removing
dependencies on system-log-daemon no longer applies, packages can
and should again issue dependencies on system-log-daemon where
deemed appropriate by their maintainers. The Technical Committee
suggests that Policy be updated to reflect this change.
Do you agree with these proposed changes?
Helmut
Hello, Yes, I think this does need to be in there. That's a lot of additional text :) We should probably keep the options all about the same length?
Hi, Thanks for your comments. I had drafted A) to try and avoid being prescriptive about the adoption of the systemd-journald-is-syslog suggestion, but the result seems to have been too ambiguous. And if Helmut wants an option D, then obviously it should be added. I agree with Sean that adding a large extra paragraph to option B isn't great. I've tried to express the above in a revised draft without producing too much wall-of-text: Regards, Matthew ===8<=== In Bug #1084924, the Technical Committee was asked about a mass bug filing that aimed to remove all dependencies (except Provides: and Conflicts:) upon the system-log-daemon virtual package. Whilst the wording of policy in this area is unclear, the Technical Committee notes that long-standing practice in this area as reflected by policy was that packages could declare appropriate dependencies upon the system-log-daemon virtual package. The Technical Committee also acknowledges that on systemd systems, journald can serve the purpose of system-log-daemon, but that systemd also supports installing a separate system-log-daemon. A) The Technical Committee affirms that it is reasonable for a package to declare any suitable dependency upon the system-log-daemon virtual package. As journald can serve as system-log-daemon either alone or alongside a separate system-log-daemon, this should be expressed in the systemd packaging, by shipping a systemd-journald-is-syslog dummy package or some other suitable mechanism. The Technical Committee suggests that Policy be updated to clarify this, and that maintainers who removed such dependencies as a result of the mass bug filing consider restoring them. B) The Technical Committee notes that logging may be provided by a container runtime, or by journald (by itself or in concert with a separate system-log-daemon), and that it is no longer practical to express the availability or otherwise of a logging daemon via package dependencies. Therefore, the Technical Committee agrees that packages should now only declare Provides: and Conflicts: relationships with the system-log-daemon virtual package. The Technical Committee suggests that Policy be update to reflect this change. C) The Technical Committee resolves that this is a de facto attempt to change Policy, and that the Policy process should be used to consider whether to change Policy relating to system-log-daemon from the status quo of packages being able to declare any reasonable dependency upon system-log-daemon to the state where only Provides: and Conflicts: may be used. Until that process is concluded, dependencies upon the system-log-daemon should not be removed (unless they are incorrect on the merits of an individual case). D) The Technical Committee notes that logging daemons can now co-exist with each other. Therefore, their should stop conflicting with one another, and systemd-sysv should now Provides: system-log-daemon. Given this change, packages canand should again issue dependencies on system-log-daemon where deemed appropriate by their maintainers. The Technical Committee suggests that Policy be updated to reflect this change. N) None of the above / Further Discussion. ===8<===
Matthew Vernon <matthew@debian.org> writes:
^^^^^
they
^^^^^^
can and
Cheers, Phil.
Hi Sean, We have a history of augmenting resolutions with rationales. I see that additional text as a rationale that is appended outside of the resolution in case it passes. For instance in #914897 our rationale is far longer. Whilst we are voting on the "what" (and all of the proposed solutions rightly focus on the "what", thanks Matthew), answering "why" is quite important for gathering acceptance. When I read a resolution, I very much want to have a basic understanding of "why". The split into resolution and rationale allows the reader to decide whether they're interested. I'd also welcome rationale texts for the other proposed resolutions. Helmut
Hello, I call for votes on the below ballot. The vote is open for 7 days, or until the outcome is beyond doubt. Regards, Matthew In Bug #1084924, the Technical Committee was asked about a mass bug filing that aimed to remove all dependencies (except Provides: and Conflicts:) upon the system-log-daemon virtual package. Whilst the wording of policy in this area is unclear, the Technical Committee notes that long-standing practice in this area as reflected by policy was that packages could declare appropriate dependencies upon the system-log-daemon virtual package. The Technical Committee also acknowledges that on systemd systems, journald can serve the purpose of system-log-daemon, but that systemd also supports installing a separate system-log-daemon. A) The Technical Committee affirms that it is reasonable for a package to declare any suitable dependency upon the system-log-daemon virtual package. As journald can serve as system-log-daemon either alone or alongside a separate system-log-daemon, this should be expressed in the systemd packaging, by shipping a systemd-journald-is-syslog dummy package or some other suitable mechanism. The Technical Committee suggests that Policy be updated to clarify this, and that maintainers who removed such dependencies as a result of the mass bug filing consider restoring them. B) The Technical Committee notes that logging may be provided by a container runtime, or by journald (by itself or in concert with a separate system-log-daemon), and that it is no longer practical to express the availability or otherwise of a logging daemon via package dependencies. Therefore, the Technical Committee agrees that packages should now only declare Provides: and Conflicts: relationships with the system-log-daemon virtual package. The Technical Committee suggests that Policy be update to reflect this change. C) The Technical Committee resolves that this is a de facto attempt to change Policy, and that the Policy process should be used to consider whether to change Policy relating to system-log-daemon from the status quo of packages being able to declare any reasonable dependency upon system-log-daemon to the state where only Provides: and Conflicts: may be used. Until that process is concluded, dependencies upon the system-log-daemon should not be removed (unless they are incorrect on the merits of an individual case). D) The Technical Committee notes that logging daemons can now co-exist with each other. Therefore, they should stop conflicting with one another, and systemd-sysv should now Provides: system-log-daemon. Given this change, packages can and should again issue dependencies on system-log-daemon where deemed appropriate by their maintainers. The Technical Committee suggests that Policy be updated to reflect this change. N) None of the above / Further Discussion.
Hi, I Vote A > C > N > B > D Regards, Matthew
Re: Matthew Vernon
I vote
C > B = D > A
Christoph
I vote
B > N > C > A = D
Rationale:
I cannot vote A or D above N, because both of them override systemd
maintainers without - in my view - having seeked their feedback in a
sufficient manner. My understanding is that both A and D require a
3:1 majority under constitution 6.1.4.
I strongly favour B for the reasons given in my earlier mail. No other
proposal simultaneously supports all use cases. Options A and D fail at
serving the container use case (where logging is provided externally).
C is effectively giving up on deciding the matter. I rather see us
taking another round and trying to reach a conclusion (after contacting
systemd maintainers) than punting on the decision. Deferring it to
policy maintainers.
Helmut
Hello,
I vote
C > A > N > B = D
Hi Matthew (2024.12.20_13:31:33_-0400) I vote: B > N > D > A > C I am persuaded by Helmut's rationale for voting A below N. I'm not saying that it's not an acceptable outcome. But, practically, I don't see any thing to be gained by this option winning, without the systemd maintainers following through with packaging changes to support it. And this is overriding them, so it would benefit from engagement with them. D works better with the involvement of the systemd maintainers, without their changes, the typical Debian system would end up with systemd + a non-journal syslog, which isn't optimal default. Thus I vote it below N, too. While system-log-daemon is named as a virtual package in Policy, I don't see that as policy taking any strong ownership of it. Rather I see policy documents the existence of this virtual package, and instructs packages to use it "where appropriate". Thus I vote C last. This leaves B as the only acceptable option. The container case is compelling. I think it's expected for systems to have logging available, and init systemsi without built-in logging should probably depend on system-log-daemon. This is not specified in the option (sorry, I didn't catch this problem at the draft stage), but it's the logical outcome. Stefano
Hi, I don't think this is obviously the case - it's not that the systemd maintainers have done a thing that we require them not to do (or vice versa). Likewise, whilst we haven't canvassed maintainers of every system-log-daemon implementation to check they can be modified to co-exist as D posits, I don't think voting for D is overriding all of those maintainers. Regards, Matthew
Hi,
The vote has run for a week, and the winner is option C:
"""
In Bug #1084924, the Technical Committee was asked about a mass bug
filing that aimed to remove all dependencies (except Provides: and
Conflicts:) upon the system-log-daemon virtual package. Whilst the
wording of policy in this area is unclear, the Technical Committee notes
that long-standing practice in this area as reflected by policy was that
packages could declare appropriate dependencies upon the
system-log-daemon virtual package. The Technical Committee also
acknowledges that on systemd systems, journald can serve the purpose of
system-log-daemon, but that systemd also supports installing a separate
system-log-daemon.
The Technical Committee resolves that this is a de facto attempt to
change Policy, and that the Policy process should be used to consider
whether to change Policy relating to system-log-daemon from the status
quo of packages being able to declare any reasonable dependency upon
system-log-daemon to the state where only Provides: and Conflicts: may
be used. Until that process is concluded, dependencies upon the
system-log-daemon should not be removed (unless they are incorrect on
the merits of an individual case).
"""
We should probably email interested parties in the new year.
Voting machinery output:
Starting results calculation at Fri Dec 27 15:59:31 2024
/--ABCDN
V: 14253 matthew
V: 3212- myon
V: 41342 helmut
V: 24143 spwhitton
V: 41532 stefanor
Option A "Dependencies on s-l-d are OK"
Option B "Provides & Conflicts only"
Option C "Defer to policy"
Option D "s-l-ds should co-exist"
Option N "None of the above"
In the following table, tally[row x][col y] represents the votes that
option x received over option y.
Option
A B C D N
=== === === === ===
Option A 2 2 2 3
Option B 3 2 3 3
Option C 3 3 4 3
Option D 2 0 1 1
Option N 2 2 2 4
Looking at row 2, column 1, B
received 3 votes over A
Looking at row 1, column 2, A
received 2 votes over B.
Option A Reached quorum: 3 >= 2
Option B Reached quorum: 3 >= 2
Option C Reached quorum: 3 >= 2
Dropping Option D "s-l-ds should co-exist" because of Quorum (2 > 1)
Option A passes Majority. 1.500 (3/2) > 1
Option B passes Majority. 1.500 (3/2) > 1
Option C passes Majority. 1.500 (3/2) > 1
Option B defeats Option A by ( 3 - 2) = 1 votes.
Option C defeats Option A by ( 3 - 2) = 1 votes.
Option A defeats Option N by ( 3 - 2) = 1 votes.
Option C defeats Option B by ( 3 - 2) = 1 votes.
Option B defeats Option N by ( 3 - 2) = 1 votes.
Option C defeats Option N by ( 3 - 2) = 1 votes.
The Schwartz Set contains:
Option C "Defer to policy"
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The winners are:
Option C "Defer to policy"
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Regards,
Matthew