Hey. I've tried the new packagin, but while all my config and data files are in place, and while the server "runs", there is no logging (neither to stdout/err for systemd nor /var/log/zookeeper .. not even any files get created in there). Connecting via zkCli gives: # /usr/share/zookeeper/bin/zkCli.sh SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Connecting to localhost:2181 Welcome to ZooKeeper! JLine support is disabled SLF4J: Failed to load class "org.slf4j.impl.StaticMDCBinder". SLF4J: Defaulting to no-operation MDCAdapter implementation. SLF4J: See http://www.slf4j.org/codes.html#no_static_mdc_binder for further details. <and there it just hangs> Perhaps some missing dependency? Thanks, Chris.
I should perhaps add, that I have installed the zookeeper packages (zookeeper zookeeperd libzookeeper-java) from testing into stable (bullseye), all other dependencies were already met with bullseye versions. Also, according to https://www.slf4j.org/codes.html#StaticLoggerBinder and there the question "Failed to load class org.slf4j.impl.StaticLoggerBinder"... the files "slf4j-nop.jar slf4j- simple.jar, slf4j-log4j12.jar, slf4j-jdk14.jar or logback-classic.jar" are needed,... but any except for the latter is in place. All are in /usr/share/java. Though manually adding this to the CLASSPATH in /etc/zookeeper/conf/environment doesn't seem to help.
I got a bit further: Setting: CLASSPATH="/etc/zookeeper/conf:/usr/share/java/zookeeper.jar:/usr/share/java/slf4j-simple.jar" i.e. adding the ":/usr/share/java/slf4j-simple.jar" helps a bit... The server seems to start now, and via zkCli, I can `ls` my paths and `get` values. But there's still no logging (neither to file nor to systemd)... ...and zkCli used to show a "prompt" like: [zk: localhost:2181(CONNECTED) 3] commands go here but now, there is nothing, though I can enter commands. May be unrelated though
And here we go:
CLASSPATH="/etc/zookeeper/conf:/usr/share/java/zookeeper.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/log4j-1.2.jar"
Seems to do the trick to get logging to /var/log/zookeeper/foobar .
The zkCli shows still no prompt, though.
It also needs the /usr/share/java/log4j-1.2.jar, /usr/share/java/slf4j-log4j12.jar alone ain't enough.
I wonder a bit whether the classpath is just completely mess up?
When installing, I needed the following dependencies:
[INSTALL, DEPENDENCIES] junit4:amd64 4.13.1-2
[INSTALL, DEPENDENCIES] libactivation-java:amd64 1.2.0-2
[INSTALL, DEPENDENCIES] libapache-pom-java:amd64 18-1
[INSTALL, DEPENDENCIES] libasm-java:amd64 9.1-1
[INSTALL, DEPENDENCIES] libatinject-jsr330-api-java:amd64 1.0+ds1-5
[INSTALL, DEPENDENCIES] libcommons-cli-java:amd64 1.4-2
[INSTALL, DEPENDENCIES] libcommons-io-java:amd64 2.8.0-1
[INSTALL, DEPENDENCIES] libcommons-logging-java:amd64 1.2-2
[INSTALL, DEPENDENCIES] libcommons-parent-java:amd64 43-1
[INSTALL, DEPENDENCIES] libdropwizard-metrics-java:amd64 3.2.6-1
[INSTALL, DEPENDENCIES] libeclipse-jdt-core-java:amd64 3.24.0+eclipse4.18-1
[INSTALL, DEPENDENCIES] libel-api-java:amd64 3.0.0-3
[INSTALL, DEPENDENCIES] libfindbugs-annotations-java:amd64 3.1.0~preview2-3
[INSTALL, DEPENDENCIES] libguava-java:amd64 29.0-6
[INSTALL, DEPENDENCIES] libhamcrest-java:amd64 1.3-9
[INSTALL, DEPENDENCIES] libjackson2-annotations-java:amd64 2.12.1-1
[INSTALL, DEPENDENCIES] libjackson2-core-java:amd64 2.12.1-1
[INSTALL, DEPENDENCIES] libjackson2-databind-java:amd64 2.12.1-1+deb11u1
[INSTALL, DEPENDENCIES] libjaxb-api-java:amd64 2.3.1-1
[INSTALL, DEPENDENCIES] libjctools-java:amd64 2.0.2-1
[INSTALL, DEPENDENCIES] libjetty9-extra-java:amd64 9.4.39-3+deb11u1
[INSTALL, DEPENDENCIES] libjetty9-java:amd64 9.4.39-3+deb11u1
[INSTALL, DEPENDENCIES] libjffi-java:amd64 1.2.7-11
[INSTALL, DEPENDENCIES] libjffi-jni:amd64 1.2.7-11+b1
[INSTALL, DEPENDENCIES] libjnr-constants-java:amd64 0.10.1-1
[INSTALL, DEPENDENCIES] libjnr-enxio-java:amd64 0.32.3-2
[INSTALL, DEPENDENCIES] libjnr-ffi-java:amd64 2.1.7-1
[INSTALL, DEPENDENCIES] libjnr-posix-java:amd64 3.0.45-2
[INSTALL, DEPENDENCIES] libjnr-unixsocket-java:amd64 0.18-4
[INSTALL, DEPENDENCIES] libjnr-x86asm-java:amd64 1.0.2-5.1
[INSTALL, DEPENDENCIES] libjsp-api-java:amd64 2.3.4-3
[INSTALL, DEPENDENCIES] libjsr305-java:amd64 0.1~+svn49-11
[INSTALL, DEPENDENCIES] liblog4j1.2-java:amd64 1.2.17-10+deb11u1
[INSTALL, DEPENDENCIES] libmail-java:amd64 1.6.5-1
[INSTALL, DEPENDENCIES] libnetty-java:amd64 1:4.1.48-4
[INSTALL, DEPENDENCIES] libnetty-tcnative-java:amd64 2.0.28-1
[INSTALL, DEPENDENCIES] libnetty-tcnative-jni:amd64 2.0.28-1
[INSTALL, DEPENDENCIES] libservlet-api-java:amd64 4.0.1-2
[INSTALL, DEPENDENCIES] libservlet3.1-java:amd64 1:4.0.1-2
[INSTALL, DEPENDENCIES] libslf4j-java:amd64 1.7.30-1
[INSTALL, DEPENDENCIES] libsnappy-java:amd64 1.1.8.3-1
[INSTALL, DEPENDENCIES] libsnappy-jni:amd64 1.1.8.3-1
[INSTALL, DEPENDENCIES] libsnappy1v5:amd64 1.1.8-1
[INSTALL, DEPENDENCIES] libspring-beans-java:amd64 4.3.30-1
[INSTALL, DEPENDENCIES] libspring-core-java:amd64 4.3.30-1
[INSTALL, DEPENDENCIES] libtaglibs-standard-impl-java:amd64 1.2.5-2.1
[INSTALL, DEPENDENCIES] libtaglibs-standard-spec-java:amd64 1.2.5-2.1
[INSTALL, DEPENDENCIES] libtomcat9-java:amd64 9.0.43-2~deb11u4
[INSTALL, DEPENDENCIES] libwebsocket-api-java:amd64 1.1-2
[INSTALL, DEPENDENCIES] libzookeeper-java:amd64 3.8.0-10
[INSTALL, DEPENDENCIES] zookeeperd:amd64 3.8.0-10
[INSTALL] zookeeper:amd64 3.8.0-10
Wouldn't there need to be some CLASSPATH settings for more or less all
of them?
I guess there's at least a problem with that:
2022-11-28 21:41:34,760 - WARN [main:AdminServerFactory@58] - Unable to load jetty, not starting JettyAdminServer
java.lang.NoClassDefFoundError: javax/servlet/Servlet
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at org.apache.zookeeper.server.admin.AdminServerFactory.createAdminServer(AdminServerFactory.java:43)
at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:1070)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.getQuorumPeer(QuorumPeerMain.java:246)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:177)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:137)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:91)
Caused by: java.lang.ClassNotFoundException: javax.servlet.Servlet
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 8 more
Thanks,
Chris.
Hello Chris, On Mon, 28 Nov 2022 20:49:22 +0100 Christoph Anton Mitterer <calestyo@scientia.org> wrote: > Package: zookeeper > Version: 3.8.0-10 > Severity: grave > Justification: renders package unusable > > > Hey. > > I've tried the new packagin, but while all my config and data files are in place, > and while the server "runs", there is no logging (neither to stdout/err for systemd > nor /var/log/zookeeper .. not even any files get created in there). > > Connecting via zkCli gives: > # /usr/share/zookeeper/bin/zkCli.sh > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". > SLF4J: Defaulting to no-operation (NOP) logger implementation > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. > Connecting to localhost:2181 > Welcome to ZooKeeper! > JLine support is disabled > SLF4J: Failed to load class "org.slf4j.impl.StaticMDCBinder". > SLF4J: Defaulting to no-operation MDCAdapter implementation. > SLF4J: See http://www.slf4j.org/codes.html#no_static_mdc_binder for further details. > <and there it just hangs> > > > Perhaps some missing dependency? Thanks for the bug report (and the follow-up precisions you sent)! Yet I fail to reproduce it on testing. I installed zookeeper and zookeeperd on testing, then ran $ /usr/share/zookeeper/bin/zkCli.sh specifying nothing about the classpath (it is already handled in the MANIFEST.MF file of the zookeeper jar), and I could enter commands (though with no prompt, as you say). (However, I also get the hanging issue if I do not install zookeeperd.) I guess there is some difference in the versions of the dependencies between stable and testing, as you underline that the dependencies come from stable. Concerning the SLF4J warning: in the past I already stumbled upon this and visited the URI https://www.slf4j.org/codes.html#StaticLoggerBinder which is in the warning message. Because of the last paragraph in the relevant section therein, I was unsure I should choose a SLF4J binding as this would impose my choice to the end user. What do you think? Finally, as: - I think there is a mismatch somewhere in the versions of the dependencies between stable and testing; - I cannot reproduce on a system with dependencies fetched from testing only, I would tend to discuss only the SLF4J issue and ignore the rest. Please tell me what you think, > > > Thanks, > Chris. > > Thanks to you,
Hey Pierre. When you've started your zookeeper (I assume you also use the sysvinit script, respectively the virtual systemd unit generated from that?), what does your process' command line show, e.g.: # ps ax | grep org.apache.zookeeper.server.quorum.QuorumPeerMain 3156 ? Sl 167:08 /usr/bin/java -cp /etc/zookeeper/conf:/usr/share/java/zookeeper.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/log4j-1.2.jar -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=true -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,ROLLINGFILE org.apache.zookeeper.server.quorum.QuorumPeerMain /etc/zookeeper/conf/zoo.cfg Especially, which CLASSPATH (seems to be the -cp)? Is your test system clustered (i.e. more than one ZK node?)? Well I guess that's clear, cause then nothing would have started zookeeper?! That would be quite surprising, IMO,.. cause the issue seems to be quite clearly the missing CLASSPATH stuff. I mean if there would be some other package, that is used to start the java process... and that other package would have a different version and thus be incompatible, I could imagine this. But it seems as if zookeeper's sysvinit script does all on it's own and simply execute java in the end. So it's hard to imagine, that something interferes, and somehow breaks the CLASSPATH there?! Yeah, also stumbled over that via the error message. Well since that two infamous security holes, log4j has a somewhat damaged reputation ;-) ... but apart from that I think this would make the most sense here. As I've said in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1025012#20 : /usr/share/java/slf4j-log4j12.jar wasn't enough for me and I needed to add /usr/share/java/log4j-1.2.jar to the CLASSPATH instead, in order to get output to /var/log/zookeeper I think we should look into both, and at best split things apart. I mean I can try to reproduce the whole thing on a sid or testing only systemd an see whether it still fails to start, but right now I'd be quite surprised why it wouldn't start because of that... and even *if* that would be the reason, then bullseye to bookwork upgrades need to be supported, and therefore the package (zookeeper) would need to have correct versioned dependencies (*if* it's because of a incompatible version of some dependency). Thanks, Chris.
Hello Chris,
Le 06/12/2022 à 23:33, Christoph Anton Mitterer a écrit :
Yes this is the -cp argument.
I got
4558 ? Sl 0:02 /usr/bin/java -cp
/etc/zookeeper/conf:/usr/share/java/zookeeper.jar
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.local.only=true
-Dzookeeper.log.dir=/var/log/zookeeper
-Dzookeeper.root.logger=INFO,ROLLINGFILE
org.apache.zookeeper.server.quorum.QuorumPeerMain
/etc/zookeeper/conf/zoo.cfg
Could you please send the contents of your
/etc/zookeeper/conf/environment? I think this is where the classpath is
set, and we might have caught a wrong bullseye-to-bookworm upgrade case
here.
I humbly admit I cannot answer as I am not a zookeeper user... but if
you tell me where to look I might tell.
Sure :) I was stating this because here is the only way I could get the
same kind of error as yours.
As you suggest (see my request above), we don't have the same classpath
and this might be because you already had zookeeper installed from old
bullseye package and then upgraded. I think /etc/zookeeper/conf contents
is set up by update-alternatives and maybe something is not working well
in our post-install treatments in the case of an upgrade from an
already-installed zookeeper.
[...]
You're right, this makes sense then.
That's right, seeing your classpath contents in the top part of your
last mail makes me think there is something wrong in the upgrade process
and this should be addressed -- let's see this. I would be interested in
your /etc/zookeeper/conf/environment contents, as I said above.
Thanks also to you,
found 3.8.0-11+deb12u1 thanks On Tue, Dec 06, 2022 at 11:33:07PM +0100, Christoph Anton Mitterer wrote: [...] Reproducible with the following: * apt install zookeeperd * systemctl start zookeeper * observe no logs written to /var/log/zookeeper as the default zk conf would like to do The fix for me was indeed to add log4j and its slf4j backend jars to CLASSPATH in /etc/zookeeper/conf/environment HTH, Filippo
Hey. Removing these tags since I think, I give the requested information now and Filippo confirmed the issue. :-) Actually I think the severity should be grave, as it makes the package useless ;-) But that would likely kick it from trixie (unless resolved in time), which I would want either, since it’s rather simple to fix by appending the CLASSPATH as described above. @Pierre, I’ve just seen you’ve requested my /etc/zookeeper/conf/environment ... that slipped through, sorry. I’ve attached it (which is my version with the modifications) to this mail. The diff from the example version shipped with the package in bookworm is: --- environment 2024-06-16 12:34:19.000000000 +0200 +++ /etc/zookeeper/conf/environment 2025-01-12 06:33:54.855932755 +0100 @@ -10,4 +10,4 @@ # line tools such as `zkCli.sh` and not for the actual server JAVA=/usr/bin/java If clustered, you’d need more ZKs running on different hosts, and you’d have a configuration in zoo.conf like: server.1=zk1.example.org:2182:2183;2181 server.2=zk2.example.org:2182:2183;2181 server.3=zk3.example.org:2182:2183;2181 and each of them would have the same config, except for the myid file which would contain the number of the respective server (1, 2, 3) only. Nope, fresh setup. :-) It was really already missing from the example environment file shipped in the package. btw: Not sure whether update-alternatives should be used there. So again, sorry that I’ve somehow overseen your mail. Really not sure why. Thanks, Chris.