Dear maintainer,

I'm running commands that use the `mvn` and capture the output coming from it.

More specifically, I'm catching the version of a Maven project with a command
like the following:

  mvn --batch-mode --quiet -DforceStdout help:evaluate
-Dexpression="project.version"


Up until Debian Bullseye, the output from the command above was a single string
with only the version, as intended. For example: "1.0.0-SNAPSHOT".

After upgrading to Debian Bookworm, the `mvn` command now outputs escape
characters, regardless of the flags passed to it.
The example above has now become: "ESC[0m1.0.0-SNAPSHOTESC[0m".

I tested with the upstream Maven package of the same version, and could narrow
down the problem as coming from the libjansi-java jar.

If I downgrade it back to the Debian-packaged version 2.4.0-1, the old
behaviour is restored.

This change seems to have been due to this commit:
https://salsa.debian.org/java-
team/jansi/-/commit/7f186cd4fc22308d7769db8eeeca26b560b81b1a


You can probably reproduce this very simply with the following:
(running `mvn` anywhere should work - it doesn't have to have a Maven project
and the error message is irrelevant to reproduce what we need)

  mvn --batch-mode --quiet | less


* With libjansi-java 2.4.0-2, escape characters are seen in the output, like
so:

ESC[0m[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
(...)
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/NoGoalSpecifiedException
ESC[0m
(END)



* With libjansi-java 2.4.0-1 (downloaded from snapshot.debian.org), escape
characters are not seen in the output:

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
(...)
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/NoGoalSpecifiedException
(END)


Thank you and best regards,
Luís Picciochi Oliveira

Hi Luís,

Thank you for the bug report.  I don't know the full story of why the
change in behavior between 2.4.0-1 and 2.4.0-2, but I
agree that escape sequences shouldn't be sent when maven is running
in batch mode and that we should address the question of colorized
output referenced in that commit [1] another way.

Emmanuel, do you recall what prompted the change?  I didn't see a bug,
but can propose adding a new switch to generate colorized output even
when a TTY isn't detected.  That will maintain the expected behavior
with --batch-mode, and we can adjust our build tooling to pass the new
switch if we really think we need it.

Thank you,
tony

[1] https://salsa.debian.org/java-team/jansi/-/commit/7f186cd4fc22308d7769db8eeeca26b560b81b1a

I think the issue is that when Maven runs in pbuilder, the TTY isn't
detected and colors get disabled (batch mode isn't used for Debian
builds though). If anything is changed please ensure that building with
pdebuild preserves the colors.

Emmanuel Bourg

Looking at the sources (finally), jansi already supports a mode to
"force" [1] escape sequences, even when the output is not a terminal.  I
propose that we revert the patch and update the Java build tooling to
set the property as desired for our builds.

Once I get this tested and the property configured, I'll upload to
experimental.

Thanks,
tony

[1] https://salsa.debian.org/java-team/jansi/-/blob/738159f052f027fc0816c95ecc38a424f8aa3637/src/main/java/org/fusesource/jansi/AnsiConsole.java#L83-86

Hello,

Bug #1039607 in jansi reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/java-team/jansi/-/commit/64fffcabaeb123ea799e82ce436bf4439595b516

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1039607

We believe that the bug you reported is fixed in the latest version of
jansi, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1039607@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
tony mancill <tmancill@debian.org> (supplier of updated jansi package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Sun, 24 Dec 2023 13:13:03 -0800
Source: jansi
Architecture: source
Version: 2.4.0-3
Distribution: experimental
Urgency: medium
Maintainer: Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>
Changed-By: tony mancill <tmancill@debian.org>
Closes: 1039607
Changes:
 jansi (2.4.0-3) experimental; urgency=medium
 .
   * Team upload.
 .
   [ Debian Janitor ]
   * Apply multi-arch hints. + libjansi-java: Add Multi-Arch: foreign.
 .
   [ tony mancill ]
   * Revert bypass TTY detection patch (Closes: #1039607)
     Colorized output on non-TTYs can be forced with -Djansi.mode=force
     For the mvn wrapper, use MAVEN_OPTS="-Djansi.mode=force" mvn ...
   * Upload to experimental to preserve current colorization behavior
     until other Debian Java build toolchain components are updated to
     force colorized output.
   * Freshen years in debian/copyright
   * Bump Standards-Version to 4.6.2
   * Set Rules-Requires-Root: no in debian/control
   * Add lintian-overrides for Files-Excluded without source repack.
     This can be removed when either 2.4.0+ds or 2.4.1 are uploaded.
Checksums-Sha1:
 ff57302cbdac63213feff7689b61a36223334abb 2047 jansi_2.4.0-3.dsc
 6efe39feaee760484ff51a5121c1a51ce56dacd7 3908 jansi_2.4.0-3.debian.tar.xz
 abf665e26264aab0d194e978da73e9df55694322 14304 jansi_2.4.0-3_amd64.buildinfo
Checksums-Sha256:
 98b0985b7ba105e38d45c7ff30f5bdcf00c8754038ed0d9adce7e81cc4244522 2047 jansi_2.4.0-3.dsc
 b9de9673398f5edbb05770f0c4b9edbd6f7971703de2d0f04d989ddfe15c9aaf 3908 jansi_2.4.0-3.debian.tar.xz
 12cb188275e94f9e3e0b5e1db706f68ce8bc0605d8859b14fb44ed5fba27cfc8 14304 jansi_2.4.0-3_amd64.buildinfo
Files:
 b31db14348d1463073ee23ff5943cb3c 2047 java optional jansi_2.4.0-3.dsc
 97afbf858c05381f8142b17441dc91a6 3908 java optional jansi_2.4.0-3.debian.tar.xz
 55cb5c8891bcbb11a1d362dbe35876ce 14304 java optional jansi_2.4.0-3_amd64.buildinfo
-----BEGIN PGP SIGNATURE-----

iQJIBAEBCgAyFiEE5Qr9Va3SequXFjqLIdIFiZdLPpYFAmWIp8cUHHRtYW5jaWxs
QGRlYmlhbi5vcmcACgkQIdIFiZdLPpYDjxAAi/9v1GJapGsrAB0zLMYZBZHInw3x
DFpHBFekuIgjm72vQr2bWpFxfJkqseQZXazAfazxAa/tF7UPpooJJmJTmNfBLeOO
YmxdcELn6FL1HNY9nefaS1J5EhXCKBd8fgmebKuI6KMXUqebsZKel7iw5YCmns0L
GyCKj1ErDflLROStS0DV9VR0sIfzMV0WQQW11L0IDceyG6MYdc1oamRlwibzK//7
NLNgWTtJr5ozqWlcaMO9pONLeTHXtAI7TEdCPeXGqbGs8Q2dfz/bXa4hzOMiJNhH
zcMlqxlAcbb9C+FHM//9UTsA1yPUetRxxthSnGsf06buGvnVxUjWueIJrnRCYc4f
H9kb+OFsPY0eePdOB2s3vnYVaC1NKPhBXP/WUewThcxKd0rWtjEEE6dcB8sUHFaQ
FzjnUaI/vGN2rWAeETG0OrASkPUyrP/kx0AV7RjjDkSBGEoKFpaHoVV9JXr4qBVa
O48PYGrduh23bzA5WXPb1VzrcjEFAYYrnwFYv04jDFF9ppCYhLyezN0w2lgfAWFf
pHe1KHpiDLTaoMBWTEjWVp0Tpj5Iy/TnWUrHCBhmxV5985+XZ8W8mPeBT/AYryZ7
cs8/m3Yn9F2qFdkl6jUeq1iKjreGlOhiYr8gKnjmWrzn0P507CjjdSSgHyN6pAXz
jjvesJMj8Yfb4oo=
=+R3D
-----END PGP SIGNATURE-----

The following are now available in experimental:

libjansi-java_2.4.0-3_all.deb
maven-debian-helper_2.6.5~exp1_all.deb

The maven-debian-helper change is here [1].  The two packages have been
tested in conjunction to ensure that:

- Maven output is colorized by default when invoked during Debian
  package builds that depend on maven-debian-helper.

- Direct `mvn` Maven output is not colorized by default.

- MAVEN_OPTS, or any other mechanism to set the JVM property jansi.mode,
  can be used to specify the mode and override the default behavior.

I believe this covers the (large) majority of Debian build cases and
restores the upstream behavior, but please let me know if I've missed
something.  The version of gradle in Debian depends on libjansi1-java,
and so wasn't impacted by the patch to jansi (version 2.x).

Assuming there aren't any concerns, I plan to upload maven-debian-helper
and then the jansi package to unstable on January 7th.

Thanks,
tony

[1] https://salsa.debian.org/java-team/maven-debian-helper/-/commit/082b5b639880e8b3b1c1e98f5ae4f649e3b83b67

Good

That's an issue, mvn output should be colorized on a terminal by
default, that's the upstream behaviour.

Emmanuel Bourg

Ah, that's right.  And it's a little odd, because at this point jansi
is no longer patched and our maven packaging doesn't patch for this
behavior.

Thank you for bringing this up.  I will figure out why this is
occurring.  It's likely what led to the jansi patch in the first
place.

After looking a bit closer, it's not so odd after all.  Debian's jansi
(2.x) JAR differs from the upstream version bundled with Maven in that
it doesn't include the jansi-native libraries and doesn't depend on
anything that could provide them.

The Debian jansi-native package is for 1.x and doesn't actually provide
the arch-dependent bits anyway (see [1]), so the default TTY detection
(jansi.mode=default) functionality is broken because all of the native
calls are broken.  Perhaps it's more surprising that jansi.mode=force
or the patch work as well as they do.

This will take a bit to sort out, since we either need to build the
arch-specific packages, which will need to go through NEW, or come up
with a pure-Java implementation that behaves like upstream.  (Not
directly related, but it would be nice to migrate to upstream 2.4.1 [2],
which uses moditect [3], which would need to be packaged.)

My intuition is that it will be better to update the Debian jansi
package to create the necessary arch-any packages so our version more
closely resembles upstream, and we are better positioned if/when jansi
merges into jline, as mentioned here [4].

Thanks,
tony

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=921994
[2] https://github.com/fusesource/jansi/releases/tag/jansi-2.4.1
[3] https://github.com/moditect/moditect
[4] https://github.com/fusesource/jansi/issues/277

Le 01/01/2024 à 00:27, tony mancill a écrit :

moditect can be ignored, it's mostly used to build JPMS module info
files with Java 8. We can add the missing module-info.java files
directly if that's really needed.

Emmanuel Bourg

After giving this some more thought, I don't think we want a jansi
source package that builds arch-any packages.  jansi (already) has a
circular build dependency on itself via maven, but there's no
bootstrapping problem because the packages are currently arch-all.

Since the native library is desired but optional, we could create a
separate source package and add a Recommends dependency to
libjansi-java.  It may even be possible leverage the existing
jansi-native, but I haven't looked into the question of compatibility
yet.

However, for the short-term, I believe we can achieve the desired
behavior and fix the issue for most use cases by

1. Patching the mvn wrapper script in our maven package to set
jansi.mode=force (and thus colorize output) unless the --batch-mode
option is provided.

2. Moving forward with the current jansi package in experimental.

That restores the desired --batch-mode behavior but maintains
colorized output by default and during Debian package builds.

Would that be acceptable?

I am preparing an upload of maven to experimental (version 3.8.7-2~exp1)
that implements the changes described above.  I will push the packaging
changes to the debian/experimental branch in the Salsa repo.  The patch
for mvn can be found here:

https://salsa.debian.org/java-team/maven/-/blob/4501966b3678135deacca45bc1a918380f6dac47/debian/patches/03_jansi_behavior.patch

When used in conjunction with the jansi package in experimental, the
behavior is:

**output is colorized**
mvn

**output is non-colorized**
mvn -B
mvn --batch-mode

The patched mvn script also checks for MAVEN_JANSI_PROPERTY in the
environment, which allows users to override the behavior if desired.
I don't expect this to be needed very often, but it may be useful if
we modify the behavior of jansi in the future, and I wanted there to
be an escape valve if the patched behavior is problematic for some
unforeseen use case.

The invocations below demonstrate the behavior of Debian's jansi that
led to the patch in 2.4.0-2, i.e., that without the native jansi
libraries or -Djansi.mode=force, the default TTY detection fails and
results in non-colorized output.

MAVEN_JANSI_PROPERTY="" mvn
MAVEN_JANSI_PROPERTY= mvn

For completeness:

MAVEN_JANSI_PROPERTY="-Djansi.mode=default" mvn  --> non-colorized
MAVEN_JANSI_PROPERTY="-Djansi.mode=strip" mvn    --> non-colorized
MAVEN_JANSI_PROPERTY="-Djansi.mode=force" mvn    --> colorized

With these changes, we shouldn't need the patched maven-debian-helper I
proposed previously, since the output of Maven invoked during builds
will be colorized.

I believe this will address the issues reported in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1039607 and
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1059147

Feedback welcome.  We can coordinate here before any migrations from
experimental to unstable.

Thanks,
tony

Am 08.01.2024 um 07:03 schrieb tony mancill:

Thank you for the fix. To test we created a chroot environment with the
following upgrades:

apt install libjansi-java/experimental
apt install maven/experimental libmaven3-core-java/experimental libcommons-cli-java/unstable libcommons-lang3-java/unstable

The issue reported in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1059147 as well as our
original issue are resolved.

Thank you for your testing.

Are there any concerns before I upload maven and jansi to unstable?  We
can always revisit the changes if we discover any issues.

I have uploaded maven_3.8.7-2 to unstable.  Once it is in the archive
and made it out to the mirrors, I will upload jansi.

Le 05/01/2024 à 07:22, tony mancill a écrit :

Hi Tony,

Was this plan fully implemented? Maven builds from the command line and
inside pbuilder are no longer colorized currently.

Emmanuel Bourg

Hi Emmanuel,

The uploads of src:maven and src:jansi were completed in mid-February.
I ended up not uploading the change to maven-debian-helper in
experimental because I didn't think it was necessary.

mvn on the command-line produced colorized output by default on my
system and in clean unstable and trixie chroots (attached screenshot).
If that's not working for you, then let me know.  Perhaps more work is
needed on the patch in src:maven.

However, I'm not seeing colorized output on my system during sbuild, so
there are still something amiss.  I notice that "--batch-mode" is being
explicitly passed to the maven invocation via this code [1], which
explains that behavior.

I will have to look into why the tput (no longer?) succeeds during sbuild.
In fact, it's failing for plain `dpkg-buildpackage` in my terminal, even
though the following (negated condition) test generates output:

$ perl -e "if ((`tput colors 2>/dev/null` >= 8)) { print 'foo'; }"

So I think the issue is in maven-debian-helper and that the other
changes are working as expected.

You should be seeing colorized output on the terminal unless you
explicitly override it.  Let me know if you're seeing something else.

Thank you,
tony

[1] https://salsa.debian.org/java-team/maven-debian-helper/-/blob/c03c5403539eb4dc35947b35f51d72d74620637a/share/perl/maven.pm#L53-55

Hello,

is there currently a workaround in Bookworm, before the updated package
arrives there?  I have the same problem as the OP, when extracting the
package version during my build processes.

Best,
Daniel