Package: java-common
Version: 0.22
Severity: wishlist
Tags: patch

Here is the discussed proposal java policy as txt and also as tar.gz archive,
including all written scripts and manpages

java-config and java-config(1)
java-config-update and java-config-update(1)
java-config-file(5)
findjava, findjava(1), findjavarc(5) and a test script
dh_java (with inline manpage)

As it is a complete rewrite, I don't include the diff of the policy.xml

Changes to tha last discussed version are minor:
* ant-environment: droped include/linux and added a sentence, that the
  JNI Header files should be includeable from there.
* cosmetical errors
* added my name to the author list.

And now the proposed and discussed policy:
-------------------------------------------------------------
Debian policy for Java

Jan Schulz

Ola Lundqvist

Stephane Bortzmeyer

the debian java mailinglist

   Abstract

   This is the java policy for Debian. It begins with a
   background description, continues with the real policy and
   ends with some advices to java packagers.

   The policy covers java virtual machines, an environment to
   compile java programms, java programs and java libraries.
     _________________________________________________________

   Table of Contents
   1. Background
   2. Policy

        2.1. Java virtual machines and runtime environments

              2.1.1. bin/java
              2.1.2. JNI library path

        2.2. Java Development Tools

              2.2.1. Ant Environment
              2.2.2. javac and javadoc tools

        2.3. Java Browser Plugin
        2.4. Classpath
        2.5. Java libraries
        2.6. Java programs
        2.7. Building Java packages
        2.8. Main, contrib or non-free

   3. Advices to Java packagers
     _________________________________________________________

Chapter 1. Background

   There are several "subpolicies" in Debian. They all want to
   make the Debian Policy more precise when it comes to a
   specific subject. See the Emacs subpolicy in package
   emacsen-common for instance. There are other subpolicies for
   programming languages: Perl, Python.

   Feel free to report comments, suggestions and/or disagrements
   to the java-common package (<java-common@packages.debian.org>)
   or the Debian Java mailinglist <debian-java@lists.debian.org>.
   Change requests should be sent as a bug to the java-common
   package.
     _________________________________________________________

Chapter 2. Policy

   Packages written in Java are separated in two categories:
   programs and libraries. Programs are intended to be run by
   end-users. Libraries are intended to help programs to run and
   to be used by developers.

   Both are shipped as Java bytecode (*.class files, packaged in
   a *.jar archive) and with an "Architecture: all" since Java
   bytecode is supposed to be portable. It may additionally be
   shipped as machine code, as produced for example by the GNU
   Compiler for Java, in a separate architecture-specific
   package.

   This policy does not yet address the issue of documentation
   (for instance HTML pages made with javadoc).
     _________________________________________________________

2.1. Java virtual machines and runtime environments

   Debian package managment relies havily on the fact that if you
   install a piece of software, it will work. This can't be
   satisfied by different java virtual machines, even sun derived
   ones, so that all java virtual machines are treated seperatly.
     _________________________________________________________

2.1.1. bin/java

   Packages, which provide a java virtual machine have to setup a
   java-config file (see below) for this virtual machine. The
   java-config file must include the variable declaration for
   JAVA_COMMAND, which has to point to the java virtual machine
   binary/wrapper. Other variables, as stated in the findjava man
   page, may be added, if the java virtual machine can fullfill
   the requirements for this variables.

   A alternative for /usr/bin/java and the corresponding manpage
   may be setup by every package, which provides a java virtual
   machine. The priority should be set to 200. This command must
   not be used by any debian package.

   All java virtual machines must setup a dir structure in
   /usr/lib/name (where name is the name of the java virtual
   machine) with this content: bin/java, which starts the java
   virtual machine. They must set the java.home property to
   /usr/lib/name.

   All java virtual machines must depend on java-common.
     _________________________________________________________

2.1.2. JNI library path

   Some Java classes implement their routines using a "native"
   language (such as C). This native code is compiled and stored
   in dynamic libraries (such as JNI modules) that are loaded at
   runtime. If a java virtual machine supports native code, it
   must include the directory /usr/lib/jni in its search path for
   these dynamic libraries, even if that has to be setup via a
   wrapper scripts.
     _________________________________________________________

2.2. Java Development Tools

   As there is almost no control over different java compilers,
   package should either use /usr/bin/ant to compile and build
   java packages, or directly access the required tools. They
   must not use the alternative managed /usr/bin/javadoc or
   /usr/bin/javac. The ant environement is handled via the
   java-config system (see below).
     _________________________________________________________

2.2.1. Ant Environment

   Packages, which can be used with ant to compile java code must
   setup a directory structure in /usr/lib/name (where name is
   the name of the corresponding java virtual machine (see
   above)), which includes bin/javadoc, which should be of the
   same API version as the virtual maschine, includes with the
   JNI header files includeable from there.

   They also must include a java-config file (see below) which
   includes the variable declaration for JAVA_HOME, which points
   to /usr/lib/name, ANT_BUILD_COMPILER with the short name or
   full qualified classname of the java compiler,
   ANT_BOOTCLASSPATH, which is a JARS-like list of the
   bootclasspath jars and the JARS entry, with the jars needed to
   run this java compiler.

   If the package can't satisfy everything of this requirements
   by themself, it must depend on other packages, which include
   the missing functionality, and setup the directory structure
   accordingly.

   Packages must depend on java-common.
     _________________________________________________________

2.2.2. javac and javadoc tools

   Packages, which include a Java compilers may setup a
   alternative for /usr/bin/javac and its manpage. Packages may
   setup a alternative for /usr/bin/javadoc. In both cases, the
   priority should be 200. Packages must not use this filenames
   to access a java compiler or javadoc.
     _________________________________________________________

2.3. Java Browser Plugin

   If a package provide a Browser plugin, it needs to setup a
   alternative for /usr/lib/mozilla/plugins/javaplugin_oji.so and
   provide java-browser-plugin.
     _________________________________________________________

2.4. Classpath

   To make classpath issues as easy as possible, each package,
   which includes public (library) jar files must add a
   java-config file in /var/share/java-config and place this jars
   in /usr/share/java. The java-config file must be named the
   same as the package, which contains it. Example:
   libant1.5-java includes the java-config file
   /var/share/java-config/libant1.5-java

   If more than one package includes certain functionality/API,
   this packages should use a agreed upon virtual package name
   and the alternative system to handle the java-config file with
   that name.

   A java-config file is a sh compatible file, which only sets
   variables. In the case of a library package, it must set JARS
   and DEPENDS, even if they are empty. It may also add CONTRIB.
   ANT_BUILD_COMPILER, ANT_BOOTCLASSPATH and every variable
   beginning with "JAVA_" are reserved. Other variables may be
   used privatly.

   The content of a java-config file is as follows:

     * JARS must be set with all public jar files, seperated by
       ':'
       JARS="/usr/share/java/first.jar:/usr/share/java/second.jar
       "
     * DEPENDS must be a space or colon seperated list of
       packages, which this jar needs to have on the classpath at
       runtime. DEPENDS="otherpackage-java libant1.5-java"
     * CONTRIB is a likewise list o packages, to which classpath
       this jars should be added (plugin system). Example: a
       package adds a task to ant. This package would then set
       CONTRIB="libant1.5-java".
     * Packages, which have contributed their jars to other
       packages need to call the appropiated update-* script in
       the postrm (in case of removal) and postinst (update and
       install) scripts.
     _________________________________________________________

2.5. Java libraries

   Libraries are not separated between developers (-dev) and
   users versions, since this is meaningless in Java.

   Java libraries packages must be named libXXX[API version]-java
   (without the brackets), where the version part is optional and
   should only contain the necessary part. The version part
   should only be used to avoid naming colisions. The API version
   refers to the version of the public API of that package. The
   XXX part is the actual package name used in the text below.

   Their classes must be in jar archive(s) in the directory
   /usr/share/java, with the name packagename[API
   version][-extraname].jar. The extraname is optional and used
   internaly within the package to separate the different jars
   provided by the package.

   A package must depend on the disjunction of all JVMs with
   which it has been tested succesfully.

   This applies only to libraries, not to the core classes
   provied by the runtime environments.

   Some Java libraries rely on code written in a "native"
   language, such as JNI (Java Native Interface) code. This
   native code is compiled into separate dynamic libraries which
   are loaded by the Java virtual machine at runtime.

   If a Java library relies on native code, the dynamic libraries
   containing this compiled native code should be installed into
   the directory /usr/lib/jni. These dynamic libraries should be
   shipped in a separate architecture-specific package named
   libXXX[version]-jni. The package containing the Java bytecode
   (generally libXXX[version]-java) should depend on this
   package.

   There may be situations, such as with very small packages,
   where it is better to bundle the Java code and the native code
   together into a single package. Such packages should be
   architecture-specific and follow the usual
   libXXX[version]-java naming convention.
     _________________________________________________________

2.6. Java programs

   Programs must have executable(s) in /usr/bin and be
   executable. They must run without specific environment
   variables (see Policy 10.9), for instance CLASSPATH. They must
   respect the Policy rules for executables (for instance a
   manual page per executable, see Policy 13.1).

   Packages, which need to find a java virtual machine in the
   startscript of their programm must use the /usr/bin/findjava
   programm for this task.

   If you need jars, which are not in the same package as
   programm, the /usr/bin/java-config programm should be used to
   setup the classpath.

   If programms have their own auxiliary classes, they may be in
   a jar file in /usr/share/java. The name of the jar should then
   follow the same naming conventions as for libraries. Packages
   should not have public (library) jars and private jars
   together in one package. Instead, the package should be split
   into the reusable library package and a application package.
   Java programms may use the java-config file system to handle
   this part of the classpath. In case of programms, which can be
   enchanced by plugins, they must use the java-config file
   system.

   A package must depend on the disjunction of all JVMs with
   which it has been tested succesfully.

   Applications may honor the user set JAVA_HOME evironment
   variable. This should be clearly stated in the manpage and may
   state it at runtime. The programm does not need to make sanity
   checks, whether this java virtual maschine will work or not.

   Application, which allow to pre-set or overwrite the CLASSPATH
   should state this in the manpage and may state it at runtime.

   There is no naming rules for programs, they are ordinary
   programs, from the user point of view.
     _________________________________________________________

2.7. Building Java packages

   If a package uses ant to build a package it must build depend
   on the required ant environments and use /usr/bin/java-config
   to access this ant build environment.

   If a package doesn't use ant to build the package, it must
   build depend on a specific package for each required tool and
   call this tools directly.
     _________________________________________________________

2.8. Main, contrib or non-free

   About politics: packaging Java stuff changes nothing to the
   rules Debian uses to find if a program is free or not. Keep in
   mind the following:

     * If your source package can compile (correctly) only with
       non-free tools, it cannot go to main. If your package
       itself is free, it must go to contrib.
     * If your binary package can run (correctly) only with
       non-free java virtual machine it cannot go to main. If
       your package itself is free, it must go to contrib.
     _________________________________________________________

Chapter 3. Advices to Java packagers

   Warning: These are just advices, they are not part of the
   policy.

     * Be sure to manage all build and runtime dependencies by
       hand in debian/control. dh_java makes this task a little
       easier. It can also setup the java-config file. The CDBS
       includes helper classes to handle ant based sources.
     * You can suppress many calls in debian/rules which are
       meaningless for Java, like dh_strip and dh_shlibdeps.
     * Source package handling is painful, since most Java
       upstream programs come with .class files. I suggest to
       make a new .orig tarball after cleaning them, otherwise,
       dpkg-source will complain.
     * Java properties files are probably better under /etc and
       flagged as configuration files (this will be integrated in
       the policy, one day).
-------------------------------------------------------------

Enjoy, Jan

Jan Schulz wrote:

As I've already mentioned in
http://lists.debian.org/debian-java/2003/debian-java-200309/msg00124.html
you (or anybody else) can't replace the whole Java Policy. You need to
submit individual proposals for the individual changes (e.g. the naming
of JARs in /usr/share/java or the use of your tools) and you need people
(strictly speaking Debian developers) who second your proposals. Just
because nobody complained when you posted your last proposal does not
mean that everybody agrees. I have not seen anybody who has seconded
your proposal (maybe I missed a few mails?).

The process for changes to the the Java Policy is not different than the
process for the Debian Policy
(/usr/share/doc/debian-policy/policy-process.txt.gz in the debian-policy
package). Please remember, we don't have the constitutional power to
force other Debian developers to implement the specification. Changes
can only be made if (most of) the people that have to implement it
agree. The situation is even worse since the Blackdown package are not
part of Debian at all, so they are not forced to follow the Java Policy.

I can include these in the java-common package but I will not replace
the Java Policy. These scripts need to prove their usability first
before they can be made mandatory.

Stefan

retitle 212863 [PROPSAL] New java policy including tools to manage the changes
thanks

Hallo Stefan,

* Stefan Gybas wrote:

I can't see, why not?

I think that breaking this proposal into small changes doesn't make
sense. More or less, it tries to change the way in which java
packagess are made.

I understand, that the main debian policy is a a document, which
enforces 'common practise'. IMO, the debian java policy is different,
as it is a document, which describes some rules, which are only
usefull, if all packages agree on them (like the virtual packages now
or the scipts in the proposal) and implement them.

It's not a ammendment but a replacement, as IMO the old policy does
not work properly. It also tries to fill most of the holes, which the
old policy left open (see the 'quetions' section).

I don't expect to have this changes implemented before sarge is
released.

Yes, thats why I sent this bugreport. Sorry, that it wasn't titled
properly.

That's exactly, why I'm sending this now as a PROPOSAL to the
java-common pakage: to get the feedback if this changes are acceptable.

[scripts]

Yes, I understood that. They are not meant as the 'last word', but I
also wanted them to get a broader audiance.

I will add wishlist bugs to the JVM packages and try to get a patch
into mpkg-j2sdk, so that they can be started to use in packaging
scripts and bootstrap scripts.

Thanks for the feedback!

Jan

I still don't like the findjava idea. What is the goal? It looks like
this script provides a common interface to all of the java execution
systems (compilers, JITs, interpreters or otherwise) by concentrating
shell script adapters into a single file. I think it is much more
maintainable to define the calling conventions and then require each
system (in the language of its choice) to provide a "java" file that
provides the common calling convention.

With the findjava script it looks like I would need to submit a patch
anytime Kaffe's command line conventions changed rather than just fixing
an adaptor that resides in my own package.

In message:  <1065555500.32697.11.camel@sarge>
             Ean Schuessler <ean@brainfood.com> writes:

I think there are two goals:

1) Standardize the invocation interface, so that it is feasible to have
   java packages that will have a hope of running on a VM that the
   packager did not directly support.

2) Localize the invocation interface, so that the hunt-and-find-a-good-VM
   logic isn't replicated in all the myriad java packages.

This is all to make it easier on the packagers of end-user java programs,
not to make it easier for the VM, compiler, etc packagers.

I think that you are right in your assessment of needing to submit a
patch to findjava whenever Kaffe's command line changes, and I agree that
this is suboptimal.  I also think it would be good to define a standard
interface that packages like Kaffe could support, but I do not think that
is sufficient, because we cannot hope to get Sun (or any of the other
proprietary vendors) to conform to that standard.  For them, we need
something external like findjava.  (Simply ignoring Sun and the other
non-free java implementations is not really an option... that'll just
alienate our users who need some feature that hasn't yet made it into
one of the free VMs.)

If we both define an standard interface for the nicely packaged
implentations, and make findjava use that interface when possible
(but still have special-case foo for the unpackageables like Sun's),
then I think that we'll come as close as possible to the best of both
worlds.

- Alex

Hello T.,

Tuesday, October 7, 2003, 10:41:52 PM, you wrote:

The discussion has shown, that we can't standardisize this much more
than which is already done. This should have been done by sun or any
other body and I don't think that thjis is possible in a debian
policy.

I hope that all java binaries will implement the basic things like
name <vm-specific-params> -classpath <jarfiles, seperated by ':'>
mainclass <app options>
or react on a set CLASSPATH

This seems to be the case with all tested JVS (and using sablevm
wrapper). Everything else seems to be madness to try, because
everybody has a differenet opinion on where to stop and what would
be better.

Yes.

Yes.

No. Findjava will output the things, which kaffe specifies in its
/usr/share/java-config/kaffe file, nothing more. If a application
package chooses to play around with other options, it has to
implement the logic for this, based on the findjava output, and hope
that kaffe won't break it with the next version.

Thats exactly the point. With the findjava implementation it's easy
to do something like

# tests, if sablevm is used directly and not the wrapper.
isSableVM(){
if [ "$1" = /usr/bin/sablevm ] ; then return 0 ; fi return 1;
}
PACKAGES="..."
JAVACMD="$(findjava ...)"
if isSableVM $JAVACMD ; then
   CLASSPATH_OPTION="--classpath"
else
   CLASSPATH_OPTION="-classpath"
fi
$JAVACMD $CLASSPATH_OPTION `java-config $PACKAGES` com.example.Main

Usually such logic won't be nessesary. The main usecase is looking
like this:

# package, which jars need to be on the classpath
DEPEND_LIST="package1 package2 package3"
# Known working java binaries, which can run this package app.
WORKING_JAVA="kaffe gij sun"
export CLASSPATH="$(java-config --all $DEPEND_LIST):app-main.jar"
JAVACMD="$(findjava $WORKING_JAVA)"
$JAVACMD com.example.Main

Thanks for the comments!

Jan

Hello Ean,

Tuesday, October 7, 2003, 9:38:20 PM, you wrote:
discussion in debian-java has shown, that the alternative machnism
isn't enough and especial isn't reliable.

findjava will be called with the list of the 'known working' java
implementations and will either return one of them, which is
installed or exit with an error code. This will ensure that the
called java command is known to be working with your package. The
basic equivalent of this code is

for java in /usr/bin/kaffe /usr/lib/j2sdk1.4/bin/java ... ; do
   if [ -x $java ] ; then
      JAVACMD="$java"
      break
   fi
done

Instead you do
JAVACMD="$(findjava kaffe sun-java-1.4 ...)"

So the difference is:
* The code is in one place -> less bugs
* Its abstracted by packaging names and not places, where the binary
  is -> palces can change...
* the java binary automatically get some parameter, which the java
  binary maintainer things usefull.
* you can explicitly ask for a special kind of java binary (one,
  which is able to run in server mode), without putting much more
  logic into the startscript.

It only provide an 'search mechanism' to the binary, which can
execute java byte code.

This is basicly, what the findjava mechnism does.

Nope, you will change the java-config file in the kaffe package,
which specifies the java command. See the manpage of
java-config-file(5) in the below package.

You, as the maintainer of kaffe, will have the right and power to
change, which kaffe command is called and with which parameter (in
the basic run). You can also specify, how the kaffe binary should be
started if the app should be run in server or client mode (if you
know more interesting cases, I will implement them).

Please have a look at the package available from
deb http://www.katzien.de/debian/java ./
-> new-java-policy package
It includes java-config files examples (this files would be provided
by the package containing the jars/java binary and not one single
package) and the findjava sript and manpage. Please have a look at
it and try it.

Thanks for the comments!

Jan

I still don't understand what this achieves that alternatives do not.
There is nothing particularly special about Java that requires a more
elaborate alternatives mechanism than any other interpreter. If the
wrapper script for each VM does its job properly then the classpath
should get set to what it needs to be and the VM will be invoked with
all the proper conventions.

It would seem to me that findjava will simply invoke whichever VM it
finds first in its list of VMs and that will be that. It loses the
priority mechanism of the alternatives scheme and doesn't really add
that much that cannot be done with proper wrapper scripts for each VM.

I may need to go back and read the earlier discussions more carefully
but I never saw how the findjava script adds anything that cannot be
achieved using the usual (and up til now sufficient) mechanisms that
Debian already provides.

Sorry for the continued stubborn-headedness.

Hello Ean,

Wednesday, October 8, 2003, 6:22:49 AM, you wrote:

Imagine this situation:

Application needs a special feature, which is implemented in about
half the java alternatives. /usr/bin/java will be set by all java
binaries and in this case, /usr/bin/java is set by a package, which
does not implement the requirements. There is a second installed
java package, which does implement it.

-> apt will see all dependencies fullfiled, app will call
/usr/bin/java and will not run.

The workaround would be to follow the symlinks, but that would be
the working *around* teh alternative system *and* you would end up
with the 'findjava' code in every starter script (as the fallback,
when /usr/bin/java isn't working).

You get a similar 'feature' with the /etc/findjavarc and
'PREFERRED_...' variable.

IMO there is. See the discussion in debian-java, where this was
turned over and over again.

This isn't the problem at all. The problem is that some java VMs
will be able to start something and others will not.

Please read the manpage of findjava and how it is used. It does not
invoke anything, it just outputs the 'java command' to stdout. It
garantees, that this outputed command is available on this system.
If you need more than this, you can still do some magic based on
this.

The priorites were also discusssed and the 'would be' consensus was,
that we can't assign different priorities without having a big row.
based on what facts will you assign priorities? And you could still
end up in a situation, where a lower priority JVM will run a app,
but a hiher priority JVM will not.

IMO for the basic function (starting a app with a classpath and a
main class) it's already now enough. For more, you cannot specifiy it
*enough* to satisfy every situation (like: enable the jitter with
what option? Enable what GC with what option? What subset would you
implement?) and IMO SUN should do this and not we.

IMO the today mechanism is *not* sufficient. Have a look at the
starterscripts of all big java papps how they find the java binary:
all uses JAVA_HOME for it and /usr/bin/java as a fallback.

Please read the script and what it does. Also there is a lot of
mails about the 'alternatives are enough to find a java binary'
discussion during the second edition of the proposal.

Jan

Hi all,

There seems to be some misunderstanding going on. Having not taken
sides, I thought I would try to help clarify the positions.

1) Necessity for findjava

I think Jan explained this well in his last message. Package A might
work with kaffe or gij but not sablevm, while package B works with gij
or sablevm but not kaffe. Alternatives cannot handle this situation.
Findjava is the tool that allow the startup scripts of A and B to
declare this fact, while being abstract over binary locations.
Furthermore, one can invent more abstract tags to pass to finjava, like
'awt', 'server' or whatever.

2) Independence of the JVM packages

That's Ean's point. Each JVM has its own command line arguments, so code
is needed to translate a standardized set of command line arguments to
the JVM own format. We should not put this translation code in a common
package, because then each maintainer will need to update it from time
to time, and it will be a mess.
This is already done by the wrapper scripts. So findjava should not
return the 'raw' binary, but the wrapper scripts (ie
/usr/bin/gij-wrapper, not /usr/bin/gij).
One thing needed is to put such a (minimal) standard of command-line
arguments in the Java policy, so JVM packagers not what to implement.
I'm not sure if we got there yet. Ean wrote:
It's true that we cannot handle every option. But we can handle them
progressively as the need appears. Whether or not Sun should do it,
nothing prevents us from making our own standard if needed (ie if "Sun
compatible" is not enough). Remember, we are not dictating the world a
format, just making some decision internal to Debian about the API
between JVM callers and JVMs. For Sun JVM, the mpkg-j2sdk could install
our own wrapper if necessary.


I hope I represented well the opinions (obviously, complain if not) and
that will help understanding the situation.

Cheers,

Daniel

Hello Daniel,

Wednesday, October 8, 2003, 12:11:24 PM, you wrote:

Thanks for summarizing it up!

IMO, and after having alook at sablevm, kaffe and gij during teh
discussion: all of them are compatible enough to be used in the
normal sense:
export CLASSPATH
-> will result in a classpath, setup from basic classes and the
   added ones.
JAVACMD Mainclass options

IMO the only switch, which needs additional specification would be
'-classpath' to include the runtime environment by default. I
remember kaffe not setting it, but I think Ean made sure it is
included during the 1.1 update.

findjava will return, what is specified. I really hope that gij and
sablevm will add a java-config file for their wqrappers and not for
the main app.

IMO we are there now. But lets see what other people thing what
needs to be included in this list. The internal (GC, jit and so on)
are handled by findjava, debuging options are better handled on a
per JVM basis as well and this should probably be done via either a
findjava switch or a via using 'OVERWRITE_VM_COMMAND' (or whatever
it was...), so you exactly know which VM is called. So the only
uscase is starting a app and there it's enough to use CLASSPATH and
the main class.
Nope, me wrote that :)

Yes, also true, but IMO just *now* there isn't any other situation,
which isn't handled by teh wrapper scripts and all other commands.
  ^^^^^^^^^^
:) Sun isn't compatible to each release, which I learned during the
discussion and why I had to stop using teh alternative system
altogether...

True, but very confusing...

I think you summariesed it really well! Thanks!

Jan

Ok, I begin to see the motivation. Findjava overrides (replaces) the
alternatives mechanism because the proper "alternative" for a given java
app may vary from application to application. Certain apps that know
they can run on certain free VMs may take advantage of certain specific
features and so forth and therefore want a common way to search for
those apps locations.

This seems like a reasonable motivation. However, my concern would be
that these tunings can become so app <--> vm specific that findjava does
not really support their needs or contains elaborate tunings specific to
a single app/vm relationship. It seems more straightforward to me to
have the "for java in kaffe, sable, foo, bar" logic reside in the
kick-off script for the actual app that knows it wants to do special
free VM oriented things.

For instance, if the Tomcat maintainer decides that compiling certain
baseline classes with GCJ before running the main system with GIJ is a
good idea then I can't see that findjava will elegantly accomodate that.
The idea itself may be sound but probably doesn't belong in a common
package and actually probably belongs in something like
tomcat-gcj-booster.deb or somesuch.

All things considered, I think that my preference is to have each VM
provide some Debian-specific tightened up version of $JAVA_HOME that we
specify in java-policy and then have $JAVA_HOME be managed by
alternatives. Therefore, /usr/lib/javahome will be an alternative that
points into a set of directories provided by some VM that looks and
feels like a Sun style JAVA_HOME. That's just me but I think its the
sanest option that will get the most things running the quickest.

Yes. That is where I'm at.

Nope, not me.

Ean Schuessler wrote:

Turn 'proper "alternative"' into 'working "alternative"' and you've got
the original motivation ;)

the way I understood it, it has little to do with specific tunings, and
more with giving the application packager a simple, common way to tell
the user which VMs will work with the package, while at the same time
giving the user some choice, and allowing him to overwrite the choice
the packager made.

For example, it seems to be impossible for a non-root user, to overwrite
the java alternative, whereas the proposed scheme allows the user to
specify a different (maybe even working ;) jvm that he one that comes up
on top of the alternative system. Given that currently some applications
may run with some VMs, but not with others, and not on all platforms,
findjava seems like a good compromise to me, which takes the idea of
alternatives system, and extends it to be more flexible to be able to
cope with current VM situation, i.e. debug everywhere ;)

Wouldn't specifying gij as the only VM in the findjava setup file that
can run the gcj-ed tomcat be enough? You certainly wouldn't want to
recompile tomcat with gcj on every invocation? ;)

The trouble with JAVA_HOME is that even if debian specifies it, hardly
any java application developer would be bothered to follow debian's
specification, instead of whatever ad-hoc pseudo-standard de-jour
JAVA_HOME seems to ben in whatever the current JVM release on the
developer's target/developement platform is. From my experience with
dealing with open source java developers, to most of them JAVA_HOME is
like crack: once they get hooked on accessing sun's internals directly,
it's very hard to get them off the quick fix.

While kaffe now follows a more jre like file layout, some things will
fail nevertheless (like code trying to access tools.jar in order to use
Sun's javac, a classical ant, or tomcat fallacy). Code that relies on
JAVA_HOME is VM dependant by design, and runs on free VMs by pure
chance. That it often runs nevertheless is mostly the fruit of hard work
trying to find ways to work around, ahem, sloppy programming ;)

So I'm quite opposed to debian trying to standardise/faciliate something
that is bad practice. Instead, it would be, in my  opinion, better to
teach the open source java developers to avoid using Sun's internals in
their code.

Both debian-java-home and findjava are attempts to deal with the same
problem. But debian-java-home doesn't do much good: it adds some more
burden on package maintainers, giving developers some hope that their
apps may run on the VMs, if they use JAVA_HOME at all. But it's not
giving users the security that a particular java application they
installed will actually run on their system.

Findjava, on the other hand, doesn't care about java home at all. It is
just a mechanism for users and application packagers to come together on
a VM-application combination that is supposed to work. If the app uses
JAVA_HOME and a VM provides enough of it for an app to run, then fine,
the applicatin packager can add the VM to the list of VMs his
application should work on. But assuming that just because a VM provides
a debian specific JAVA_HOME directory structure some JAVA_HOME using app
will run, is wrong, in my experience ;)

Could the findjava script then be split apart into VM specific 'plugins'
to do the work? then the VM package maintainers could independently
update 'findjava-kaffe' from 'findjava-sablevm', while keeping the
calling interface for the main 'findjava' script.

cheers,
dalibor topic

export JAVA_HOME=/usr/lib/kaffe (not so hard)

These issues are orthogonal. The directory structure of a VM that
launches an app has no impact on whether it can call Sun internal code
or not. It comes down to what the ClassLoaders will find.

A noble ambition but I don't see us having much influence on the general
community of Java authors out there. The JAVA_HOME convention is a
reality, crappy as it may be, and many Java applications attempt to use
it. Requiring Debian Java VMs to present some semblance of the JAVA_HOME
"standard" is going to make more apps run on more VMs for Debian.
Dealing with the use of Sun internals is a border case that needs to be
addressed seperately for those apps that do so.
to specify their own preference for a VM to run against and I'm not
against centralizing the logic that finds that VM.

Maybe something like:

/etc/findjava/kaffe:
  JAVA_HOME=/usr/lib/kaffe
  JAVA_BIN=/usr/lib/kaffe/bin/java
  BASE_CLASSES=$JAVA_HOME/jre/lib/rt.jar
  MX=-mx
  COMMAND_LINE_CLASSPATH=-classpath=$BASE_CLASSES:$CLASSPATH
  ..etc.. or something like that...

or something... but then kaffe would provide /etc/findjava/kaffe as part
of its package and findjava would pick it up and invocation time.

Hello Dalibor,

Replying here, becasue I haven't seen the post from Ean.

Thursday, October 9, 2003, 7:17:19 PM, you wrote:

Yes, the tuning is something you get for free. I can't really
imagine tuning a java package in a way, that it becomes so VM
specific, that a 'findjava' mechnism does not anymore work. I think,
if that happens, than it's time to Depends: and use only this
'one-and-only' VM. Or use different starter classes in different
packages.

On the other hand, I will try to extent the findjava command with
all interesting options you might find valuable to add. As I already
said, I can imagine a '--mem128' command to allocate 128MB of
memory (would be mapped by the sun VMs to '-Xm128m' IIRC).
*if* possible. And there is a 'overwrite' mechanism, which is
described as 'testing only' in the manpage.

This would result in a native tomcat, which would not use a 'java
bytecode interpreter', but would be a normal app under normal debian
policy. And not under the debian policy for java "bytecode" apps.

No, that would result in errors, when the user specifys a 'overwrite
VM'. Compiled to native *must* AFAIK run with gcj and not with any
other java VM.
certain internal things via a variable in the java-config-file,
which is in turn read, when findjava is called with certain options.
For example, this here might end up in a JVM, optimised for server
use and allocate 128MB of Heap [please note: the mem thing isn't
yet implemented!]:

findjava --server --mem128 ibm-java-1.4 sun-java-1.4 kaffe

It will try to find the best VM available and try to optimise it.
The java-config-file for sun-java-1.4 might look like this:

JAVACMD="/usr/lib/j2sdk1.4/bin/java"
SERVER="-server"
CLIENT="-client"
MEM128="-Xm128m" # don't remember what the right options was...
MEM256="-Xm256m"

Resulting in a outputted command like
/usr/lib/j2sdk1.4/bin/java -server -Xm128m

Dalibor might add the kaffe equivalent...

Jan

Hi Ean,

thanks for your quick reply!

Ean Schuessler wrote:

Which only works for those apps that use JAVA_HOME to find the java
executable to run themselves. Not for the others.

I've gone over this with Jan, and it seems that most apps use JAVA_HOME
for two things:

a) access Sun internal code, where having a JAVA_HOME layout doesn't
help the free VMs run that code anyway.

b) running with a VM that's not in $PATH. Which is what findjava is made
for ;)

c) avoiding free VMs like kaffe: See
http://lists.codehaus.org/pipermail/jcontainer-commits/2003-July/000399.html
for example:
# Checking for JAVA_HOME is required on *nix due
# to some distributions stupidly including kaffe in /usr/bin
there providing a JAVA_HOME doesn't help either.

d) to work around Cygwin's unix paths vs. Windows paths issues:
http://koala.ilog.fr/batik/mlists/batik-dev/archives/msg03231.html
there providing JAVA_HOME doesn't make sense for debian either.

Is there any other use of JAVA_HOME you've seen? Where there is
something gained by using JAVA_HOME that a) can not be provided by
findjava and at the same time b) is portable through java runtimes?

I believe that's wishful thinking. The JAVA_HOME variable could be set
by findjava script as well, if it would matter that much. But from my
experience of wrestling with poorly programmed java applications, most
of the stuff uses JAVA_HOME for trivial things, like finding the java
binary, or utterly non-portable mess ;)

Yep. A pox on their house and all that! ;)

Maybe it would be better to maintain the findjava-vm-binding as a
separate package, so that bugs in one don't force exclusion of the other
from testing?

cheers,
dalibor topic

Hello Ean,

Thursday, October 9, 2003, 7:33:22 AM, you wrote:

True. And this package will depend *only' on gcj and will not touch
any java related things like jar files or java bytecode
interpreeter. This things will be handled --more or less-- under
normal debian policy.
JAVA_HOME is good for three things:
* ant, which depends on the java property java.home set and some
  apps in java.home/bin/*
* other apps, which rely on internal things in java.home
* uses '$JAVA_HOME/bin/java to start the app

The first is done in the 'ant-environment', which specifies a 'kind
of' java.home, but the only requirement is 'it runs ant'.

The second can't properly be helped in any other way than
'Depends: <only sun dereived JVMs>'

The third can be trivaly helped by patching the startup script,
which would probably anyway be needed bvecause of 'free VMs', which
are surely not considered (think: internal options) in this
startscripts.

Did I miss something?

Anyway, even if we specify a 'JAVA_HOME' layout, it will not help us
with the alternative system. The problem with alternactives are
quite different from why a JAVA_HOME layout would be usefull or not.
No matter we are using '/usr/bin/java' or
'/usr/lib/java-home[/bin/java]', the same problems will show up. We
would then need a 'findjavahome' script.

Jan

They often use JAVA_HOME to find the executable, the base class
libraries, the compiler and jar. Of course, that presents another set of
issues with regard to alternatives. If JAVA_HOME points at
/usr/lib/kaffe then how can javac = jikes and so forth.

I'm not saying that JAVA_HOME kicks ass or is even sane, it is just a
widely used convention.

Other than what I said earlier, no.

It could, if all Debian VMs provide a JAVA_HOME-like structure.

Sensible enough.

Hello Ean,

Thursday, October 9, 2003, 10:26:10 PM, you wrote:

The base classes shouldn't matter apart from ant  bootclasspath,
which is already included in the ant-environment.
implementation based on zip classes and the compiler can be
specified via short names or classfiles.

Have a look at the 'ant-environment' in teh proposed policy.

Yes, and all usecases, which are interesting for debian are IMO done
in slighly different, but much more saner ways with the new policy.

No. /usr/bin/java is expected to be a JVM, not something,w hich
outputs a VM command. USer will use it for things like 'my first
java app' and so on.
be clear, that this *is* *already* *done* via the java-config-files.

It is not as flexible as a
'execjava <_*lots*_ of options> mainClass' with different
implemntations, but I can't think of a situation, where so much
flexibility is needed. If you look at the java apps in debian, none
of them does any JVM specific tuning. findjava is flexible as well,
up to a point and I think that this point is *far* above any need
of a package.

All of you: please read the proposed policy and the scripts, which
are mentiond there and also the helper scripts. There is no need to
hit each otheres over the head with additional requirements, if much
of the work is are already done. :)

They are easily getabale by
deb http://www.katzien.de/debian/java ./
-> apt-get install new-java-policy

Proposed policy:
zless /usr/share/doc/new-java-policy/policy.txt.gz

Manpages:
java-config(1), java-config-file(5), findjava(1), findjavarc(5),
dh_java(1), update-java-config(1)

And try the scripts:
findjava [--server|--client|] kaffe sun-java-1.4 sablevm gij
java-config [--all | --contrib |--classpath] $PACKAGE
(where $PACKAGE is one of `ls /usr/share/java-config/`
dh_java with a debian-dir with package.java and package.jars (see
manpage)

For the ant-environment, see the java-config-files of kaffe and
sun-java-1.4 in /usr/share/java-config. I hope once this is common,
I can persuade Stephan to include such things in the Common Debian
Build System, so that they can be used like with the 'findjava'
script.

Jan

Hello,

Thursday, October 9, 2003, 11:43:00 PM, I wrote:

s/Stephan/Stefan/

Sorry about that...

Jan

Hallo,

There seems to be no more questions, but unfortunatelly nonone has
said something like 'do it' or 'file bugs with patches' or 'go
testing' or whatever until now.

I'm a little scared to go on, and ask Stefan for the temporary
inclusion of the findjava and other scripts in java-common or the VM
and package maintainer to include java-config-files, without such
comments..

Also, I would like to start working on a patch for mpkg-j2sdk to
include all the changes and to make the new policy bits working with
the new policy.

Aparat from the naming convention for the jar files, I can't see any
'conflicts' when having both policy implemented (and that's no big
deal...). That's why I would like to take the same steps like the
recent "resolvconf transition", and supply patches, which supports
both versions of policy. At first I will get in changes for runtime
matters and finish with the 'ant-environment' changes at build-time.

So, what I would like to hear is not a 'seconded' as required for a
policy change, but something like 'lets see some testing and if
successful we can go on and change the policy' from some debian
developers.

Looking at the list of Java package maintainerns, I would like to get
the ok from at least some of (hopefully all maintainer with more than
two packages with 'Depends: .*java.*' and who represent ~70% of the
java packages):

Arnaud Vandyck
Stefan Gybas
Takashi Okamoto
Ola Lundqvist
Ben Burton
Mark Johnson
Adam Majer
Stephen Zander
Robert Bihlmeyer
Stephen Zander
Tollef Fog Heen
Grzegorz Prokopski

I'm especially interested in the OK from Arnaud, Stefan, Takashi and
Ola, who are representing more than 50% of the above list.

Nice greetings, Jan

Hi.

My only comment is that since sarge is purportedly so close to freeze
and since this is more or less a complete policy rewrite, I think we
should keep current java policy until after sarge.

I'm perfectly happy to have the scripts/etc in java-common/etc now, as long
as they don't interfere with current behaviour and current policy.

Ben. :)

Hallo Ben,

* Ben Burton wrote:

Yes, thats sensible.... :) Anyway, as I stated, I can't see a problem
in implementing the new behaviour. Apart from a few packages, it will
be something like

if [ -x /usr/bin/findjava ] ; then
  POSSIBLE="kaffe sun-java-1.4 ..." ;
  JAVACMD="$(findjava -client $POSSIBLE)";
fi
if [ "$JAVACMD" = "" ] ; then
   # current magic to find a java
fi
if [ -x /usr/bin/java-config ] ; then
  PACKAGES="...."
  CLASSPATH="$(java-config --all $PACKAGES)";
else
   # current classpath magic
fi

So that's one :)

Jan

Same for me,  wait until sarge is out and  let's restart the discussions
about a new policy.

Hallo Arnaud,

* Arnaud Vandyck wrote:

Sorry, but that was exactly the reply I was *not* hoping for. IMO
there is enough discussion and the whole thing needs some testing. I'm
also willing to write most of the initial patches. What I'm not
willing is just sit quiet until the *next* (and next...) discussion.

To integrate this changes would be in most cases (we don't have that
many java *apps*, but lots of libs) a
'cp debian/package.jars debian/package/usr/share/java-config/package'
in debian/rules (or add dh_java during binary-indep).

For VM maintainer it will be a cp and maybe some looking into the doc
and adding the right flags, where I missed something.

In the apps it would be CLASSPATH="$(java-config -all $PACKAGE)",
where $PACKAGE is all converted packages (I will start with the
low-level libs and work my way up...) and adding the if .. else .. fi
block around the current 'find java'-magic.

All this *does* *not* interfere with current policy.

But for all this I need something more than 'wait for the sarge
release and *restart* discussing *then*'. I've no idea how far the
sarge release is (the last bit I read said two weeks behind), but it
will take at least another two month (more like after new year). There
are several attemps to add something to the policy in the BTS and all
have died. I'm not willing to let that happen to this attempt.

So what I'm asking now for is either

* Yes, I find this propsal reasonable and think it will work. I intent
  to add the patches to my packages, if they don't interfere with
  current policy.
OR
* No, this proposal sounds like it will not work and I don't want to
  make any changes to my packages.
OR
* IMO this points (give list!) are not well thought out and need to be
  looked into again.

I intent to give up proposing this changes, if the second opinion is
consensus or if noone replys to this.

I'm sorry to say it this hard, but IMO it does no good to say 'later'
all the time. Until now, there was almost no reaction form DDs and if,
the questionable points were hopefully answered or the appropriated
changes were made. What I want is some more comitment to this, so that
I see that the work on this isn't wasted.

And I'm also more than a bit fed up with this 'no reaction' attempt,
especially when there is almost no work doing the change or just
reading through the propsal and add a opinion.

So, please make up your mind.

Jan

[a minor rant about how there's not much progress on Java in Debian]

I don't anything particularly useful to reply to the points that Jan
makes, largely because I've given up on Java per se as part of Debian.
There's a very fundamental reason for this: Debian is about free
software, and Java is not free.

Don't get me wrong; free things can be made with Java, just as they
can be with any other language... and once compiled (preferably to
native code) they can be distributed just like any other free code.
They have dependencies which need to be fulfilled to run (libraries,
VMs, what have you), and those can also be handled in all the normal
ways.  We don't need a special policy for that.

The only complication for Java really is in this odd idea that you
should be able to run bytecode designed for one VM in a different
VM... and that somehow we as designers of the Debian system ought
to support this idea.  Yes, I know that various vendors have hyped
this idea a lot, but the various VM incompatibilities that have
turned up in discussion reveal the relative unworkability of this
idea.  It's like asking code to work under Windows and Linux and
AIX and MacOS 9 all at the same time... for simple stuff, it's no
big deal, but as soon as you try to use any advanced features of
any environment (like file-locking, perhaps), compatibility with
the other environments pretty much vanishes (unless you have
separate code for each environment, which is a maintenance
nightmare).

The whole reason we're even having this discussion is that people
find it unpalatable to install multiple VMs because different
packages that they've installed prefer different environments.
After watching the arguments and learning of the difficulties,
I just say "get over it".  Guess what?  If you have programs
written in csh and bash and ksh, then you have to install multiple
shells.  The same is basically true of Java: if you have programs
written for kaffe and for sablevm and for Sun's Java, then you
have to install multiple VMs.  Trying to convince programs to run
in a non-preferred environment is just asking for pain.

Java libraries are perhaps a sticking point; most people won't
want to install a separate version of xerces or cryptix for each
and every VM.  However, once we dispense with this notion of
trying to make programs run under any VM, then libraries can just
be installed int some well known directory, and the programs can
include all the appropriate things in their classpaths without
difficulty.  It's trying to make the classpath generation generic
that complicates things.

So no, I don't think we need the sorts of changes to the Java
Policy that have been proposed.  I think we ought to define a
library location, specify that programs should have wrappers
to invoke a package-preferred VM with a workable classpath, and
leave the rest up to standard policy and dependency handling.
If a package wants to support multiple VMs, that's fine... but
the differences are great enough that I can't imagine any end-user
packages actually doing so.

This all wouldn't be an issue if the primary implementation (Sun's)
was free software... then we wouldn't have any more difficulty
than perl, as an example.  But that's not the case for Java, and
the fracturing of the execution environment that results from
having multiple competing implementations makes it completely
unrealistic to treat Java as a plug-and-play environment where
the end user can pick and choose base components without affecting
the things they put on top of it.

- Alex

Hallo Jan,

sorry Jan ;)

I agree with  you but let me explain  (if I have to) why I  did not take
part of all these discussions:

- I had a lot of work at the beginning of the 'java policy' thread;
- then went on holliday;
- then had a lot of work (until late november!).

So I did not follow all  the thread. I appreciate the summary effort and
after reading this mail I have no special argument to stop your proposal
but I do not  want to second it if it's a MUST  in the policy. You know,
something that makes a bug 'serious'. First  I'm not yet a DD so even if
I can make  the changes soon, I'll  need to ask a sponsor,  I'm not sure
when my packages  will be uploaded. Second, I'm sure  some DD just don't
have the  time to change all their  packages.

 OK, it's a  line change for a  library, but you also have  to build the
 .jars file  to complete (also, I  did not catch if  there was different
 jar groups depending on the virtual machine?.. exemple: I don't know if
 all     the    jvm's     have    javax.sql.*,     it's     needed    by
 libcommons-dbcp-java... and  tomcat4. There was an attempt  to make the
 javax.sql.* package  also with jta (but  non free) but I  drop the idea
 because classpath project did  reimplemented it... just an exemple that
 comes in my mind, maybe they are others).

It seems to be really easy ;) Ean, any comment? :-D

Well, if I understand, we can slowly begin the work of implementing your
idea  _without a policy  change_ and  when every  packages does  use the
findjava wrapper a good policy text will be ready for inclusion into the
java policy.

Sorry for the restart the discussion! ;)

Yes it  seems resonable but  I don't  know if it  will work on  the long
term...   and I  think the  dependency of  libraries in  java is  a java
problem  that must  be solved  in  a java  way: META-INF/MANIFEST.MF  or
something  like this  also in  libraries, not  only in  applications. Or
maybe  an   xml  file   or  a  property   file  in  META-INF   like  the
META-INF/services but I don't know  this purpose very well. I'll discuss
that with a co-worker and will try to study that in the futur weeks.

again sorry.

I don't  know if it's the  better solution but  if I'll be there  to try
your javafind package. I'll look for the informations you provide in the
archives but  once again, I'm  not yet a  DD and I'm very  busy teaching
java at the moment ;)

'Fed up' is hard! Jan, it's not  (at least for me) an easy area and your
mails (Dalibor also  like the long mails ;)) are  not easy to understand
after a  day full of  work and  stress (I don't  tell you about  my wife
complaining  because  I  read  my  mail  :-D).  I  can  understand  it's
frustrating and I have to congratulate you for your work.

Make up your mind
-----------------

1° I don't think it's the better solution...;
2° ...but it seems to be a good one at the moment;
3° I don't think your proposal must be marked as 'MUST' or 'REQUIRED';

So (even  if I'm not  yet a DD),  I second your  proposal if point  3 is
respected.

Last thing, I did not follow _the whole_ discussion and am not completly
familiar with policy changes and so on.

Very last  thing, I do prefer focus  on more urgent things.  For me, the
most important thing  at the moment for the  debian-java community is to
have gjdoc in the best shape to be able to build the javadoc with a free
tool, next, ship  gjdoc with kaffe (I'm helping the  kaffe team but have
not the time at the moment). After this major goal, I'd like to have the
most free  jvm and compilers  in the next  stable release... and  in the
best shape as possible.

I'd like to contribute more to this  thread and I hope to have an easier
and more 'java oriented' solution one day. Thanks for your work Jan, and
sorry to be not so responsive on this thread.

Best regards,

Hallo Arnaud,

* Arnaud Vandyck wrote:

I understand that the proposal, as it is, is *not* "secondable", as it
misses some testing first. Debian policy is based on 'common practise'
(which the old java policy, in most parts, was surely not), so I hope
I can implement wide parts of it before asking for a policy change.

But without any 'make it so' comments, or some of the bigger java
maintainers, which change their packages, I will not get this changes
made and so the policy will not get tested and in the end nothing
changes.

I don't ask for a upload only for the sake of adding the changes for
the porposal. What I ask for is 'if I have to upload the apckage the
next time, this file will also be part of it'. I don't expect to file
100 wishlist bugs during the next days, but I will spread most of the
changes over the next weeks.


No. The java runtime requirement is 'located' in teh java apps (which
have to call 'findjava' in the starter script).

Anyway, this is a really good point, which I overlooked during teh
discussion. This will basicly mean that we can remove the

|A package must depend on the disjunction of all JVMs with
|which it has been tested succesfully.

Part from the library section. Or replace it with

|A library only package should not depend on any java virtual machine
|package.

Yes, that's also still something I waiting for :) Although I think I
will do much of the work, as I want to write the patch for mpkg-j2sdk...

Thanks, thats the first time someone said this. I hope some other will
also write something along this lines.

I don't much mind the restarting but much more the 'wait until after
the discussion/...' part.

If you do this, then you should start this discussion on a sun ML, so
it is included there. But this will probably not work, as SUN will
simple refuce to accept that there are 'non licensed' JVM. And thats
the primary goal for such mechanism, for the rest it is too less
'finetuned' to make any difference.

Anyway, I think this proposal is a big difference to the 'maybe it
will start, maybe not' situation, which is currently happening, when
you don't use some special tricks (JAVA_HOME), which are undocumented
and not debian conform.

but with 15 package to change you are quite a big 'do it' opinion :)

Better to the 'best' sollution or to the current solution?

Thanks for teh reply!

Jan