* Package name    : libjwat-java
  Version         : 1.2.1
* URL             : https://sbforge.org/display/JWAT/JWAT
* License         : Apache-2.0
  Programming Lang: Java
  Description     : Java Web Archive Toolkit

A collection of libraries to use for reading, writing and validating ARC,
WARC and GZip files. Also includes various helper classes to help with
different types of input streams. Finally there are also classes to help
with HTTP, character encoding and other Internet related protocols.

Additional information:
 - Why is this package useful/relevant?
   The binary Debian package of JHOVE, a java tool for format-specific
   identification, validation, and characterisation of digital objects,
   currently lacks support for EPUB, GZIP, PNG, and WARC files due to a
   missing dependency. Adding libjwat-java to the Debian archive as
   requested in this bug report would make it possible to ship JHOVE
   including those extra modules with Debian too.
   See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1037934
 - How do you plan to maintain it?
   At Zuse Intstitute Berlin, we have prepared debian/* packaging files
   for an initial upload. These will be made available in a follow up to
   this bug report. We are looking for a sponsor, however, as there is
   no Debian developer in our team. Specifically, I am hoping for
   support by the Debian Java Maintainers because they are looking after
   the JHOVE package in Debian. As explained above, extending the
   capabilities of JHOVE is the main reason for filing this bug report.

Please have a look at the following patches. They might be suitable as
an initial seed for a salsa repository for the new package. Please apply
and simply pull in the upstream sources by means of uscan.

Thank you in advance for any support you can provide.

Cheers,

Elias

Sorry for all those emails, but I have just realised that
debian/README.source needed fixing. The reason is that I started out
with the test suite disabled but have managed to get it running after
all.

So, I have added another patch to the previous two and will append all
three to this message.

Cheers,

Elias


Am 19. Juli 2023 um 23:16 schrieb Elias Oltmanns:

Hello Elias,
reviewing the package.  Everything looks good from the standpoint of
constructing the .deb.

I haven't uploaded yet because I am not yet sure how (or whether it is
even necessary) to document the license and copyright of a few of the
test resources.  In particular, these files:

Files: jwat-arc/src/test/resources/IAH-20080430204825-00000-blackbook.arc
       jwat-arc/src/test/resources/IAH-20080430204825-00000-blackbook.arc.gz
       jwat-gzip/src/test/resources/IAH-20080430204825-00000-blackbook.warc
       jwat-gzip/src/test/resources/IAH-20080430204825-00000-blackbook.warc.gz
       jwat-warc/src/test/resources/IAH-20080430204825-00000-blackbook.warc
       jwat-warc/src/test/resources/IAH-20080430204825-00000-blackbook.warc.gz

For which the decopy [2] utility generates a very messy copyright entry
that ends with:

    License: CC-BY-NC-SA-ND-3 or Expat or GPL or LGPL-2.1+

It's conceivable that these WARC [3] files contain copyrighted materials
and that uploading them as components of the source package would be
considered redistribution, but I am admittedly not well-versed enough in
this area to say for sure without looking into the contents in more
detail.

It would be nice to be able to (a) use the files as-is so that we don't
have to either (b) remove the files and disable tests, or (c) replace
the files and rewrite the tests that access them.  I spot-checked a few
tests and they appear to expect to be able to locate specific contents
in the archive, so (c) would be non-trivial and could result in the
package being quite difficult to maintain over time, since any upstream
changes to those tests would require updating the patch(es).

Let me know if you have any thoughts on this.  Otherwise, I will follow
up once I have a chance to look through the test resources in more
detail.

Thank you,
tony

[1] https://salsa.debian.org/java-team/libjwat-java
[2] https://tracker.debian.org/pkg/decopy
[3] https://www.loc.gov/preservation/digital/formats/fdd/fdd000236.shtml

Hi Tony,




Am 21. Juli 2023 um 08:08 schrieb tony mancill:
[...]
https://archive.org/download/ExampleArcAndWarcFiles/
(See also https://archive.org/details/ExampleArcAndWarcFiles)
that it contains a webcrawl of archive.org (or part of it). It does
include many files of different formats and media types partly carrying
their own license information. This is why decopy lists so many
different licenses.

There might be false positives, though. This is because the warc file
contains web pages listing details about other resources including
license information. At least some of those resources are not included
in the warc file themselves, so the license might actually not be
applicable to any material in the warc file.

Passing the term "-nd" to the editor's search function produces good
examples. The first occurrence appears on a site providing details about
some podcast which is licensed CC-BY-NC-ND. The podcast itself, however,
is not part of the warc file. Unfortunately, there are quite a few
matches of "-nd" that would need checking and I haven't worked out a
good approach to make this actually feasible. Here is one interesting
observation though:

$ grep -ae "^Content-Type:" IAH-20080430204825-00000-blackbook.warc \
    | cut -d' ' -f2 | sort | uniq
application/http;
application/warc-fields
application/x-javascript
application/x-shockwave-flash
image/gif
image/jpeg
image/png
text/anvl
text/css
text/dns
text/html
text/html;
text/plain
text/plain;
text/xml

In particular, a lot of Content-Types are missing from this list in
relation to the resources mentioned as being licensed under some
CC-BY-ND license.

Since this is from archive.org, their terms of service apply:
https://archive.org/about/terms.php

This might turn out to be a bit to restrictive for DFSG, since it
includes this passage:
    Access to the Archive’s Collections is provided at no cost to you
    and is granted for scholarship and research purposes only.

It makes sense for them to take this rather defensive approach since
they provide a lot of content from different sources. On the other hand,
the warc file appears to be intentionally prepared for testing and demo
purposes and uploaded by someone at archive.org. That is why I had hoped
for a more permissive license, but could not find any indication of it.

Contacting archive.org and asking for license clarification might be an
option. I am not sure whether I would hold my breath, but it seems to me
that removing the files in question might turn out to be the only
alternative. Then again, we could disable the test suite, after all, so
the build would not depend on the presence of those files.

Woud do you thin?

Best wishes,

Elias

Hello Elias,

I intend to remove files for which we don't have clear licenses from the
DFSG repacked tarball and disable the tests that depend upon them.
Apologies for the delay here - it will be a few days yet before I will
have the package for an upload.

Thank you,
tony