--- Please fill out the fields below. ---

   Package name: zfs-linux-git
        Version: 0.8~
Upstream Author: Brian Behlendorf <behlendorf1@llnl.gov>
            URL: http://www.zfsonlinux.org/
        License: CDDL
    Description:
 ZFS is an advanced file system and volume manager which was originally
 developed for Solaris. It provides a number of advanced features like
 snapshots, clones, live integrity checksums, deduplication, compression,
 encryption, and much more. The port to the Linux kernel includes a
 functional and stable SPA, DMU, ZVOL and ZFS Posix Layer (ZPL).
 .
 This package is a very close fork of the existing zfs-linux package
 already in Debian, but aims to track git master and the ntrim2-next
 patch set adding trim support by Tim Chase.
 .
 BEWARE: THIS VERSION OF ZFS IS EXPERIMENTAL. IT MAY EAT YOUR DATA.



Just like the parallel packaging of wine-unstable and wine, I have
packaged [1] a -git version of zfs (and spl). Actually, the version is
a rebase of a PR [2] by Tim Chase [3] to add trim/unmap support to ZFS.
This appears to be getting close to a mainline inclusion, and there has
gathered much interest. The git mainline already includes encrypted
datasets.

I am proposing adding this package to the experimental repository (as
it definitely does not meet the quality and stability requirements for
any Debian release). The exposure of these experimental features to
the broader Debian community provides a better test-bed for the
zfsonlinux mainline (and for this ntrim2-next PR especially). This
testing improves the quality of the released version, protecting
Debian users not as willing to test out the newest features
filesystems.

Moreover, because the packaging is---and will remain, to the best of my
ability---close to the released-version ZFS packaging, changes
migrating into the release version from master that require adaptations
to the Debian packaging will have already been addressed (by myself).
This should alleviate some of the demands on the already overburdened
maintainers of zfs-linux.

There are a few considerations I'd like to highlight:

1. The packages should be difficult to accidentally pull in, but
still satisfy any dependencies by third-party packages for zfs. I've
done this by changing the name, and adding both a Conflict: and
Provides: dependency on the original zfs-linux package name.

2. ZFS and SPL are relatively tightly coupled software. I have
opted for a tight release schedule, where both the ZFS and SPL
commit hashes are tracked for both ZFS and SPL packaging. This
avoids strange bugs with out-of-date SPL versions, but will
require re-releasing SPL versions for every ZFS release.

This can be adjusted if it is decided that it is important.

3. I'm tracking a not-yet-mainlined patch set, and its role is
expected to become more stable once that patch set is mainlined.
The package name anticipates this outcome.

[1] https://github.com/aerusso/pkg-zfsonlinux-git
[2] https://github.com/dweeezil/zfs/tree/ntrim2-next
[3] https://github.com/dweeezil

(CC-ing the existing ZoL Debian packaging team, although there is hardly
ever any response there..)

as an upstream contributor and with my downstream/derivative hat on I
have an alternative proposal:

I've been pondering cleaning up and pushing the existing Debian
packaging (or some variant thereof) upstream to replace the ugly "build
rpms and use alien + various hacks to get some sort of .deb output"
"build system" that is currently used.

it seems like upstream is just lacking the people and knowledge to do
this, but it would be welcome (see config/deb.am). would you be
interested in teaming up on this?

there are quite a lot of people running git master or the latest 0.7.x
release on some variant of Debian/Ubuntu that get tripped up on every
other release because of missing files and lack of integration into
Debian tooling, probably more than would be willing/able to track
experimental ;)

this would also have the big upside of being able to integrate the
upstream Debian packaging directly into zfs-buildbot infrastructure,
hopefully keeping most of the packaging-related issues we've seen in the
past from being part of any tagged releases in the future.

the only real downside I see is the fact the the debian/ folder would
need to be excluded when importing stable upstream releases into Debian
proper, in favor of cherry-picking the relevant packaging changes only
as separate commits afterwards (but this is easy with the existing GBP
workflow, and something that a lot of packages with upstream debian/
folders handle just fine).

I think that this is a great idea, and I would definitely like to
collaborate on this. I'll start a discussion with you off-list about
this. Maybe a feature request on upstream's bug tracker.
I was already sold. An automatically up-to-date upstream apt repository
would be a great boon for zfsonlinux adoption, and possible, like you're
saying.

I still think that getting an experimental package into Debian would
improve access to Debian users. Moreover, another (perhaps the most)
import objective of my proposal is to *reduce* workload on the existing
zfsonlinux team. I'll support any proposal that makes their life easier,
and my though is that a continuously up-to-date Debian experimental
package that can be directly imported on, e.g., release-day should help
out.

I have packaging of zfsonlinux (for upstream git revisions) that
is in need of review [1]. It builds, and zfs-dkms builds as well.
I have only done very superficial testing (i.e., the zfs module
loads, you can create a pool).

This was somewhat nontrivial because upstream recently merged spl,
the "Solaris Porting Layer", into zfsonlinux. In the short term,
this made packaging more challenging, but in the long term it will
make maintenance much easier.

Highlights from this new packaging:

1. Upstream now ships explicit an statement of kernel version
compatibility [2]. I've integrated that into the debian packaging,
so the maintainer will no longer have to update that manually.

2. Tunable parameter to put architecture independent zfs scripts
in a Debian FHS compliant location [3]. Hopefully, future
additions of scripts will use this parameter and Debian will get
that compliance "for free". I expect this to be merged relatively
soon, further simplifying the packaging.

3. debian/update-git , which automatically builds a changelog
for an upstream git revision. Another tool to simplify an
ambitious user's desire to build a recent git version.

I'd appreciate feedback.

Thanks,
Antonio Russo

[1] https://salsa.debian.org/aerusso-guest/zfs/commits/debian/git
[2] https://github.com/zfsonlinux/zfs/pull/7571
[3] https://github.com/zfsonlinux/zfs/pull/7597

as promised, and unfortunately delayed a bit longer than I wanted.
thanks for the initial push - some of the points are more for a
discussion with upstream regarding their inclusion of some variant of
this, some are for debian experimental.

- compat 12 is IMHO too new for anything except experimental, it's still
  subject to change.
- given the different licences and thus parts of the archive, I am not
  sure whether we can merge zfs-dkms and spl-dkms inside Debian
- which distros do we want to support upstream? Debian Stretch+, Ubuntu
  Xenial/Bionic/Cosmic? just the latest ones?
- compat 10 or 11? Stretch only has 10 without backports..

debian/rules:
- chmod a-x files should be on separate lines, otherwise git blame is
  stupid ;)
- same for copied/installed scripts that need to be listed explicitly
  (DKMS)
- the dmks.mkconf call does not belong into dh_auto_install
  (semantically). does it need to be there or can we move it to
  dh_auto_configure or dh_auto_build ? why not keep it in
  override_dh_dkms?
- same for the "make dist" call, which should probably be run before the
  build to prevent mistakes in "make dist" from tainting DKMS sources
  with build products?
- debian/git-version.sh could benefit from some comments/rationale ;)
- debian/git-version.sh does not handle actual release tags correctly
  (the resulting package version is a native one)
- debian/git-version.sh should probably somehow handle adding a
  pkgrelease suffix as well? maybe as optional, second parameter
  (defaulting to 1), and in case it is ommitted but d/changelog contains
  the same version with -1, bump it to -2?
- debian/update-git: it would be cool to pre-populate the changelog with
  a shortlog since the previous version (if the previous version matches
  either the git-describe or release tag versioning scheme,
  alternatively we could just encode the git commit in the changelog
  like "gbp dch --snapshot" does?)

I'll do some test builds and check the resulting packages later on -
thanks for getting the ball rolling!

dh_missing was added in debhelper 10.3. I'll remove the use, and suffer
the deprecation warning.
It would be a real pity if we had to keep these packages separated.
Everything is much easier to maintain and build in a single package.
Moreover, actually separating the now-merged upstream packaging is going
to be a challenge---and probably will introduce weird bugs. If, for
some reason, this must be separated, I think we should consider
automatically building these binaries on the user's machine, like
say libdvd-pkg, before we consider splitting splitting the packaging:
It would be unacceptable to allow a bug in a filesystem driver to be
introduced to work around a licensing issue.

I'll formulate the question carefully, and submit it to debian-legal.
I will say that the CDDL specifically allows for inclusion in a
"Larger Work", and similarly, GPL says that "mere aggregation" is
insufficient to have the license apply to the rest of it.

What *will* be important is making sure that each source file license
is properly tracked in copyright---so as to ensure each distributed
non-source file be subject to at most one of the two licenses.

As for the section: I imagine it would stay in contrib---for the same
reasons it was originally placed there.

I only have experience with Debian. The Ubuntu packaging looks very
different---in particular, they look like they build the module with
the kernel (https://wiki.ubuntu.com/ZFS). I don't see any reason to
try to support that.
I say we target 10.
Changed.
It's in override_dh_dkms, and clean it up there too.
Cleaned up as suggested. Might as well, since you've noticed it.
This sounds reasonable. I was only ever targeting this script at
upstream git snapshots. My use case is a script that just checks out
and builds the next git version, making it available for use.
git-shortlog should be able to do this relatively easily. I'll get
around to these last two points eventually.

Thanks for looking at this!

Antonio

Antonio Russo:

You can use dh_missing with a simple "debhelper (>= 10.3~)" in
Build-Depends in any compat level.  Though in compat level 11 (or
earlier) you will have to explicitly ask dh_missing to do something
(e.g. by using "dh_missing --list-missing").


Thanks,
~Niels

(trimmed down a bit)

I guess there is a sane argument for merging the source and binary
packages since upstream does the same. We have to ensure proper
transitioning though, especially for spl-dkms to zfs-dkms, both inside
Debian, and for users switching to the snapshot packages built from
upstream git. If debian-legal/ftp-masters don't object, I am all for it
:)

This was more meant for upstream inclusion - the more and older distros
we want to support, the more attention we need to pay to tooling
compatibility. I'm sure there are a lot of Ubuntu users that would like
to use upstream snapshot and "most current release" packages. Whether
the Ubuntu maintainers want to re-use parts of the then upstream
packaging or not is of course up to them.

That Ubuntu ships the modules pre-compiled with their kernel is not an
issue - zfs-dkms would contain a newer version, and thus be used. Maybe
it makes sense to offer two build profiles for DKMS and kmod style
binary modules? There is some support for this in d/rules, but I haven't
tried the -modules stuff yet. Ubuntu is not the only downstream that
does not build ZFS DKMS packages, but ships pre-built modules, and
individual users might also want to build the modules once and deploy on
many hosts. I know DKMS supports this as well nowadays, but skipping
that extra step would still be nice..

Starting with 0.8, it would be nice to also build official upstream
release packages with proper Debian packaging. In that spirit, I am all
for supporting both "build from release tag" and "build from arbitrary
commit".