#1033410 RM: tiledb -- ROM as full builds not possible within Debian

Package:
ftp.debian.org
Source:
ftp.debian.org
Submitter:
Dirk Eddelbuettel
Date:
2023-04-06 17:06:03 UTC
Severity:
normal
Tags:
#1033410#5
Date:
2023-03-24 15:46:40 UTC
From:
To:
The TileDB source package uses cmake and find_package for external resources;
the upstream team maintaining the core library is going to switch to vcpkg
soon.

A full featured TileDB really needs the AWS SDK which we still do not have,
would benefit from the GCS SDK (not in Debian), ditto for Azure as well as
integration with several other components which is difficult as upstream pins
some parts we have (CapnProto, Catch2, ...) but in different versions.

TileDB is an excellent product, and a lovely C++ library that can be used
well from Python and R.  But it is not a good fit within Debian, and after
maintaining for about two years as a caretaker maintainer (it was brought and
then left by someone else) I have come to the conclusion that it is not a
good fit for Debian.

Full disclosure: I work at TileDB and build the library all the time (mostly
on current Ubuntu as my dev env) and am hence quite familiar with various
extensions and optional components.  We are short-changing our users by not
enabling them, but we cannot enable them and stay true to Debian packaging
guidelines.

Happy to expand if there are questions.

Thanks,  Dirk
#1033410#10
Date:
2023-03-24 21:17:19 UTC
From:
To:
and

There are rdepends that will have to be addressed first (build-dep/depends
removed or package removed):

Checking reverse dependencies...
# Broken Depends:
tiledb-py: python3-tiledb
tiledb-r: r-cran-tiledb

# Broken Build-Depends:
genomicsdb: libtiledb-dev
tiledb-py: libtiledb-dev (2.7.0~ >=)
tiledb-r: libtiledb-dev

Dependency problem found.

Please remove the moreinfo tag once these have been addressed.

Scott K
#1033410#15
Date:
2023-03-25 12:48:38 UTC
From:
To:
On 24 March 2023 at 17:17, Scott Kitterman wrote:
| On Fri, 24 Mar 2023 10:46:40 -0500 Dirk Eddelbuettel <edd@debian.org> wrote:
| >
| > Package: ftp.debian.org
| > Severity: normal
| >
| > The TileDB source package uses cmake and find_package for external resources;
| > the upstream team maintaining the core library is going to switch to vcpkg
| > soon.
| >
| > A full featured TileDB really needs the AWS SDK which we still do not have,
| > would benefit from the GCS SDK (not in Debian), ditto for Azure as well as
| > integration with several other components which is difficult as upstream pins
| > some parts we have (CapnProto, Catch2, ...) but in different versions.
| >
| > TileDB is an excellent product, and a lovely C++ library that can be used
| > well from Python and R.  But it is not a good fit within Debian, and after
| > maintaining for about two years as a caretaker maintainer (it was brought
| and
| > then left by someone else) I have come to the conclusion that it is not a
| > good fit for Debian.
| >
| > Full disclosure: I work at TileDB and build the library all the time (mostly
| > on current Ubuntu as my dev env) and am hence quite familiar with various
| > extensions and optional components.  We are short-changing our users by not
| > enabling them, but we cannot enable them and stay true to Debian packaging
| > guidelines.
|
| There are rdepends that will have to be addressed first (build-dep/depends
| removed or package removed):
|
| Checking reverse dependencies...
| # Broken Depends:
| tiledb-py: python3-tiledb
| tiledb-r: r-cran-tiledb
|
| # Broken Build-Depends:
| genomicsdb: libtiledb-dev
| tiledb-py: libtiledb-dev (2.7.0~ >=)
| tiledb-r: libtiledb-dev

Well yes. They all have to be removed too.

As I argued, which you did not engage with, TileDB is hard to get 'right' in
Debian so as someone with domain expertise (as I work) suggest we should in
fact remove it.

I will file RMs for genomicsdb, tiledb-py, tiledb-r. The last one is a ROM.

Dirk
|
| Dependency problem found.
|
| Please remove the moreinfo tag once these have been addressed.
|
| Scott K
| x[DELETED ATTACHMENT signature.asc, application/pgp-signature]
#1033410#18
Date:
2023-03-25 12:48:38 UTC
From:
To:
On 24 March 2023 at 17:17, Scott Kitterman wrote:
| On Fri, 24 Mar 2023 10:46:40 -0500 Dirk Eddelbuettel <edd@debian.org> wrote:
| >
| > Package: ftp.debian.org
| > Severity: normal
| >
| > The TileDB source package uses cmake and find_package for external resources;
| > the upstream team maintaining the core library is going to switch to vcpkg
| > soon.
| >
| > A full featured TileDB really needs the AWS SDK which we still do not have,
| > would benefit from the GCS SDK (not in Debian), ditto for Azure as well as
| > integration with several other components which is difficult as upstream pins
| > some parts we have (CapnProto, Catch2, ...) but in different versions.
| >
| > TileDB is an excellent product, and a lovely C++ library that can be used
| > well from Python and R.  But it is not a good fit within Debian, and after
| > maintaining for about two years as a caretaker maintainer (it was brought
| and
| > then left by someone else) I have come to the conclusion that it is not a
| > good fit for Debian.
| >
| > Full disclosure: I work at TileDB and build the library all the time (mostly
| > on current Ubuntu as my dev env) and am hence quite familiar with various
| > extensions and optional components.  We are short-changing our users by not
| > enabling them, but we cannot enable them and stay true to Debian packaging
| > guidelines.
|
| There are rdepends that will have to be addressed first (build-dep/depends
| removed or package removed):
|
| Checking reverse dependencies...
| # Broken Depends:
| tiledb-py: python3-tiledb
| tiledb-r: r-cran-tiledb
|
| # Broken Build-Depends:
| genomicsdb: libtiledb-dev
| tiledb-py: libtiledb-dev (2.7.0~ >=)
| tiledb-r: libtiledb-dev

Well yes. They all have to be removed too.

As I argued, which you did not engage with, TileDB is hard to get 'right' in
Debian so as someone with domain expertise (as I work) suggest we should in
fact remove it.

I will file RMs for genomicsdb, tiledb-py, tiledb-r. The last one is a ROM.

Dirk
|
| Dependency problem found.
|
| Please remove the moreinfo tag once these have been addressed.
|
| Scott K
| x[DELETED ATTACHMENT signature.asc, application/pgp-signature]
#1033410#23
Date:
2023-03-25 13:11:59 UTC
From:
To:
On 25 March 2023 at 07:48, Dirk Eddelbuettel wrote:
|
| On 24 March 2023 at 17:17, Scott Kitterman wrote:
| | On Fri, 24 Mar 2023 10:46:40 -0500 Dirk Eddelbuettel <edd@debian.org> wrote:
| | >
| | > Package: ftp.debian.org
| | > Severity: normal
| | >
| | > The TileDB source package uses cmake and find_package for external resources;
| | > the upstream team maintaining the core library is going to switch to vcpkg
| | > soon.
| | >
| | > A full featured TileDB really needs the AWS SDK which we still do not have,
| | > would benefit from the GCS SDK (not in Debian), ditto for Azure as well as
| | > integration with several other components which is difficult as upstream pins
| | > some parts we have (CapnProto, Catch2, ...) but in different versions.
| | >
| | > TileDB is an excellent product, and a lovely C++ library that can be used
| | > well from Python and R.  But it is not a good fit within Debian, and after
| | > maintaining for about two years as a caretaker maintainer (it was brought
| | and
| | > then left by someone else) I have come to the conclusion that it is not a
| | > good fit for Debian.
| | >
| | > Full disclosure: I work at TileDB and build the library all the time (mostly
| | > on current Ubuntu as my dev env) and am hence quite familiar with various
| | > extensions and optional components.  We are short-changing our users by not
| | > enabling them, but we cannot enable them and stay true to Debian packaging
| | > guidelines.
| |
| | There are rdepends that will have to be addressed first (build-dep/depends
| | removed or package removed):
| |
| | Checking reverse dependencies...
| | # Broken Depends:
| | tiledb-py: python3-tiledb
| | tiledb-r: r-cran-tiledb
| |
| | # Broken Build-Depends:
| | genomicsdb: libtiledb-dev
| | tiledb-py: libtiledb-dev (2.7.0~ >=)
| | tiledb-r: libtiledb-dev
|
| Well yes. They all have to be removed too.
|
| As I argued, which you did not engage with, TileDB is hard to get 'right' in
| Debian so as someone with domain expertise (as I work) suggest we should in
| fact remove it.
|
| I will file RMs for genomicsdb, tiledb-py, tiledb-r. The last one is a ROM.

Done in #1033453 and #1033454 as RMs for tiledb-{py,r}, as well as in
#1033455 as a nudge to no longer use the optional build-depends on TileDB.

Dirk

| Dirk
| |
| | Dependency problem found.
| |
| | Please remove the moreinfo tag once these have been addressed.
| |
| | Scott K
| | x[DELETED ATTACHMENT signature.asc, application/pgp-signature]
|
| --
| dirk.eddelbuettel.com | @eddelbuettel | edd@debian.org
#1033410#26
Date:
2023-03-25 13:11:59 UTC
From:
To:
On 25 March 2023 at 07:48, Dirk Eddelbuettel wrote:
|
| On 24 March 2023 at 17:17, Scott Kitterman wrote:
| | On Fri, 24 Mar 2023 10:46:40 -0500 Dirk Eddelbuettel <edd@debian.org> wrote:
| | >
| | > Package: ftp.debian.org
| | > Severity: normal
| | >
| | > The TileDB source package uses cmake and find_package for external resources;
| | > the upstream team maintaining the core library is going to switch to vcpkg
| | > soon.
| | >
| | > A full featured TileDB really needs the AWS SDK which we still do not have,
| | > would benefit from the GCS SDK (not in Debian), ditto for Azure as well as
| | > integration with several other components which is difficult as upstream pins
| | > some parts we have (CapnProto, Catch2, ...) but in different versions.
| | >
| | > TileDB is an excellent product, and a lovely C++ library that can be used
| | > well from Python and R.  But it is not a good fit within Debian, and after
| | > maintaining for about two years as a caretaker maintainer (it was brought
| | and
| | > then left by someone else) I have come to the conclusion that it is not a
| | > good fit for Debian.
| | >
| | > Full disclosure: I work at TileDB and build the library all the time (mostly
| | > on current Ubuntu as my dev env) and am hence quite familiar with various
| | > extensions and optional components.  We are short-changing our users by not
| | > enabling them, but we cannot enable them and stay true to Debian packaging
| | > guidelines.
| |
| | There are rdepends that will have to be addressed first (build-dep/depends
| | removed or package removed):
| |
| | Checking reverse dependencies...
| | # Broken Depends:
| | tiledb-py: python3-tiledb
| | tiledb-r: r-cran-tiledb
| |
| | # Broken Build-Depends:
| | genomicsdb: libtiledb-dev
| | tiledb-py: libtiledb-dev (2.7.0~ >=)
| | tiledb-r: libtiledb-dev
|
| Well yes. They all have to be removed too.
|
| As I argued, which you did not engage with, TileDB is hard to get 'right' in
| Debian so as someone with domain expertise (as I work) suggest we should in
| fact remove it.
|
| I will file RMs for genomicsdb, tiledb-py, tiledb-r. The last one is a ROM.

Done in #1033453 and #1033454 as RMs for tiledb-{py,r}, as well as in
#1033455 as a nudge to no longer use the optional build-depends on TileDB.

Dirk

| Dirk
| |
| | Dependency problem found.
| |
| | Please remove the moreinfo tag once these have been addressed.
| |
| | Scott K
| | x[DELETED ATTACHMENT signature.asc, application/pgp-signature]
|
| --
| dirk.eddelbuettel.com | @eddelbuettel | edd@debian.org
#1033410#29
Date:
2023-03-25 13:17:48 UTC
From:
To:
wrote:

As an FTP Team member, I rarely second guess maintainers about if a package
should be removed.  I generally assume you know better and it's not like
Debian can force anyone to maintain a package.

I do, however, think the rationale is important to document what we are doing
so that if someone comes along later and wants to understand why something was
done, it's written down.

Thanks for the other rm bugs.  I'll process the group next time I do removals.

Scott K
#1033410#34
Date:
2023-03-25 13:37:18 UTC
From:
To:
On 25 March 2023 at 09:17, Scott Kitterman wrote:
| On Saturday, March 25, 2023 8:48:38 AM EDT Dirk Eddelbuettel wrote:
| > On 24 March 2023 at 17:17, Scott Kitterman wrote:
| > | On Fri, 24 Mar 2023 10:46:40 -0500 Dirk Eddelbuettel <edd@debian.org>
| wrote:
| > | > Package: ftp.debian.org
| > | > Severity: normal
| > | >
| > | > The TileDB source package uses cmake and find_package for external
| > | > resources; the upstream team maintaining the core library is going to
| > | > switch to vcpkg soon.
| > | >
| > | > A full featured TileDB really needs the AWS SDK which we still do not
| > | > have,
| > | > would benefit from the GCS SDK (not in Debian), ditto for Azure as well
| > | > as
| > | > integration with several other components which is difficult as upstream
| > | > pins some parts we have (CapnProto, Catch2, ...) but in different
| > | > versions.
| > | >
| > | > TileDB is an excellent product, and a lovely C++ library that can be
| > | > used
| > | > well from Python and R.  But it is not a good fit within Debian, and
| > | > after
| > | > maintaining for about two years as a caretaker maintainer (it was
| > | > brought
| > |
| > | and
| > |
| > | > then left by someone else) I have come to the conclusion that it is not
| > | > a
| > | > good fit for Debian.
| > | >
| > | > Full disclosure: I work at TileDB and build the library all the time
| > | > (mostly on current Ubuntu as my dev env) and am hence quite familiar
| > | > with various extensions and optional components.  We are short-changing
| > | > our users by not enabling them, but we cannot enable them and stay true
| > | > to Debian packaging guidelines.
| > |
| > | There are rdepends that will have to be addressed first (build-dep/depends
| > | removed or package removed):
| > |
| > | Checking reverse dependencies...
| > | # Broken Depends:
| > | tiledb-py: python3-tiledb
| > | tiledb-r: r-cran-tiledb
| > |
| > | # Broken Build-Depends:
| > | genomicsdb: libtiledb-dev
| > | tiledb-py: libtiledb-dev (2.7.0~ >=)
| > | tiledb-r: libtiledb-dev
| >
| > Well yes. They all have to be removed too.
| >
| > As I argued, which you did not engage with, TileDB is hard to get 'right' in
| > Debian so as someone with domain expertise (as I work) suggest we should in
| > fact remove it.
|
| As an FTP Team member, I rarely second guess maintainers about if a package
| should be removed.  I generally assume you know better and it's not like
| Debian can force anyone to maintain a package.

Thanks for your understanding. I am really torn. I love the package and
library (and as I mentioned, work there and use it daily albeit from source).

But one really wants a 'full' build with AWS support in particular (ditto for
eg genomicsdb).  I picked the package up when it lingered after a sole
initial upload a year earlier, and had hope to also engage with my colleague
to accomodates builds such as Debian. That has not happened. At all.
Upstream has eyes first on 'cmake builds' (as do other projects) as well as
on conda, and it is a little difficult to argue with.

As I have not been able to make that change, and as I dislike the continued
patching to enforce pins (fewer than two years ago but still) I think we
should let it go.

But you as ftp team member have a wider view and could overrule me here. I
won't argue, but I also won't maintain and I think it is 'cleaner' for all to
remove.

If the situation ever changes (there are (stalled, sadly) attempts to get the
AWS SDK in, or at least the part we need here) we can revisit. At its core it
is clean and performant C++ library.  But it really wants an expanded build.

| I do, however, think the rationale is important to document what we are doing
| so that if someone comes along later and wants to understand why something was
| done, it's written down.

Fully agreed.

| Thanks for the other rm bugs.  I'll process the group next time I do removals.

I appreciate your work in the release, as I have for years. My ~ 180 other
packages should be in good shape.

Dirk

| Scott K
| x[DELETED ATTACHMENT signature.asc, application/pgp-signature]
#1033410#37
Date:
2023-03-25 13:37:18 UTC
From:
To:
On 25 March 2023 at 09:17, Scott Kitterman wrote:
| On Saturday, March 25, 2023 8:48:38 AM EDT Dirk Eddelbuettel wrote:
| > On 24 March 2023 at 17:17, Scott Kitterman wrote:
| > | On Fri, 24 Mar 2023 10:46:40 -0500 Dirk Eddelbuettel <edd@debian.org>
| wrote:
| > | > Package: ftp.debian.org
| > | > Severity: normal
| > | >
| > | > The TileDB source package uses cmake and find_package for external
| > | > resources; the upstream team maintaining the core library is going to
| > | > switch to vcpkg soon.
| > | >
| > | > A full featured TileDB really needs the AWS SDK which we still do not
| > | > have,
| > | > would benefit from the GCS SDK (not in Debian), ditto for Azure as well
| > | > as
| > | > integration with several other components which is difficult as upstream
| > | > pins some parts we have (CapnProto, Catch2, ...) but in different
| > | > versions.
| > | >
| > | > TileDB is an excellent product, and a lovely C++ library that can be
| > | > used
| > | > well from Python and R.  But it is not a good fit within Debian, and
| > | > after
| > | > maintaining for about two years as a caretaker maintainer (it was
| > | > brought
| > |
| > | and
| > |
| > | > then left by someone else) I have come to the conclusion that it is not
| > | > a
| > | > good fit for Debian.
| > | >
| > | > Full disclosure: I work at TileDB and build the library all the time
| > | > (mostly on current Ubuntu as my dev env) and am hence quite familiar
| > | > with various extensions and optional components.  We are short-changing
| > | > our users by not enabling them, but we cannot enable them and stay true
| > | > to Debian packaging guidelines.
| > |
| > | There are rdepends that will have to be addressed first (build-dep/depends
| > | removed or package removed):
| > |
| > | Checking reverse dependencies...
| > | # Broken Depends:
| > | tiledb-py: python3-tiledb
| > | tiledb-r: r-cran-tiledb
| > |
| > | # Broken Build-Depends:
| > | genomicsdb: libtiledb-dev
| > | tiledb-py: libtiledb-dev (2.7.0~ >=)
| > | tiledb-r: libtiledb-dev
| >
| > Well yes. They all have to be removed too.
| >
| > As I argued, which you did not engage with, TileDB is hard to get 'right' in
| > Debian so as someone with domain expertise (as I work) suggest we should in
| > fact remove it.
|
| As an FTP Team member, I rarely second guess maintainers about if a package
| should be removed.  I generally assume you know better and it's not like
| Debian can force anyone to maintain a package.

Thanks for your understanding. I am really torn. I love the package and
library (and as I mentioned, work there and use it daily albeit from source).

But one really wants a 'full' build with AWS support in particular (ditto for
eg genomicsdb).  I picked the package up when it lingered after a sole
initial upload a year earlier, and had hope to also engage with my colleague
to accomodates builds such as Debian. That has not happened. At all.
Upstream has eyes first on 'cmake builds' (as do other projects) as well as
on conda, and it is a little difficult to argue with.

As I have not been able to make that change, and as I dislike the continued
patching to enforce pins (fewer than two years ago but still) I think we
should let it go.

But you as ftp team member have a wider view and could overrule me here. I
won't argue, but I also won't maintain and I think it is 'cleaner' for all to
remove.

If the situation ever changes (there are (stalled, sadly) attempts to get the
AWS SDK in, or at least the part we need here) we can revisit. At its core it
is clean and performant C++ library.  But it really wants an expanded build.

| I do, however, think the rationale is important to document what we are doing
| so that if someone comes along later and wants to understand why something was
| done, it's written down.

Fully agreed.

| Thanks for the other rm bugs.  I'll process the group next time I do removals.

I appreciate your work in the release, as I have for years. My ~ 180 other
packages should be in good shape.

Dirk

| Scott K
| x[DELETED ATTACHMENT signature.asc, application/pgp-signature]
#1033410#42
Date:
2023-03-29 20:36:12 UTC
From:
To:
Hey Scott,

Thanks for removing the Python and R client packages for TileDB, I think you
can remove the main one as the sole other (optional) reverse dependency
updated its build -- genomicsdb via #1033456.  If I overlooked any other
contigency, is there anything else I can help with?

Cheers, Dirk
#1033410#45
Date:
2023-03-29 20:36:12 UTC
From:
To:
Hey Scott,

Thanks for removing the Python and R client packages for TileDB, I think you
can remove the main one as the sole other (optional) reverse dependency
updated its build -- genomicsdb via #1033456.  If I overlooked any other
contigency, is there anything else I can help with?

Cheers, Dirk
#1033410#50
Date:
2023-03-31 20:47:31 UTC
From:
To:
We believe that the bug you reported is now fixed; the following
package(s) have been removed from unstable:

libtiledb-dev |   2.15.0-1 | amd64, arm64, armel, armhf, i386, mips64el, mipsel, ppc64el, s390x
libtiledb2.13 |   2.15.0-1 | amd64, arm64, armel, armhf, i386, mips64el, mipsel, ppc64el, s390x
    tiledb |   2.15.0-1 | source

------------------- Reason -------------------
ROM as full builds not possible within Debian
----------------------------------------------

Note that the package(s) have simply been removed from the tag
database and may (or may not) still be in the pool; this is not a bug.
The package(s) will be physically removed automatically when no suite
references them (and in the case of source, when no binary references
it).  Please also remember that the changes have been done on the
master archive and will not propagate to any mirrors until the next
dinstall run at the earliest.

Packages are usually not removed from testing by hand. Testing tracks
unstable and will automatically remove packages which were removed
from unstable when removing them from testing causes no dependency
problems. The release team can force a removal from testing if it is
really needed, please contact them if this should be the case.

We try to close bugs which have been reported against this package
automatically. But please check all old bugs, if they were closed
correctly or should have been re-assigned to another package.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1033410@bugs.debian.org.

The full log for this bug can be viewed at https://bugs.debian.org/1033410

This message was generated automatically; if you believe that there is
a problem with it please contact the archive administrators by mailing
ftpmaster@ftp-master.debian.org.

Debian distribution maintenance software
pp.
Scott Kitterman (the ftpmaster behind the curtain)
#1033410#55
Date:
2023-04-06 12:39:16 UTC
From:
To:
Hi Scott,

Thanks for the continued attention to the tiledb removal. I appreciate it!

There is a build remaining in experimental.  Can / should that go too?  Does
it need another bug report?  Or is experimental seen as 'outside the
wheelhouse' ?

Best regards,  Dirk
#1033410#58
Date:
2023-04-06 12:39:16 UTC
From:
To:
Hi Scott,

Thanks for the continued attention to the tiledb removal. I appreciate it!

There is a build remaining in experimental.  Can / should that go too?  Does
it need another bug report?  Or is experimental seen as 'outside the
wheelhouse' ?

Best regards,  Dirk
#1033410#61
Date:
2023-04-06 12:42:10 UTC
From:
To:
Please file another bug for that.

Scott K
#1033410#64
Date:
2023-04-06 17:03:52 UTC
From:
To:
On 6 April 2023 at 12:42, Scott Kitterman wrote:
| Please file another bug for that.

Sure thing -- now done in #1034024.

Dirk

| Scott K
|
| On April 6, 2023 12:39:16 PM UTC, Dirk Eddelbuettel <edd@debian.org> wrote:
| >
| >Hi Scott,
| >
| >Thanks for the continued attention to the tiledb removal. I appreciate it!
| >
| >There is a build remaining in experimental.  Can / should that go too?  Does
| >it need another bug report?  Or is experimental seen as 'outside the
| >wheelhouse' ?
| >
| >Best regards,  Dirk
| >