These source packages embed nodejs module serialize-javascript
without offering it as virtual binary package:

 node-compression-webpack-plugin
 node-copy-webpack-plugin
 node-uglifyjs-webpack-plugin

Please embed in only one source package provided as versioned virtual package,
and drop in other source packages instead depending on the virtual package.

Severity raised since the lack of virtual package blocks upgrading node-terser.


 - Jonas
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAl/Iz5AACgkQLHwxRsGg
ASG6Dg//V6dsWFrsdb1fOvjtGMZNv/hpab1B9x9tP4aJdqm42r4VNlnd3m2Di8vQ
OKPUZBMmCgLbsWzEILkNGy33kg+FfCvOWiHTLY+dZHxKIkfn6qXU/crEqoGFI4Id
CQHgGBXgN10teuclHyjCX3kmYLFYc9QdzMRf4dK+v3+EP0LXAMehu3WqMhifewD5
CNdgU327gmtx5Jv6kaeILSFLx+jAZ+hhCetBfQHTTkWhmx2QZmPSjjv8Lb6piy6U
2GZKths9sqBOFTeAhByqO+Tli5qYgcvfrgGVl2avKBwKGEAF+/jXPw8cq1Jqojhx
Hdb2ZFfEQWNZ9LNt8VThyuuOa83wKwLYmx8PFF/rCf5hESKAt6H61uF4KqbIfyhn
EvW7FIhkY7rBIVgFfA+X2ol9JtpJ2vBFyJC3OS23Ymkh8KEv0Pozl/20/Xztizef
l23219Jetv9b+4gDaF1gZZEWZTtpd9NRgDfKpnp21z/8Q+oqQmJtpkaa8fVYTkPo
2NIgSvtN27YAofG3xbfaKJkZ1zhFO0wv5qkDow3E8Tt6QOctkxc6IqC8NOyY8VDX
5IIdRqiUnC2ivw2kGV5NPs1xyGSnuWgrKM5AvSG6FOAN9VVXR62xv2ri+3w+2L+k
xWdf1Rrw1aEnC3iPWVaqopBmIhQ8B1oetd396vECKtyonSGxpTw=
=DRO0
-----END PGP SIGNATURE-----

Le 03/12/2020 à 12:44, Jonas Smedegaard a écrit :
node-compression-webpack-plugin,node-copy-webpack-plugin,node-uglifyjs-webpack-plugin

Hi,

for now, dh-sequence-nodejs adds a "Provides" item for modules installed
in root nodejs directories. Do we want to declare a "node-foo" for
submodules (installed in a <package>/node_modules directory) ?

Cheers,
Xavier

Le 03/12/2020 à 14:24, Xavier a écrit :

Note that the future lintian database (classification tags) will permit
to see node modules everywhere.

Quoting Xavier (2020-12-03 14:35:25)

[...]
for each embedded Nodejs module, properly versioned with the module's
own version as first segment then "~" then source package version.

I cannot see a reason for *any* embedded Nodejs module to stay hidden,
but if someone comes up with some exceptional cases for that, then the
reasoning should be explicitly documented in either README.source or
README.Debian (and possibly in long description too).

Everywhere?

I should be able to declare this in some other package:

   Build-Depend: node-serialize-javascript (>= 5)

That is not possible today, because no packages provide that name
(despite 3 packages containing some version of it).

That will be possible if tommorrow one of those packages adds this:

  Provides: node-serialize-javascript (= 5.0.1)

That will *not* be possible, however, if tommorrow dh-sequence-nodejs
automatically adds this for all three packages:

  Provides: $embeddedmodule (= ${embeddedmodule:Version})

...because then it is not deterministic which of them has priority.


 - Jonas

Le 03/12/2020 à 15:12, Jonas Smedegaard a écrit :

I chose that because such modules are not directly usable using a
`require("foo")`, but I can change
each time it finds a node_module/foo/package.json or
<main nodejs>/foo/package.json or <main nodejs/foo.js. Then we will be
able to see nodejs embedded module in all Debian packages.

NB2, you can also take a look at
https://lintian.debian.org/tags/nodejs-module-not-declared.html : it
shows node module installed in nodejs main dirs (not in node_modules/
for now).

If we decide to change this ~policy, nodejs-module-not-declared should
also be updated.

But in this case, we will have some not-directly-usable node-* virtual
packages.

It does (see our discussion about acorn) but only for main installed
modules (and if DD didn't omit ${nodejs:Provides} of course)
Xavier

NB: maybe I misunderstood part of your explanations and then my
explanations are perhaps out of subject

Quoting Xavier (2020-12-03 15:44:48)

I am less interested in why historically some pattern was chosen.

My interest is in what pattern to use going forward.

Code in package have dependencies on code in other packages.

We need to be able to declare dependencies between pieces of code.

For JavaScript, code is grouped as "Nodejs modules".

A a concrete example, Nodejs module rollup-plugin-terser depends on
Nodejs module serialize-javascript.

Going forward (regardless of why historically some other pattern was
chosen), Debian package node-rollup-plugin-terser needs to be able to
express a build-dependency on Debian package providing the Nodejs module
serialize-javascript.

Lintian can help check if a package is valid.

Lintian cannot make a package declare as virtual Debian packages the
Nodejs modules it has embedded.

A web page (generated by lintian or written any other way) cannot make a
package declare as virtual Debian packages the Nodejs modules it has
embedded.

I am pretty sure that hiding generally usable embedded code violates a
"should" somewhere in Debian Policy.

Therefore, if Debian JavaScript Policy says that generally useful
modules should be *hidden* instead of provided as virtual packages, then
I consider Debian JavaScript Policy broken.

Why not-directly-usable?

Obviously we should not _only_ declare "Provides:" but also make sure
that the code is actually placed in the correct location below
/usr/share/nodejs and check testsuite and establish autopkgtest and all
the other things that is required for packaging code.

Embedded code is not magically excluded from getting properly packaged.

What I talk about is that generally usable embedded node modules should
be "installed modules".

Hope I made it clearer now... :-)


 - Jonas

Le 03/12/2020 à 16:36, Jonas Smedegaard a écrit :

If so, lintian should reports "error", not info/warning when a JS lib is
embedded. Ftpmasters didn't reject such packages (see jquery copies for
example).

Anyway, let's decide.

I'm not in favor of your proposal: if I need a node-very-few-module
which is provided by a package with many dependencies, I'm not happy to
depend on it. To apply that, we should then choose with precaution in
which package we will embed each code. Packaging JS is already a hudge
work, I'm not sure we need more complexity.

@JS-Team, does anyone have any other opinion? Else which option do you
choose? I'll update pkg-js-tools with your choice of course.

I agree with yadd here. We should not complicate js packaging more than it already is.

Quoting Xavier (2020-12-03 17:33:18)

Ideally lintian should identify and report all errors.

Ideally all packaging work could be automated.

Reality is slightly different. though.

I am not targeting you, nor am I targeting dh-sequence-nodejs, nor am I
targeting lintian, nor am I targeting JavaScript Team Policy.  You
brought those up in this _discussion_ about a package that I filed a
bugreport against.

Decide on what exactly?  My way or the highway?  Your way or the highway?


I notice you added [JS Policy] to the subject, but this is bug#976331
which is specifically about three Debian packages all embedding Nodejs
module serialize-javascript without any of them providing
serialize-javascript as a binary package.


 - Jonas

Quoting Pirate Praveen (2020-12-03 18:07:56)

We should make our js packaging *less* complicated.

More importantly, we should make js *maintenance* less complicated!

But most importantly, we should obey all "should"s in Debian policy:
https://www.debian.org/doc/debian-policy/ch-source.html#embedded-code-copies


 - Jonas

Le 03/12/2020 à 18:21, Jonas Smedegaard a écrit :

One other time I may have misunderstood; if so, I'm very confused.

If ftpmasters ask us to minimize package number by embedding
to-little-modules and if then we decide to publish them as separated
binary packages, I don't succeed to understand the benefit. Then we
should return back to previous policy: one source = one package?

Quoting Xavier (2020-12-03 18:42:17)

My understanding is that ftpmasters dislike small *source* packages.

Small source packages is a burden in *every* package tracking in Debian.

Small binary packages is also a burden in *some* package tracking.

Zero-size binary packages is also a burden in *some* package tracking,
but I don't hear ftpmasters complain about task packages.


This bug 976331 is *not* about repackaging embedded modules as separate
*source* packages, but only about exposing embedded modules as *binary*
packages - either virtual or real ones.


 - Jonas

Quoting Xavier (2020-12-03 17:33:18)

I don't recognize the above as being something I proposed (and I am not
really sure what it says).

Maybe there is a misunderstanding somewhere, and we do not really
disagree as much?

Please, let's try make sure we at least agree what we disagree on :-)

I do _not_ blame you on linguistic failures here, Xavier - it might
_seem_ like I am better at english than you but I might very well be
worse than you in explaining myself and understaning what you write...


 - Jonas

I suggest you embed serialize-javascript in node-terser and if you wish
you can expose it as virtual package via Provides.

Connecting unrelated packages together is making things complicated for
transitions and also increases installed packages size.

Also 'should' is not 'must' in policy.

Quoting Pirate Praveen (2020-12-03 20:09:05)

Yes, a possible solution to this issue is to introduce a 4th embedding.

That might be your understanding of keeping packaging simple.  Not mine.

That might be your understanding of team maintenance.  Not mine.

Yes, a possible solution to this issue is to introduce the embedded
module in another package _instead_ of these three.  An example of a
package providing tiny modules are real binary packages (which ftpmaster
evidently had no problem accepting) is is node-file-entry-cache which
also provides related packages node-flat-cache and node-write.  An
example of a package providing a tiny module as virtual package is
node-uuid also providing virtual package node-types-uuid.

Yes, another possible solution to this issue is to introduce the
embedded module as part of a batch of related related modules, none of
them being the "main" one (depending on the meaning of "main").  An
example of that (evidently also acceptable to ftpmasters) is
jsbundle-web-interfaces providing binary packages node-auth-header,
node-standard-error, node-standard-http-error and
node-webidl-conversions.

And apples are not oranges.

If you meant to put words into my mouth, then please don't.


 - Jonas

Le 03/12/2020 à 19:17, Jonas Smedegaard a écrit :
ftpmaster rejection since you pushed node-serialize-javascript).

But: I was able to upload a lot of packages this year because I
automatized many things. So splitting all mixed packages means manually
regenerating debian/control, debian/rules, debian/*.install,... This
means less uploads, more obsolescence and then less security (and also
less interest in doing such manual stuff).

Quoting Xavier (2020-12-03 21:19:48)

[,,,]
https://www.debian.org/doc/debian-policy/ch-source.html#embedded-code-copies

Great that you develop tools to maintain packages more efficiently and
more automated.  If done in compliance with Debian Policy, that is.

Do you say that it is not possible to automate packaging with embedded
modules provided as virtual or real packages?  Why do you think you can
only develop efficient automating routines with hidden modules?

Here is a concrete example of a minimal (manual) that I propose to do
for one of the three packages involved in this bugreport (and then have
the other 2 packages drop the embedded module ans instead depend on the
now accessible-as-package-name module):
https://salsa.debian.org/js-team/node-cosmiconfig/-/commit/125869f9

I don't understand how that cannot be automated.

I do see how it requires coordination across packages, to decide which
one source package should contain the embedded module, instead of *all*
source packages containing *duplicate* modules - but that coordination
work is *necessary work, not optional.  It is not ok to package the
Apache2 server with embedded zlib library, regardless if it is easier to
automate without unentangling things done too tight upstream. Nodejs is
no different from that!

ftpmaster wants to avoid too tiny source packages, but ftpmaster does
not want duplicate code.


 - Jonas

Le 06/12/2020 à 02:14, Jonas Smedegaard a écrit :
 * components not declared in debian/nodejs/root_modules: installed in a
   subdirectory and not "provides"
 * components listed in debian/nodejs/root_modules: installed in main
   nodejs directory and added in ${nodejs:Provides}

So when a pkg-js-tools package does not export components, just to
insert '*' in debian/nodejs/root_modules and add a "Provides:
${nodejs:Provides}"

My way is more simple and is already automatized. The problem was not
here but in separated binary packages. This is partially automatized,
see jest.

No you're wrong here, they accepted a lot of embedded copies for JS (see
Jquery for example), not for C.

Also they don't want too many little binary packages (not only source).

Quoting Xavier (2020-12-06 07:59:25)

Ok, so you are saying that current automation _is_ effective for virtual
binary packages but handling _real_ binary packages need manual work.

Sorry, I was unclear.  Let me try rephrase to clarify:

ftpmaster strongly dislikes (i.e. they will reject) too tiny source
packages, but ftpmaster does not actively encourage (even if they might
tolerate) duplicate code.

My point was not that ftpmaster rejects duplicate code (they might in
severe cases, but that is not really their task to judge on).

My point was, instead, that it is wrong to interpret the fact that
ftpmaster rejects tiny source packages as an endorsement of duplicate
source through _repeated_ embedding of same module.

It is not the task of ftpmaster to check _all_ Policy Violations - and
even if it was then a package not rejected is not a proof that the
package is optimally aligned with Debian Policy.


 - Jonas

"One line of code (=exaggeration for simple module) should be always
embedded independent of the number of packages that depend on it." -
Thorsten Alteholz / ftp master
https://alioth-lists.debian.net/pipermail/pkg-javascript-devel/2018-September/027873.html

Quoting Pirate Praveen (2020-12-06 12:08:29)

You provide a quote and share a link to its source, but do not elaborate
what _you_ want to say by sharing that quote.

What you quote, and the longer message by Thorsten, makes sense to me.

If you are trying to prove that ftpmaster _do_ explicitly endorse
duplicate source through _repeated_ embedding of same module, then where
do they say it is ok and good and encouraged and endorsed to embed
_same_ code _multiple_ times?

"constant four = 2 + 2" should always be embedded - e.g. always embedded
in src:node-sillymodules where it was provided as virtual package
node-four.


 - Jonas

Dear submitter,

as the package node-uglifyjs-webpack-plugin has just been removed from the Debian archive
unstable we hereby close the associated bug reports.  We are sorry
that we couldn't deal with your issue properly.

For details on the removal, please see https://bugs.debian.org/1014270

The version of this package that was in Debian prior to this removal
can still be found using http://snapshot.debian.org/.

Please note that the changes have been done on the master archive and
will not propagate to any mirrors until the next dinstall run at the
earliest.

This message was generated automatically; if you believe that there is
a problem with it please contact the archive administrators by mailing
ftpmaster@ftp-master.debian.org.

Debian distribution maintenance software
pp.
Thorsten Alteholz (the ftpmaster behind the curtain)