- Package:
- rapid-photo-downloader
- Source:
- rapid-photo-downloader
- Submitter:
- Antoine Beaupre
- Date:
- 2022-06-03 02:39:06 UTC
- Severity:
- normal
I'm not sure about this, but it seems to me that `upgrade.py` and `qrc_resources.py` ship binary data as source code. This is normally a red flag in Debian packages, because source code should typically be text file that are humanly understandable, or at least usable with free software tools. Basing myself on the excellent investigation done by Tino Mettler: https://salsa.debian.org/debian/rapid-photo-downloader/-/merge_requests/2#note_307363 ... i t seems like the `upgrade.py` file is fairly innocuous: it's actually an encoded ZIP file that has .mo files generated from the provided .po files. I'm not sure that needs to be removed, as there is probably an obvious source for those. The other file, `qrc_resources.py`, is more problematic. It bundles binary data like images and those don't seem to have an associated source in the source code. It's unclear if that file could be redistributed as is, as it's clearly not modifiable, and would possibly be a license violation. It seems like the source images for that file are missing from the upstream source as well, crucially. Normally, we should be able to recompile that file from source, but because those images are missing, it's not possible. I'm not an expert on those issues as I used to be, so I'm not sure about all this, but it seemed important to flag this as an issue on the package.
Hi, the images used to generate qrc_resources.py are present in the upstream git repository, but not in the release tarballs. Once they are copied into the source directory of the Debian package, qrc_resources.py can be regenerated using pyrcc5 (the PyQT resource compiler). I opened an issue in the upstream git repository and described the nature of this bug. I also asked if it is possible to provide signed release tarballs with the images included. Regards, Tino
Hi, the images used to generate qrc_resources.py are present in the upstream git repository, but not in the release tarballs. Once they are copied into the source directory of the Debian package, qrc_resources.py can be regenerated using pyrcc5 (the PyQT resource compiler). I opened an issue in the upstream git repository and described the nature of this bug. I also asked if it is possible to provide signed release tarballs with the images included. Regards, Tino
Maybe a solution that wouldn't involve upstream would be to change the way we generate the tarball, from git instead of the upstream tarball. After all, if we're going to rebuild thing from scratch ourselves anyways...
Maybe a solution that wouldn't involve upstream would be to change the way we generate the tarball, from git instead of the upstream tarball. After all, if we're going to rebuild thing from scratch ourselves anyways...
Maybe a solution that wouldn't involve upstream would be to change the way we generate the tarball, from git instead of the upstream tarball. After all, if we're going to rebuild thing from scratch ourselves anyways...
Am Wed, May 04, 2022 at 13:03:19 -0400 schrieb Antoine Beaupré: [...] I also think that would be the way to go, as the upstream author is currently fighting with health issues. https://github.com/damonlynch/rapid-photo-downloader/issues/73#issuecomment-1117546807 Regards, Tino
Am Wed, May 04, 2022 at 13:03:19 -0400 schrieb Antoine Beaupré: [...] I also think that would be the way to go, as the upstream author is currently fighting with health issues. https://github.com/damonlynch/rapid-photo-downloader/issues/73#issuecomment-1117546807 Regards, Tino
Am Wed, May 04, 2022 at 13:03:19 -0400 schrieb Antoine Beaupré: [...] I also think that would be the way to go, as the upstream author is currently fighting with health issues. https://github.com/damonlynch/rapid-photo-downloader/issues/73#issuecomment-1117546807 Regards, Tino
Oh dear yeah, screw that. Let's not bother that poor upstream already. :) Let's make a new tarball with the upstream git repo instead, can you try that? Thanks! a.
Hi, the upstream git repository has no release tags, so I guess that we have to drop uscan support and create the upstream source archive with a selfmade script. Regards, Tino
Hi, the upstream git repository has no release tags, so I guess that we have to drop uscan support and create the upstream source archive with a selfmade script. Regards, Tino
Hi, the upstream git repository has no release tags, so I guess that we have to drop uscan support and create the upstream source archive with a selfmade script. Regards, Tino
we could lay down release tags ourselves if we can figure out what those are...
we could lay down release tags ourselves if we can figure out what those are...
we could lay down release tags ourselves if we can figure out what those are...
Hi, that would be an option. I'm currently crafting a script that creates a tarball. Just removing upgrade.py doesn't work, because it is referenced in the translations and that will cause a build errors. Replacing it with an empty file works, and should not affect Debian users. Regards, Tino
Hi, that would be an option. I'm currently crafting a script that creates a tarball. Just removing upgrade.py doesn't work, because it is referenced in the translations and that will cause a build errors. Replacing it with an empty file works, and should not affect Debian users. Regards, Tino
Hi, that would be an option. I'm currently crafting a script that creates a tarball. Just removing upgrade.py doesn't work, because it is referenced in the translations and that will cause a build errors. Replacing it with an empty file works, and should not affect Debian users. Regards, Tino
maybe we could just use a +ds tarball then. add the files to redact in debian/copyright and then uscan will just do the right thing for us, no need to worry about the git stuff.
Am Wed, May 04, 2022 at 15:17:05 -0400 schrieb Antoine Beaupré:
[...]
We need the images/ and tips/ folders from the upstream git repository.
This script would work. I already prepared a package using the +dfsg upstream
source generated by the script. The resources are compiles during
build, the result looks good.
#!/bin/sh -ex
git reset --hard $1
VERSION=$(sed '/^-------------/,$d' CHANGES.md | tail -n 1 | cut -d' ' -f1)
echo > upgrade.py
git rm raphodo/qrc_resources.py
git commit -m "Remove source files with embedded binary blobs"
git archive HEAD --prefix=rapid-photo-downloader-$VERSION/ -o ../rapid-photo-downloader_${VERSION}+dfsg.orig.tar.xz
Regards,
Tino
Am Wed, May 04, 2022 at 22:12:20 +0200 schrieb Tino Mettler:
I forgot to git add the upgrade.py. Furthermore, Damon mentioned
install.py. And yes, it contains another binary blob (the same ZIP
archive as upgrade.py, I guess).
Here is an improved script:
#!/bin/sh -ex
git reset --hard $1
VERSION=$(sed '/^-------------/,$d' CHANGES.md | tail -n 1 | cut -d' ' -f1)
echo > upgrade.py
echo > install.py
git add upgrade.py install.py
git rm raphodo/qrc_resources.py
git commit -m "Remove source files with embedded binary blobs"
git archive HEAD --prefix=rapid-photo-downloader-$VERSION/ -o ../rapid-photo-downloader_${VERSION}+dfsg.orig.tar.xz
Regards,
Tino
Hi, I pushed a branch that uses the dfsg source generated by the above script. It contains one additional change to generate the qrc_resources.py file. https://salsa.debian.org/scorpi-guest/rapid-photo-downloader/-/commits/debian/unstable-0.9.33+dfsg
Hi, regarding the comments in the code, the binary blob in install.py and upgrade.py can be generated during build time using tasks/generate.py. However, both files are not part of the binary package so I see no point in doing that. So I think replacing them with empty files in the dfsg source tarball is okay. Regards, Tino
Agreed.
Hi Antoine, I'm currently thinking about missing bits for a 0.9.33 release. I didn't switch a package to dfsg source before, so I'm new to that. It looks like it should be documented in debian/copyright, e.g. why the source tarball was repacked and what is the reason. There should be a changelog entry, which also refers to this bug. Furthermore, I guess that the procedure to create the upstream tarball should be described, by providing the script that I used. This could be done in a README.Packaging. Please let me know if I am missing something. Regards, Tino
Hi Antoine, I'm currently thinking about missing bits for a 0.9.33 release. I didn't switch a package to dfsg source before, so I'm new to that. It looks like it should be documented in debian/copyright, e.g. why the source tarball was repacked and what is the reason. There should be a changelog entry, which also refers to this bug. Furthermore, I guess that the procedure to create the upstream tarball should be described, by providing the script that I used. This could be done in a README.Packaging. Please let me know if I am missing something. Regards, Tino
That all sounds about right to me!
Typically, the script lives in debian/rules, as a target
("get-orig-source"). This has actually been deprecated by Debian policy
4.1.4, interestingly:
https://www.debian.org/doc/debian-policy/upgrading-checklist.html#version-4-1-4
... but is still mentioned in the developer's reference:
https://www.debian.org/doc/manuals/developers-reference/best-pkging-practices.en.html#repackaged-upstream-source
In general, I think that means *maybe* we could try harder to get uscan
to do the right thing here, but I understand if it seems impossible. :)
The documentation about that stuff can also live in README.source:
https://www.debian.org/doc/debian-policy/ch-source.html#source-package-handling-debian-readme-source
I think that's about right, thanks for the hard work!
That all sounds about right to me!
Typically, the script lives in debian/rules, as a target
("get-orig-source"). This has actually been deprecated by Debian policy
4.1.4, interestingly:
https://www.debian.org/doc/debian-policy/upgrading-checklist.html#version-4-1-4
... but is still mentioned in the developer's reference:
https://www.debian.org/doc/manuals/developers-reference/best-pkging-practices.en.html#repackaged-upstream-source
In general, I think that means *maybe* we could try harder to get uscan
to do the right thing here, but I understand if it seems impossible. :)
The documentation about that stuff can also live in README.source:
https://www.debian.org/doc/debian-policy/ch-source.html#source-package-handling-debian-readme-source
I think that's about right, thanks for the hard work!
That all sounds about right to me!
Typically, the script lives in debian/rules, as a target
("get-orig-source"). This has actually been deprecated by Debian policy
4.1.4, interestingly:
https://www.debian.org/doc/debian-policy/upgrading-checklist.html#version-4-1-4
... but is still mentioned in the developer's reference:
https://www.debian.org/doc/manuals/developers-reference/best-pkging-practices.en.html#repackaged-upstream-source
In general, I think that means *maybe* we could try harder to get uscan
to do the right thing here, but I understand if it seems impossible. :)
The documentation about that stuff can also live in README.source:
https://www.debian.org/doc/debian-policy/ch-source.html#source-package-handling-debian-readme-source
I think that's about right, thanks for the hard work!
Hi Antoine, I just discussed this a bit on #debian-mentors in IRC. Conclusion: - autogenerated files in the upstream tarball might be okay, if they are not used for the binary packages (upgrade.py, install.py) or if they are regenerated during build time (qrc_resources.py) - the details about this should be documented in debian/copyright - the missing images to generate the qrc_resources.py file can be put into debian/missing-sources - we can discuss this issue again with the upstream author once he has recovered - it is unlikely that there will be a new upstream release before this When we follow this, we don't need to repack the upstream source, just add the images to debian/missing-sources and document all that in debian/copyright. What do you thing about this? Regards, Tino
Hi Antoine, I just discussed this a bit on #debian-mentors in IRC. Conclusion: - autogenerated files in the upstream tarball might be okay, if they are not used for the binary packages (upgrade.py, install.py) or if they are regenerated during build time (qrc_resources.py) - the details about this should be documented in debian/copyright - the missing images to generate the qrc_resources.py file can be put into debian/missing-sources - we can discuss this issue again with the upstream author once he has recovered - it is unlikely that there will be a new upstream release before this When we follow this, we don't need to repack the upstream source, just add the images to debian/missing-sources and document all that in debian/copyright. What do you thing about this? Regards, Tino
That makes total sense to me.
That makes total sense to me.
That makes total sense to me.
We believe that the bug you reported is fixed in the latest version of rapid-photo-downloader, which is due to be installed in the Debian FTP archive. A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to 1010570@bugs.debian.org, and the maintainer will reopen the bug report if appropriate. Debian distribution maintenance software pp. Tino Mettler <tino+debian@tikei.de> (supplier of updated rapid-photo-downloader package) (This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing ftpmaster@ftp-master.debian.org) Format: 1.8 Date: Tue, 17 May 2022 22:36:26 +0200 Source: rapid-photo-downloader Architecture: source Version: 0.9.33-1 Distribution: unstable Urgency: medium Maintainer: Dr. Tobias Quathamer <toddy@debian.org> Changed-By: Tino Mettler <tino+debian@tikei.de> Closes: 1010570 Changes: rapid-photo-downloader (0.9.33-1) unstable; urgency=medium . [ Tino Mettler ] * New upstream version 0.9.33 * Add Depends: for python3-showinfilemanager * Add Depends: for ifuse and libimobiledevice-utils, needed as of 0.9.29 * Remove Recommends: for libraw-bin, as it is not used anymore * Remove 0002-Disable-optional-rawkit-usage.patch, applied upstream * Don't generate analyze-pv-structure.1 as it was removed upstream * Use CHANGES.md, CHANGES.rst was dropped by upstream * Disable dh_auto_test, it causes a build failure * Add myself to Uploaders: * Update debian/copyright . [ Debian Janitor ] * Re-export upstream signing key without extra signatures. * Set upstream metadata fields: Repository, Repository-Browse. . [ Tino Mettler ] * Set upstream metadata fields: Bug-Database, Bug-Submit * Update Standards-Version to 4.6.0.1, no changes needed * Add images from upstream git repository to regenerate qrc_resources.py * Add pyqt5-dev-tools to build dependencies for pyrcc5 * Regenerate qrc_resources.py (Closes: #1010570) * Replace override_dh_auto_clean actions with debian/clean * Add details about source files with generated binary data in d/copyright Checksums-Sha1: 4c743c5d785b8019efb25628a466aed5237bfdb6 2200 rapid-photo-downloader_0.9.33-1.dsc 38ebfc50e64cfce2aace92473843aa3e5f4a422d 8501134 rapid-photo-downloader_0.9.33.orig.tar.gz c9b32dcc8ace4f2ff583df8a19e6af4c0cd09bd0 833 rapid-photo-downloader_0.9.33.orig.tar.gz.asc 2f1b09c586e004c591ff3e4190448771b954952c 3479820 rapid-photo-downloader_0.9.33-1.debian.tar.xz f43832a1f3d6a6426b022d18210b3157e8b745ec 10822 rapid-photo-downloader_0.9.33-1_amd64.buildinfo Checksums-Sha256: a90c7f246e787bc8293e2a40f19ee4dfa1daec05412dd9fba3fc8e409794fc7f 2200 rapid-photo-downloader_0.9.33-1.dsc 57e35c9f35f8c4bae96ca246ad8c1b917c6cff0a44bb550b4bb4a31936da0cc2 8501134 rapid-photo-downloader_0.9.33.orig.tar.gz 076defd4728b178a06f8d931bd368bdd6e33a8c62c89645a2aff7543501a5934 833 rapid-photo-downloader_0.9.33.orig.tar.gz.asc 58c289d68122c6c61c82fa2c220dcc7080750e544268bb1bd7ba71182566132f 3479820 rapid-photo-downloader_0.9.33-1.debian.tar.xz 13bd3cd5582d8a0b2d0ae71da84f5861572ccb751e5443e48e0decb47a1e162c 10822 rapid-photo-downloader_0.9.33-1_amd64.buildinfo Files: 1d9a8f3eb0143a3d17f11d4236475f9d 2200 graphics optional rapid-photo-downloader_0.9.33-1.dsc c440fc1fb00109b5e5f67433081a1b1a 8501134 graphics optional rapid-photo-downloader_0.9.33.orig.tar.gz 7f8043816227364fa703227266fc03e3 833 graphics optional rapid-photo-downloader_0.9.33.orig.tar.gz.asc 7da53e9affb8488977096000c66dfd02 3479820 graphics optional rapid-photo-downloader_0.9.33-1.debian.tar.xz ddd5986d837500f2e2abccb5fe8a804b 10822 graphics optional rapid-photo-downloader_0.9.33-1_amd64.buildinfo -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEexZCBNCWcjsBljWrPqHd3bJh2XsFAmKZZWEACgkQPqHd3bJh 2XuxlQf9GDCWlpNYddUjarZHK6huRIu7O1NQLKuEti/dK8OzE0OM2FFr4EO0TC5r nraBIai29taG09+1e0aTH2BMzNnB3s2CMl++ptNI5YzDIy3jgcWsjupPvNekpciu E+Mq+n0+0M6sdRE4Sli4LEMRv0FmOy/sTowD+EviP6JnsKoLPb9udcqCcFPOGvIs xHAMTtdoeexMkwlZaivqgcgwU2Aj075f98gjOL7xrpBcjWLIP1HtknHQRz0qsRCo H5LSgNgyQwih1koyvA90/dCHqz0JAdkNwWrArzdT9LcVbLpJNAuS/TbqO0QGTQ0Q bVYLhBPMybqN0dJw2RKK0lO79GT9kw== =0i5d -----END PGP SIGNATURE-----