When users install `.deb` packages that are not available in Debian via a `.chroot` hook (such as `1password`), the `./live/filesystem.packages-remove` file in the generated ISO uninstalls the packages installed via `.chroot` after the system is installed.

This was not the case until this action (which was added 12 years ago) became active a year ago:
- https://salsa.debian.org/installer-team/live-installer/-/commit/ad0ebaad
- https://salsa.debian.org/installer-team/live-installer/-/commit/ca1e1706757ecc9a4cf1fa5c637d5a9b513acee6

Because certain packages cannot be installed without `.chroot`
hooks, I recommend reverting this change. It was discussed that users should drop their `.deb` packages to the `packages.chroot` directory instead, as that is the intended way. However, certain programs such as `1Password`, `docker` (from Docker's repositories), ProtonVPN, etc. only use the `.deb` packages to add their repos to the system and not install packages, which require users to `sudo apt update && sudo apt install -y <package(s)>`.

This was detected during a Kali Linux live-build:
- https://gitlab.com/kalilinux/build-scripts/live-build-config/-/issues/61
- https://gitlab.com/kalilinux/build-scripts/live-build-config/-/merge_requests/33

On 04/02/2024 17:41, Roland Clobus wrote:
...

And I'm certain that there is a more secure way, that ensures that only
the package called '1password' will come from this repository.
The bug report was based on a kali version of live-build, so I assume
you know better than me how to do so.
Please add such command to the bug report, so I can update the
live-manual to address such use case.

With kind regards,
Roland Clobus

Hmm. My first reply didn't make it, resending...

Hello Arszilla,

I still think that removing all live-related packages in the installer
is a good idea. The processing of 'live/filesystem.packages-remove'
shows where the package management system has been circumvented.
When the .deb file is provided in either config/packages.chroot or
config/packages, it will not be installed per default, because the name
'1password' is not a package name that is known.

I then ran live-build with '--interactive=true' (which has a similar
effect as writing a hook for config/hooks/normal) and installed the
package with 'dpkg -i 1password-latest.deb' and 'apt --fix-broken install'

This indeed resulted in the file
'binary/live/filesystem.packages-remove', which will remove 1password
after installation.

But a proper registration of the foreign repository will result in
1password to be installed in the chroot, and not to be removed by the
installer.
The commands below were taken from the 1password website:
https://support.1password.com/install-linux/
---
lb config --keyring-packages ca-certificates
mkdir -p config/includes.chroot_before_packages/etc/apt/sources.list.d
mkdir -p
config/includes.chroot_before_packages/etc/debsig/policies/AC2D62742012EA22
mkdir -p config/includes.chroot_before_packages/usr/share/keyrings

echo 'deb [arch=amd64
signed-by=/usr/share/keyrings/1password-archive-keyring.gpg]
https://downloads.1password.com/linux/debian/amd64 stable main' >
config/includes.chroot_before_packages/etc/apt/sources.list.d/1password.list
curl -sS
https://downloads.1password.com/linux/debian/debsig/1password.pol >
config/includes.chroot_before_packages/etc/debsig/policies/AC2D62742012EA22/1password.pol
curl -sS https://downloads.1password.com/linux/keys/1password.asc | gpg
--dearmor --output 
config/includes.chroot_before_packages/usr/share/keyrings/1password-archive-keyring.gpg

echo "1password" > config/package-lists/1password.list.chroot
---

While preparing this mail, I had to make a local hack to ensure that
live-build would continue (adding 'apt-get update' in
chroot_install-packages, but this seems to be a proper way to handle the
foreign .deb file

Since the 1password.deb file properly registers within the package
management system, this ticket becomes a 'works for me'.

With kind regards,
Roland Clobus

Hello Arszilla,

I still think that removing all live-related packages in the installer
is a good idea. The processing of 'live/filesystem.packages-remove'
shows where the package management system has been circumvented.
When the .deb file is provided in either config/packages.chroot or
config/packages, it will not be installed per default, because the name
'1password' is not a package name that is known.

I then ran live-build with '--interactive=true' (which has a similar
effect as writing a hook for config/hooks/normal) and installed the
package with 'dpkg -i 1password-latest.deb' and 'apt --fix-broken install'

This indeed resulted in the file
'binary/live/filesystem.packages-remove', which will remove 1password
after installation.

But a proper registration of the foreign repository will result in
1password to be installed in the chroot, and not to be removed by the
installer.
The commands below were taken from the 1password website:
https://support.1password.com/install-linux/
---
lb config --keyring-packages ca-certificates
mkdir -p config/includes.chroot_before_packages/etc/apt/sources.list.d
mkdir -p
config/includes.chroot_before_packages/etc/debsig/policies/AC2D62742012EA22
mkdir -p config/includes.chroot_before_packages/usr/share/keyrings

echo 'deb [arch=amd64
signed-by=/usr/share/keyrings/1password-archive-keyring.gpg]
https://downloads.1password.com/linux/debian/amd64 stable main' >
config/includes.chroot_before_packages/etc/apt/sources.list.d/1password.list
curl -sS
https://downloads.1password.com/linux/debian/debsig/1password.pol >
config/includes.chroot_before_packages/etc/debsig/policies/AC2D62742012EA22/1password.pol
curl -sS https://downloads.1password.com/linux/keys/1password.asc | gpg
--dearmor --output 
config/includes.chroot_before_packages/usr/share/keyrings/1password-archive-keyring.gpg

echo "1password" > config/package-lists/1password.list.chroot
---

While preparing this mail, I had to make a local hack to ensure that
live-build would continue (adding 'apt-get update' in
chroot_install-packages, but this seems to be a proper way to handle the
foreign .deb file

Since the 1password.deb file properly registers within the package
management system, this ticket becomes a 'works for me'.

With kind regards,
Roland Clobus

Hi Roland,

First off, I'd like to let you know that your first email appeared (in my inbox). despite it not appearing on the website. So be rest assured :)

I should mention that this issue is not 1Password-specific. We initially discovered the presence of this behavior in https://gitlab.com/kalilinux/build-scripts/live-build-config/-/issues/61 - where another user reported that their custom packages were being removed.

I think when I tried installing 1Password with the commands listed in 1Password's article, I had a similar result. I may have to check again. However, as per my previous statement, this issue affects more packages other than 1Password.

In my testing, I have noticed that this issue affects the following packages/programs I installed in my custom ISO:
- Docker (installed from Docker's own repositories)
- Tenable Nessus
- Insomnia (https://insomnia.rest)
- Spotify
- ProtonVPN
- Obsidian (https://obsidian.md)
- Visual Studio Code
- Discord

These are just some of the packages I can remember off of my head. A small excerpt from the `syslog` found on `/var/log/installer/` directory lists these in more detail:

One of the packages I installed but not affected by this is Tailscale, which is installed by the following script based on Tailscale's own install script:

Regardless, this issue affects various popular programs. Hence, I am raising this issue to see if there could be a better way of addressing the effect desired by this change. Because IMO the last thing anyone using live-build to "cook" a custom ISO with their desired changes wants to see is the programs they desire to be removed "unknowingly" once they install their ISO and having to manually fix this issue, defeating the whole purpose of them using `live-build`.

---

I don't know if there are "major" differences between the live-build version of Debian and Kali, but according to https://pkg.kali.org/pkg/live-build / https://gitlab.com/kalilinux/packages/live-build/-/blob/kali/master/debian/changelog?ref_type=heads there are only minor adjustments to the version in Debian and Kali aimed at addressing some firmware or GRUB related issues/differences.

I don't really understand what you meant with this statement. If you could elaborate a bit further, I'd sincerely appreciate it.

Kind regards,
Arszilla

Hi again,

I forgot to mention that I have been installing 1Password using the commands that 1Password provided on their support page instead of the Debian Package. However, I realized I had missed the debsig-verify step. As a result, I modified my hook to look like the following:

I have also realized I have provided a wrongful statement about Tailscale upon examining that `filesystem.packages-remove`: It is indeed written for removal. Thus, I kindly ask you to disregard my previous statement regarding Tailscale.

Kind regards,
Arszilla

Hello Arszilla,

As you've already seen, I've prepared a MR to fix at least the issue
with Nessus,
https://salsa.debian.org/live-team/live-build/-/merge_requests/407.

After the MR is merged, at least the Nessus package can now be cleanly
installed, without using a 'chroot_hooks' step. The 'chroot_hooks' step
happens too late, so any package installed at that time will be marked
for deletion after installation.

Recently I've been looking at the way non-Debian-native packages can be
included in the live image and I remembered something with Nessus.
(https://salsa.debian.org/live-team/live-manual/-/merge_requests/39/diffs?commit_id=25088f086f818045f821038e099576e284e1c9d6)

Some of the other packages that are mentioned here are a stub that add
additional sources.list entries.
I've been thinking to add a pseudo-entry for the package list in
'config/package-lists/my_file.list.chroot': '@UPDATE_APT_NOW@', after
which the remainder of the file will be parsed.

I didn't do an experiment yet, but I was thinking about the following:
* Place a .deb in config/packages.chroot that adds an additional
sources.list entry
* Create a 'config/package-lists/my_file.list.chroot' with the content
'@UPDATE_APT_NOW@ my_desired_package_from_the_new_source'

Then the packages will be properly installed and be available after
installation as well.

Packages the need to be present in the live environment, but not after
installation can be mentioned in
'config/package-lists/my_remove_after_installation.list.chroot_live'
(this feature has been available for a long time)

With kind regards,
Roland Clobus

Hi,

same behaviour here with regular debian live-build.
i'm using a very simple hook to install libdvdcss2 via libdvd-pkg
reconfigure hook.

`
export DEBIAN_FRONTEND=noninteractive
dpkg-reconfigure libdvd-pkg
`

the remove-live-packages script removes libdvdcss2 at the end of the
installation while it keeps libdvd-pkg in place.

to solve this, i've put an empty file in
/config/includes.binary/live/filesystem.packages-remove and libdvdcss2
stays in place after installation.

on other side, i've put a late_command to remove live packages i want to
be removed at the end of the installation.

i'm aware it's not a regular way to do this, but i didn't find a way to
preserve packages installed by hooks.

kind regards,
arnault