Subject: ITP: conda -- OS-agnostic, system-level binary package manager and ecosystem Package: wnpp Owner: Andreas Tille <tille@debian.org> Severity: wishlist * Package name : conda Version : 4.6.10 Upstream Author : xx-20yy <upstream> * URL : https://conda.io/ * License : <license> Programming Lang: Python Description : OS-agnostic, system-level binary package manager and ecosystem Conda is a cross-platform, language-agnostic binary package manager. It is the package manager used by Anaconda installations, but it may be used for other systems as well. Conda makes environments first-class citizens, making it easy to create independent environments even for C libraries. Remark: This package is maintained by Debian Med Packaging Team at https://salsa.debian.org/med-team/conda
Complete missing information: * Package name : conda Version : 4.7.11 Upstream Author : 2012-2019 Anaconda, Inc. * URL : https://conda.io/ * License : BSD-3-clause Programming Lang: Python Description : OS-agnostic, system-level binary package manager and ecosystem Conda is a cross-platform, language-agnostic binary package manager. It is the package manager used by Anaconda installations, but it may be used for other systems as well. Conda makes environments first-class citizens, making it easy to create independent environments even for C libraries.
Hi Andreas, Conda should not enter the main section. As doko said, conda use MKL (non-free) by default, and prebuilt packages are linked against libmkl_rt.so (entrance lib of MKL). So if you are going to do performance tests, swithing BLAS/LAPACK alterntive to libmkl_rt for Debian packages could help you reduce the number of variables to control. Install an Anaconda instance and identify the MKL linkage: fdfind -e so -x readelf -d | rg mkl Another point supporting the "should enter contrib" claim is that Anaconda does NOT allow its software repository be mirrored without asking, which makes the packages not freely redistributable.
Note that these are distinct: - conda, the package manager itself; BSD-3 licensed - Anaconda Individual Edition, a bundle of conda + some commonly-used packages; non-DFSG-free as it includes intel-mkl and cudnn (https://docs.anaconda.com/anaconda/eula/) - Anaconda repository, the server conda uses by default (https://repo.anaconda.com/pkgs/); has a no-unauthorized-mirrors policy conda does not have to use its default server, as packages can be created with conda-build (also BSD-3 licensed) and distributed with standard web servers: https://conda.io/projects/conda/en/latest/user-guide/tasks/create-custom-channels.html Did you find an intel-mkl dependency in conda itself, or only in packages?
Hi Rebecca, Conda itself, as a python package manager indeed does not require nonfree blobs such as intel-mkl to function. Only some of the the packages it downloads require, say intel-mkl, cudnn or alike. However, if conda toolchains can be used to setup standalone servers, I'll not stand in the way as long as anyone is willing to work on that.
Test results (edited for length): I: pybuild base:217: cd /build/conda-4.8.2+dfsg/.pybuild/cpython3_3.8/build; python3.8 -m pytest tests ============================= test session starts ============================== platform linux -- Python 3.8.3, pytest-4.6.9, py-1.8.1, pluggy-0.13.0 rootdir: /build/conda-4.8.2+dfsg, inifile: setup.cfg plugins: xonsh-0.9.18 collected 648 items tests/test_activate.py .....s................s.s........ssssFsFssssssF [ 7%] tests/test_api.py ...s.s...s [ 8%] tests/test_exceptions.py ....................... [ 12%] tests/test_exports.py .. [ 12%] tests/test_fetch.py sss. [ 13%] tests/test_history.py .sss....... [ 14%] tests/test_import.py ... [ 15%] tests/test_info.py .ss [ 15%] tests/test_install.py .......s...... [ 18%] tests/test_instructions.py ..... [ 18%] tests/test_link_order.py ss [ 19%] tests/test_lock.py ...... [ 20%] tests/test_logic.py ............x [ 22%] tests/test_misc.py ...... [ 22%] tests/test_plan.py ....x.... [ 24%] tests/test_resolve.py ..........s....................................... [ 32%] . [ 32%] tests/test_toposort.py ..... [ 33%] tests/test_utils.py .. [ 33%] tests/base/test_constants.py .. [ 33%] tests/base/test_context.py ....................... [ 37%] tests/cli/test_common.py ...... [ 38%] tests/cli/test_conda_argparse.py .... [ 38%] tests/cli/test_config.py ......... [ 40%] tests/cli/test_main_info.py .. [ 40%] tests/common/test_configuration.py ....................... [ 43%] tests/common/test_io.py .. [ 44%] tests/common/test_path.py ........ [ 45%] tests/common/test_url.py ..... [ 46%] tests/common/test_yaml.py .... [ 46%] tests/common/os/test_windows.py . [ 47%] tests/common/pkg_formats/test_python.py .......s...s................. [ 51%] tests/conda_env/test_env.py ............................................ [ 58%] ....s. [ 59%] tests/conda_env/test_pip_util.py ....... [ 60%] tests/core/test_envs_manager.py .s. [ 60%] tests/core/test_index.py ..s.ssss [ 62%] tests/core/test_initialize.py ..ss..sssFFFF..s.s... [ 65%] tests/core/test_package_cache_data.py . [ 65%] tests/core/test_path_actions.py ........ [ 66%] tests/core/test_portability.py .. [ 66%] tests/core/test_prefix_data.py ...... [ 67%] tests/core/test_solve.py .............................................. [ 75%] tests/core/test_subdir_data.py FF.......... [ 76%] tests/gateways/test_connection.py ... [ 77%] tests/gateways/test_logging.py .. [ 77%] tests/gateways/disk/test_delete.py ........... [ 79%] tests/gateways/disk/test_link.py .. [ 79%] tests/gateways/disk/test_permissions.py ....... [ 80%] tests/gateways/disk/test_read.py sssssssss [ 82%] tests/models/test_channel.py ..................................... [ 87%] tests/models/test_dist.py ........ [ 89%] tests/models/test_index_record.py .. [ 89%] tests/models/test_match_spec.py ........s............................... [ 95%] ... [ 95%] tests/models/test_package_info.py . [ 96%] tests/models/test_prefix_graph.py ......... [ 97%] tests/models/test_version.py ................ [100%] =================================== FAILURES ===================================----------------------------- Captured stderr call ----------------------------- modified /tmp/d463c395/condabin/conda modified /tmp/d463c395/bin/conda modified /tmp/d463c395/bin/conda-env modified /tmp/d463c395/bin/activate modified /tmp/d463c395/bin/deactivate modified /tmp/d463c395/etc/profile.d/conda.sh modified /tmp/d463c395/etc/fish/conf.d/conda.fish modified /tmp/d463c395/shell/condabin/Conda.psm1 modified /tmp/d463c395/shell/condabin/conda-hook.ps1 modified /tmp/d463c395/lib/python3/dist-packages/xontrib/conda.xsh modified /tmp/d463c395/etc/profile.d/conda.csh no change /tmp/d463c395/lib/python3/dist-packages modified /tmp/d463c395/lib/python3/dist-packages/conda.egg-link modified /tmp/d463c395/lib/python3/dist-packages/easy-install.pth modified /build/conda-4.8.2+dfsg/conda.egg-info ==> For changes to take effect, close and re-open your current shell. <==
Hi, I needed conda for something, found the current packaging progress and fiddled a bit to get it to build. The things I found that needed to be fixed: conda has an undeclared dependency on distutils that I needed to add to the package dependency the conda/shell/bin/conda entry point was using the shebang "#!/usr/bin/env python" which on Debian assumes python 2. I added a patch to make it "#!/usr/bin/env python3" though I'm not sure how that's going to work in conda environments? conda/shell/bin/conda needs to be executable, which the standard Debian dh_install step removes the x bit from anything going into /usr/lib/python3/dist-package For the run-test command I found the "source /usr/lib/python3/dist-packages/conda/shell/bin/activate" line also needed CONDA_EXE to be set. I also found (when testing in an autopkgtest shell that conda create was a problem when running as root because it really wanted to install files into the already existing directory /usr/pkgs and conda reported a warning when being run as root. To work correctly I believe conda create needs a non-root user with a home directory to create it's configuration files in. Since the default autopkgtest runner doesn't have network access, I changed the run-test to running "conda config" (which doesn't report anything since there's no root configuration) and "conda help" I'm attaching the patches I made for discussion but I can push them to the debian-med repository if everyone's ok with that. Diane
Hi, I needed conda for something, found the current packaging progress and fiddled a bit to get it to build. The things I found that needed to be fixed: conda has an undeclared dependency on distutils that I needed to add to the package dependency the conda/shell/bin/conda entry point was using the shebang "#!/usr/bin/env python" which on Debian assumes python 2. I added a patch to make it "#!/usr/bin/env python3" though I'm not sure how that's going to work in conda environments? conda/shell/bin/conda needs to be executable, which the standard Debian dh_install step removes the x bit from anything going into /usr/lib/python3/dist-package For the run-test command I found the "source /usr/lib/python3/dist-packages/conda/shell/bin/activate" line also needed CONDA_EXE to be set. I also found (when testing in an autopkgtest shell that conda create was a problem when running as root because it really wanted to install files into the already existing directory /usr/pkgs and conda reported a warning when being run as root. To work correctly I believe conda create needs a non-root user with a home directory to create it's configuration files in. Since the default autopkgtest runner doesn't have network access, I changed the run-test to running "conda config" (which doesn't report anything since there's no root configuration) and "conda help" I'm attaching the patches I made for discussion but I can push them to the debian-med repository if everyone's ok with that. Diane
I decided to go ahead and test the package in a VM and see what happens. Currently /usr/bin/conda will error out for commands other than init or help, running init prompts for a sudo password, so I decided to look a bit further into what it was trying to do. It attempts to create the following files, if you add "-v" you can see the diffs between whats there and what conda wants to be present $ conda init --dry-run modified /usr/condabin/conda modified /usr/bin/conda modified /usr/bin/conda-env modified /usr/bin/activate modified /usr/bin/deactivate modified /usr/etc/profile.d/conda.sh modified /usr/etc/fish/conf.d/conda.fish modified /usr/shell/condabin/Conda.psm1 modified /usr/shell/condabin/conda-hook.ps1 modified /usr/lib/python3/dist-packages/xontrib/conda.xsh modified /usr/etc/profile.d/conda.csh modified /usr/condabin/conda modified /usr/bin/conda modified /usr/bin/conda-env modified /usr/bin/activate modified /usr/bin/deactivate modified /usr/etc/profile.d/conda.sh modified /usr/etc/fish/conf.d/conda.fish modified /usr/shell/condabin/Conda.psm1 modified /usr/shell/condabin/conda-hook.ps1 modified /usr/lib/python3/dist-packages/xontrib/conda.xsh modified /usr/etc/profile.d/conda.csh modified /home/diane/.bashrc It looks like by default it'll try to overwrite the /usr/bin/conda generated by setup.py with it's own script. I'm guessing we should add some of the scripts it's trying to generate to debian/ and install them into the package from there instead of running their conda init. Though I suspect some paths will still need to be adjusted as /usr/etc/profile.d does not look like a good target. I also really wonder how much of conda should be polluting the global namespace since it's really intended to be used as a user level package manager. Diane
I decided to go ahead and test the package in a VM and see what happens. Currently /usr/bin/conda will error out for commands other than init or help, running init prompts for a sudo password, so I decided to look a bit further into what it was trying to do. It attempts to create the following files, if you add "-v" you can see the diffs between whats there and what conda wants to be present $ conda init --dry-run modified /usr/condabin/conda modified /usr/bin/conda modified /usr/bin/conda-env modified /usr/bin/activate modified /usr/bin/deactivate modified /usr/etc/profile.d/conda.sh modified /usr/etc/fish/conf.d/conda.fish modified /usr/shell/condabin/Conda.psm1 modified /usr/shell/condabin/conda-hook.ps1 modified /usr/lib/python3/dist-packages/xontrib/conda.xsh modified /usr/etc/profile.d/conda.csh modified /usr/condabin/conda modified /usr/bin/conda modified /usr/bin/conda-env modified /usr/bin/activate modified /usr/bin/deactivate modified /usr/etc/profile.d/conda.sh modified /usr/etc/fish/conf.d/conda.fish modified /usr/shell/condabin/Conda.psm1 modified /usr/shell/condabin/conda-hook.ps1 modified /usr/lib/python3/dist-packages/xontrib/conda.xsh modified /usr/etc/profile.d/conda.csh modified /home/diane/.bashrc It looks like by default it'll try to overwrite the /usr/bin/conda generated by setup.py with it's own script. I'm guessing we should add some of the scripts it's trying to generate to debian/ and install them into the package from there instead of running their conda init. Though I suspect some paths will still need to be adjusted as /usr/etc/profile.d does not look like a good target. I also really wonder how much of conda should be polluting the global namespace since it's really intended to be used as a user level package manager. Diane
I fiddled with my VM and looked at a miniconda installations and figured out how conda "normally" works. The default behavior is to act kind of like a venv. The conda scripts and supporting files get copied into an install directory with a python interpreter and lib/python3 files. That ends up setting sys.prefix in to the install directory which is used to determine the root_prefix and thus where things will get installed. I was able to fake it by symlinking the debian python interpreters into $install_dir/bin/python3.9 setting that to the shebang path and adding this pyvenv.cfg to $install_dir home = /usr/bin include-system-site-packages = true version = 3.9.7 then running $install_dir/bin/conda init bash and it updated the scripts and added conda's shell initialization hook to the end of my .bashrc (And after all that and relogging conda create --name snowflake biopython did work) There might be another way to override the root_prefix to get this all working. Also the /usr/bin/conda that gets installed by setup.py tries to run conda.cli.main_pip, and it seems like maybe that should be changed to create the $install_dir, and I think that might be unimplemented by upstream. And that's enough from me for today. Diane