Snakemake/Conda install issues (temporary solution?) about covid-19-signal

[fmaguire]

This is related to the issue I made yesterday. Basically, conda is a great installer and a terrible historical dependency system because they remove old packages. The envs should be stripped down to the core tools we actually use (ideally with version specified for reproducing) and their respective dependencies should be managed automatically by conda sees fit. This should make things a bit more stable and easy to maintain over time as there are fewer 'hard coded' dependencies.

…

On Tue., May 26, 2020, 10:31 amos, ***@***.***> wrote: Same package with wrong build number for a different yaml file: Building DAG of jobs... Creating conda environment conda_envs/trim_qc.yaml... Downloading and installing remote packages. Environment for conda_envs/trim_qc.yaml created (location: .snakemake/conda/b49a1142) Creating conda environment conda_envs/ivar.yaml... Downloading and installing remote packages. Environment for conda_envs/ivar.yaml created (location: .snakemake/conda/1058a4d9) Creating conda environment conda_envs/assembly_qc.yaml... Downloading and installing remote packages. CreateCondaEnvironmentException: Could not create conda environment from /workspace/raphenar/sars-cov-2/covid-19-sequencing/conda_envs/assembly_qc.yaml: Collecting package metadata (repodata.json): ...working... done Solving environment: ...working... failed ResolvePackageNotFound: - ld_impl_linux-64==2.34=h53a641e_0 @fmaguire <https://github.com/fmaguire> @kmsmith137 <https://github.com/kmsmith137> should we not specify build numbers? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#60 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAM6WROOPP2YVW4C72EJYDTRTPABTANCNFSM4NJ55Q6Q> .

[raphenya]

This worked for me by specifying version:
conda create -c conda-forge -c bioconda -n snakemake snakemake=5.11.2 pandas

Then I get this error when running the pipeline:

Building DAG of jobs...
Creating conda environment conda_envs/ivar.yaml...
Downloading and installing remote packages.
Environment for conda_envs/ivar.yaml created (location: .snakemake/conda/1058a4d9)
Creating conda environment conda_envs/snp_mapping.yaml...
Downloading and installing remote packages.
CreateCondaEnvironmentException:
Could not create conda environment from /workspace/raphenar/sars-cov-2/covid-19-sequencing/conda_envs/snp_mapping.yaml:
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed
ResolvePackageNotFound: 
  - ld_impl_linux-64==2.34=h53a641e_0
  - binutils_impl_linux-64==2.34=h53a641e_0

[raphenya]

I re-created the environment with python version 3.6 then I got:

Building DAG of jobs...
Creating conda environment conda_envs/trim_qc.yaml...
Downloading and installing remote packages.
CreateCondaEnvironmentException:
Could not create conda environment from /workspace/raphenar/sars-cov-2/covid-19-sequencing/conda_envs/trim_qc.yaml:
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed

ResolvePackageNotFound: 
  - ld_impl_linux-64==2.34=h53a641e_0

Searching the condo channels the build number is wrong in the yaml file:

conda search ld_impl_linux-64 -c bioconda -c defaults -c conda-forge
Loading channels: done
# Name                       Version           Build  Channel             
ld_impl_linux-64                2.32               5  conda-forge         
ld_impl_linux-64                2.32      h53a641e_6  conda-forge         
ld_impl_linux-64              2.33.1               5  conda-forge         
ld_impl_linux-64              2.33.1      h53a641e_6  conda-forge         
ld_impl_linux-64              2.33.1      h53a641e_7  pkgs/main           
ld_impl_linux-64              2.33.1      h53a641e_7  conda-forge         
ld_impl_linux-64              2.33.1      h53a641e_8  conda-forge         
ld_impl_linux-64                2.34      h53a641e_2  conda-forge     

I will update the build number and see if this error goes away.

[raphenya]

Same package with wrong build number for a different yaml file:

Building DAG of jobs...
Creating conda environment conda_envs/trim_qc.yaml...
Downloading and installing remote packages.
Environment for conda_envs/trim_qc.yaml created (location: .snakemake/conda/b49a1142)
Creating conda environment conda_envs/ivar.yaml...
Downloading and installing remote packages.
Environment for conda_envs/ivar.yaml created (location: .snakemake/conda/1058a4d9)
Creating conda environment conda_envs/assembly_qc.yaml...
Downloading and installing remote packages.
CreateCondaEnvironmentException:
Could not create conda environment from /workspace/raphenar/sars-cov-2/covid-19-sequencing/conda_envs/assembly_qc.yaml:
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed

ResolvePackageNotFound: 
  - ld_impl_linux-64==2.34=h53a641e_0

@fmaguire @kmsmith137 should we not specify build numbers?

[jaleezyy]

Updating the build (or letting conda make some decisions in which build to install) restored functionality.

[jaleezyy]

Simplifying the env files would definitely make things easier as per 58.

[fmaguire]

rgi installation has had similar issues as well with versions