New commit breaks conda activation? about armor

[markrobinsonuzh]

A couple things to look at:

• whenever you get an error with an R script, look for the Rout file and post it's contents (https://github.com/csoneson/ARMOR/wiki/Troubleshooting) .. in this case, please post the contents of 'example_data/output/Rout/install_pkgs.Rout'
• have you tried in a fresh folder? We'll see what the Rout file shows, but I wonder if the conda environment/R setup is somehow not happy that various scripts and things have changed.

[csoneson]

How long does the conda environment activation "hang" for? Sometimes I have also had the problem that it takes a long-ish time to activate the conda environment for every rule. I don't really know what the reason is, and the next time I run, it is usually back to normal. You could perhaps try to see whether it's due to conda auto-updating (see here: https://stackoverflow.com/questions/44582915/is-possible-to-improve-the-activation-time-of-a-conda-environment)

[devang-mehta]

here's the Rout file:
ERROR: dependencies ‘httr’, ‘usethis’ are not available for package ‘devtools’

• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/devtools’
  ERROR: dependencies ‘devtools’, ‘gdtools’, ‘usethis’ are not available for package ‘vdiffr’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/vdiffr’
  ERROR: dependency ‘biomaRt’ is not available for package ‘GenomicFeatures’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/GenomicFeatures’
  ERROR: dependencies ‘GenomicFeatures’, ‘curl’ are not available for package ‘ensembldb’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/ensembldb’
  ERROR: dependencies ‘GenomicFeatures’, ‘ensembldb’, ‘BiocFileCache’ are not available for package ‘tximeta’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/tximeta’

The downloaded source packages are in
‘/tmp/RtmplKbsSq/downloaded_packages’
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
tximeta : failed to installError in FUN(X[[i]], ...) : there is no package called ‘tximeta’
Calls: usePackage ... suppressPackageStartupMessages -> withCallingHandlers -> lapply -> FUN
In addition: There were 22 warnings (use warnings() to see them)
Execution halted

[csoneson]

Could you post the top of the Rout file, that states the R version? And make sure that version agrees with the one specified in envs/environment_R.yaml.

[devang-mehta]

here's the start of the file. It;s the same version as that in env/environment_R.yaml

R version 3.5.1 (2018-07-02) -- "Feather Spray"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-conda_cos6-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

[devang-mehta]

I just ran it again and it moved past Activating conda environment but stopped with this error
this is from install_pkgs.Rout

• installing source package ‘curl’ ...
  ** package ‘curl’ successfully unpacked and MD5 sums checked
  Found pkg-config cflags and libs!
  Using PKG_CFLAGS=-I/usr/include/x86_64-linux-gnu
  Using PKG_LIBS=-lcurl
  ------------------------- ANTICONF ERROR ---------------------------
  Configuration failed because libcurl was not found. Try installing:
• deb: libcurl4-openssl-dev (Debian, Ubuntu, etc)
• rpm: libcurl-devel (Fedora, CentOS, RHEL)
• csw: libcurl_dev (Solaris)
  If libcurl is already installed, check that 'pkg-config' is in your
  PATH and PKG_CONFIG_PATH contains a libcurl.pc file. If pkg-config
  is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
  R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'

---

ERROR: configuration failed for package ‘curl’

• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/curl’
  ERROR: dependency ‘curl’ is not available for package ‘httr’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/httr’
  ERROR: dependency ‘httr’ is not available for package ‘gh’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/gh’
  ERROR: dependency ‘httr’ is not available for package ‘biomaRt’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/biomaRt’
  ERROR: dependencies ‘curl’, ‘httr’ are not available for package ‘BiocFileCache’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/BiocFileCache’
  ERROR: dependencies ‘curl’, ‘gh’ are not available for package ‘usethis’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/usethis’
  ERROR: dependency ‘biomaRt’ is not available for package ‘GenomicFeatures’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/GenomicFeatures’
  ERROR: dependencies ‘GenomicFeatures’, ‘curl’ are not available for package ‘ensembldb’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/ensembldb’
  ERROR: dependencies ‘httr’, ‘usethis’ are not available for package ‘devtools’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/devtools’
  ERROR: dependencies ‘GenomicFeatures’, ‘ensembldb’, ‘BiocFileCache’ are not available for package ‘tximeta’
• removing ‘/home/ruhrig/ARMOR/.snakemake/conda/17597c04/lib/R/library/tximeta’

The downloaded source packages are in
‘/tmp/Rtmpt4KFZo/downloaded_packages’
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
tximeta : failed to installError in FUN(X[[i]], ...) : there is no package called ‘tximeta’
Calls: usePackage ... suppressPackageStartupMessages -> withCallingHandlers -> lapply -> FUN
In addition: Warning messages:
1: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘curl’ had non-zero exit status
2: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘httr’ had non-zero exit status
3: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘gh’ had non-zero exit status
4: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘biomaRt’ had non-zero exit status
5: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘BiocFileCache’ had non-zero exit status
6: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘usethis’ had non-zero exit status
7: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘GenomicFeatures’ had non-zero exit status
8: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘ensembldb’ had non-zero exit status
9: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘devtools’ had non-zero exit status
10: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :
installation of package ‘tximeta’ had non-zero exit status
Execution halted

[csoneson]

Sorry about the issues you are having. I'm not sure what's going wrong here (seems it can't find libcurl); we haven't seen this issue in our tests, and if I understand correctly the pipeline worked for you previously. To try to narrow down the reasons for the problem - did you try cloning the repository into a new folder, or are you pulling the new changes into the same folder where you were working before?

[devang-mehta]

Hmm I deleted the old repository and git cloned again. this was all in my home folder.

[csoneson]

Ok, I'm a bit at a loss here, given that it worked for you from the beginning (that was on the same hardware, with the same settings, I assume). Perhaps one option could be to manually activate the appropriate conda environment, start R and see whether you can install the packages manually. If not, at least it may make it easier to debug. Another option could be to run with another way of managing R (https://github.com/csoneson/ARMOR/wiki/Managing-software). The environment definitions haven't changed the last few weeks, and the only recent change to the package installation script is to allow multiple cores, which shouldn't lead to the problems that you are seeing.

[csoneson]

@devang-mehta Just curious whether you solved the problem?

[devang-mehta]

hi @csoneson, in the end I hardwiped my computer and reinstalled everything (conda, snakemake, armor) and it seems to be working now.
I've only run the example_data at the moment. I also created and activated a separate environment just for ARMOR this time and hopefully that helps (perhaps add that in the instructions?). I still don't know what happened with the original install though.

[csoneson]

Good to hear that it worked in the end. We have some instructions in the wiki on how to create and activate a conda environment manually (point 2. in https://github.com/csoneson/ARMOR/wiki/Managing-software). If there is anything specific that we can add to this description that you think would be helpful, let us know! Thanks.