traitecoevo / dockertest Goto Github PK

They send a weekly digest with links, e.g.

What does this error message indicate?

> dockertest::build()
Error in f(".") : Hit / without finding .git

URLs are easy, e.g.

https://raw.githubusercontent.com/richfitz/remake/master/DESCRIPTION

We can cache them in the build directory, and use this to pick up more up the dependency tree.

As recommended on the [rocker wiki
(https://github.com/rocker-org/rocker/wiki#installing-binaries).

Instructions are here.

system becomes apt_packages
packages:R becomes r_packages
packages:github becomes r_github_packages

Then I guess we have r_local_packages to replace packages:local

Can you specify dependencies from other standard R repositories as well? (Or more ideally, just extract the repositories URLs from the AdditionalRepositories list of the DESCRIPTION file)

I see that in mortality_bci and associated repos we still end up with:

RUN mkdir /root/mortality_bci \
  && echo "clone.sh /root/mortality_bci" >> /root/.bashrc \
  && echo "system('clone.sh /root/mortality_bci')" > /root/.Rprofile \
  && echo "system('clone.sh /root/mortality_bci')" > /root/.littler.r

in the dockerfile. However, these are not actually used and it's no longer how we want to use these images. It's probably desirable for the package testing though.

Pull data from the yml (add a maintainer: spot) and alternatively fill with information from the R session; I think that some username/email things might be available there.

https://github.com/hadley/rappdirs

Hi,

I am trying to build a package and I am getting the following error message:

"Cloning into 'self'...
done.
Fetching DESCRIPTION for traitecoevo/callr
Fetching DESCRIPTION for traitecoevo/dockertest
Downloading crandb/latest - may take a minute

Error in match.fun(FUN) : object 'parse_deps' not found"
Is there other dependencies that I need to install?
Thank you for your help.

Vanessa

https://github.com/metacran/sysreqs

The inplace: false idea (default for projects) clones the project into a directory self with the idea that the project will have a fresh clone to work with.

• The name / idea is not very clear
• There's no obvious way of recloning and that should probably be done at launch time
• The launch commands generated by dockertest don't point the working directory there
• If they did, then output would be at risk of being obliterated by a reclone

from boot2docker to docker machine to native support, we can probably pull this out now

I have a simple problem: After following instructions to start using this tempting project in order to test my R-package, I created the "docker"-folder within my R-package root dir (as suggested), started R from within this folder and run dockertest::build().
The image building fails, giving back the following message:

dockertest::build()
Klone nach 'self' ...
Fertig.
Fetching DESCRIPTION for traitecoevo/callr
Fetching DESCRIPTION for traitecoevo/dockertest
Fehler: Could not find package root.
Zusätzlich: Warnmeldungen:
1: In file.rename(dest_tmp, dest) :
kann Datei '/tmp/RtmpsinGHy/file295f5c839d51' nicht in '/home/mreich/.cache/dockertest/github/traitecoevo/callr/DESCRIPTION' umbenennen. Grund 'Ungültiger Link über Gerätegrenzen hinweg'
2: In file.rename(dest_tmp, dest) :
kann Datei '/tmp/RtmpsinGHy/file295f1ea9a1e2' nicht in '/home/mreich/.cache/dockertest/github/traitecoevo/dockertest/DESCRIPTION' umbenennen. Grund 'Ungültiger Link über Gerätegrenzen hinweg'

I have no idea what could be the reason. I run R with and without sudo (thinking this might be a rights issue) but both resulting in the same error.
Can anyone point me to a solution? That would be great as I would really much like to use this package!

The top page of docker installation docs for mac state

"Note: This release of Docker deprecates the Boot2Docker command line in favor of Docker Machine. Use the Docker Toolbox to install Docker Machine as well as the other Docker tools."

So we should update dockertest to sue docker-machine.

@richfitz to provide basic documentation. Once done, @dfalster can try it out on traitecoevo/growth_trajectories#40

Should possibly prefer the remake module within dockertest? Or rename modules to be something other than remake.yml; make them dockertest_remake.yml

There is now a package called callr on cran, which is also used in the new rhub package. So installing either will clobber our callr package and make this version of dockertest unusable.

> library(dockertest)
Error : object ‘Sys_which’ is not exported by 'namespace:callr'
Error: package or namespace load failed for ‘dockertest’

We could rename callr, but I am reluctant to do so unless we can envisage it one day ending up on cran. Otherwise we might instead import the relevant functions into dockertest.
Thoughts @richfitz ?

I don't remember :(

@richfitz -- I'm hoping you might be able to test this on another linux machine. If the results below extend to other linux boxes then dockertest may not be working as hoped on linux.

@jscamac and I were trying to use dockerest on linux box to run some analyses for our mortality analysis and encountered some strange behaviours. Initially, we encountered the error that file remake.yml does not exist in the current wd when clearly it does:

@SCI-6297:~/mortality_bci$ remake fulldata_models_precompiled_docker
[  LOAD ] 
[  READ ]                                       |  # loading sources
<  MAKE > fulldata_models_precompiled_docker
[ BUILD ] fulldata_models_precompiled_docker    |  fulldata_models_precompil...
[  READ ]                                       |  # loading packages
Error: while reading 'remake.yml'
The file 'remake.yml' does not exist
jscamac@SCI-6297:~/mortality_bci$ ls
data    downloads  ms          R          remake_sources.yml
docker  figures    precompile  README.md  remake.yml

But what is actually happening in building the fulldata_models_precompiled_docker target is that we are loading a docker container and compiling models inside, with directory mapped back to the current wd. (This works fine on imac but failed on the linux machine.)

Using the dry-run argument of dockertest reveals the code that call that is made to docker. I then ran a simplified version of that to highlight why the above error is not working.

Basically it seems like the directory mapping is not working as it should. Running the following should map our current wd onto the folder /home/data in the docker container. That's what happens on the mac:

✗ /usr/local/bin/docker run -v /Users/dfalster/Dropbox/_research/SIEF/mortality_bci:/home/data -it traitecoevo/mortality_bci
root@246b720e3d9e:/home/data# ls
data  docker  downloads  ignore  ms  ms.pdf  Notes.md  R  README.md  remake_sources.yml  remake.yml  results

But running the equivalent on linux does not produce the desired result (note calling ls returns an empty folder):

jscamac@SCI-6297:~/mortality_bci$ /usr/bin/docker run -v /home/jscamac/mortality_bci:/home/data -it traitecoevo/mortality_bci 
root@df98b8b9fa61:/home/data# ls
root@df98b8b9fa61:/home/data# 

I got a similar result using the base ubuntu container:

jscamac@SCI-6297:~/mortality_bci$ /usr/bin/docker run -v /home/jscamac:/home/data -it ubuntu
root@6ed0501b533c:/home/data# ls
root@6ed0501b533c:/home/data# 

The machine we using is running Ubuntu 14.04.2 (LTS).

this is done in drat.builder already, and would allow things like:

traitecoevo/rrqueue@development

https://github.com/r-hub/sysreqs

After building a docker image, the launch script produced includes an absolute path, e.g. from mortality_bci

#!/bin/bash
set -e
$(boot2docker shellinit 2> /dev/null)
docker run -v /Users/rich/Documents/Projects/veg/mortality_bci:/src -it richfitz/mortality_bci $*

This won't run on collaborators machine because of absolute path. On the other hand docker requires absolute paths. Could we replace with something like generate the path with something like

#!/bin/bash
PWD=($pwd)
set -e
$(boot2docker shellinit 2> /dev/null)
docker run -v ($PWD)/..:/src -it richfitz/mortality_bci $*

If i have two VMs running and try to build with something other than default, the machine argument doesn't seem to be working.

➜ docker-machine ls                              
NAME      ACTIVE   DRIVER       STATE     URL                         SWARM
default            virtualbox   Running   tcp://192.168.99.100:2376   
mem3GB             virtualbox   Stopped                               
mem6GB             virtualbox   Running   tcp://192.168.99.101:2376   

➜  dockertest build --machine mem6GB --no-cache  
More than one machine present, taking the first
Setting up docker-machine 'default'
....

Not on last line is using "default", despite asking it to use "mem6GB"

Could use
getOption("dockertest.username", "dockertest")

It could also be an option within the yml separate from the names: bit? Not clear how that would interact though....

Latest dockertest::launch is really nice in that grabs relevant name, finds running machine and starts image. i wonder about an equivalent for pushing to/pulling from dockerhub. Not a big deal, but would complete functionality in dockertest. Currently to push we need to do something like

 eval "$(docker-machine env mem6GB)" 
docker login
 docker push traitecoevo/successional_diversity

E.g.,

docker_cmd(localhost, "build", ...)

This occurring because repo has moved from rrichfitz/dockertest -> traitecoevo/dockertest

Error message

> dockertest::build()
self
Cloning into 'self'...
done.
Sending build context to Docker daemon 11.39 GB
Sending build context to Docker daemon 
Step 0 : FROM r-base
 ---> 7d3d1c8eee06
Step 1 : RUN apt-get update   && apt-get install -y --no-install-recommends      ca-certificates     curl     git     libcurl4-openssl-dev     ssh   && apt-get clean
 ---> Using cache
 ---> 5c9e78b6017a
Step 2 : RUN install2.r --error devtools
 ---> Using cache
 ---> a9e257978c79
Step 3 : RUN installGithub.r richfitz/dockertest
 ---> Running in 2b50dc8cab49
Downloading github repo richfitz/dockertest@master
Error in download(dest, src, auth) : client error: (404) Not Found
INFO[0352] The command [/bin/sh -c installGithub.r richfitz/dockertest] returned a non-zero code: 1 
Error in docker_build(path, dockerfile, info$tagname, use_cache) : 
  Error running docker

According to this line docker test assumes Rscript is located in #!/usr/bin/Rscript, but this may not always be true. For example, on my laptop it's in #!/usr/local/bin/Rscript. Googling suggests other people have similar issue. So should we

a) Check for location of Rscript when running install_script or
b) Give instructions on creating a symlink?

thoughts @richfitz, @dfalster ?

dockertest::snapshot(filename="dockertest.yml")

Dump out a file suitable for dockertest ingest. We can use devtools' updates to package DESCRIPTIONs to work out what packages are installed via github, assume the others are all on CRAN.