Comments (15)
I agree, easy folder sharing is crucial to making rocker containers a practical day-to-day analysis tool.
I tested and added the instructions for windows folder sharing on the RStudio wiki page yesterday, but I can move them over to your new page and leave a cross-reference, what do you think?
I'll do some more testing of permissions, I didn't notice anything unexpected, but perhaps haven't yet exhausted the options. I haven't yet tried to edit and save a doc on my desktop and simultaneously in RStudio in docker.
I agree with your point about enabling diverse workflows as much as possible. For myself and my students we do almost everything in RStudio, to keep the toolkit simple (but when the students aren't looking I use emacs ;). We do some markdown writing in a web editor (http://dillinger.io/ or https://stackedit.io/) because they give word counts a nice live rendering, and we produce the final doc in RStudio with knitr. Word counts is on the feature request list for RStudio, hopefully coming soon. Editing in the browser is handy with docker/rstudio
because then we just move between browser tabs.
from rocker.
Thanks, yes, the windows linking instructions look great. Curious that you
mount to /data in the example; seems like RStudio wouldn't have write
permissions to a directory on the container mounted below ~, or can you
write as well as read files in /data from RStudio like that?
Yeah, moving them into the new page and just linking is probably good, will
help keep the RStudio wiki page brief.
On Mon, Nov 3, 2014 at 4:28 PM, Ben Marwick [email protected]
wrote:
I agree, easy folder sharing is crucial to making rocker containers a
practical day-to-day analysis tool.I tested and added the instructions for windows folder sharing on the RStudio
wiki page
https://github.com/rocker-org/rocker/wiki/Using-the-RStudio-image#folder-sharing
yesterday, but I can move them over to your new page and leave a
cross-reference, what do you think?I'll do some more testing of permissions, I didn't notice anything
unexpected, but perhaps haven't yet exhausted the options. I haven't yet
tried to edit and save a doc on my desktop and simultaneously in RStudio in
docker.I agree with your point about enabling diverse workflows as much as
possible. For myself and my students we do almost everything in RStudio, to
keep the toolkit simple (but when the students aren't looking I use emacs
;). We do some markdown writing in a web editor (http://dillinger.io/ or
https://stackedit.io/) because they give word counts a nice live
rendering. That's great with docker because then we just move between
browser tabs.—
Reply to this email directly or view it on GitHub
#74 (comment).
Carl Boettiger
UC Santa Cruz
http://carlboettiger.info/
from rocker.
I've done a bit of testing folder sharing on windows. Seems like the shortest workflow is to move or create files on the host side of the shared folder (ie. data files), then change their permissions on the container side using bash, then we can read and write bidirectionally. However, files created in the host side of the shared folder do not appear to inherit permissions from the container and their permissions need to be changed in the container (so move everything you need in at the start, then chmod). Files created on the container side will be editable on the host side also (live updating) with no further changes (so it's fine to create scripts and output in RStudio as you go).
Notes on tests, starting from docker run -dp 8787:8787 rocker/rstudio
:
- can't create new file or make new folder in /data in RStudio
- can create new file in explorer at \192.168.59.103\data
- file created on host in /data is read-only in RStudio
- can access shell via RStudio GUI, need to sudo to chmod, but cannot, password 'rstudio' doesn't work
- can access shell via
docker run -it --volumes-from my-data rocker/rstudio /bin/bash
ordocker exec -it "id of running container" bash
and thenchmod -R a+rwX data
(without sudo) to enable read and write permissions ondata
folder - after the
chmod
then runningdocker run -dp 8787:8787 rocker/rstudio
I can create new files and folders in RStudio, and edit files on the host also - after chmod, new files created on the host side of the share are not immediately writable in RStudio, and chmod needs to be done again in the container.
So, some outstanding questions:
- sudo using the shell via RStudio, what's the password?
- streamline changing permissions on the data folder? Can some steps be combined?
docker run -itdp 8787:8787 --volumes-from my-data rocker/rstudio /bin/bash
doesn't seem to start RStudio or give a shell prompt, but perhaps there is a similar one-liner that will? - create a file on the host that inherits permissions from the container?
This is my current shortest workflow:
# open a regular terminal and assuming b2d is installed already:
boot2docker ssh
# folder sharing on
docker run -v /data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-data
# change folder permissions
docker run -it --volumes-from my-data rocker/rstudio /bin/bash # or docker exec -it "id of running container" bash
chmod -R a+rwX /data # to change permissions for the shared data folder, access this on the desktop via \\192.168.59.103\data in explorer
exit # to exit bash and get back to docker prompt
# start my container & do stuff in RStudio with shared folder
docker run -itdp 8787:8787 --volumes-from my-data rocker/rstudio # connect this container to shared folder
# go to http://192.168.59.103:8787/ in the browser, log in with rstudio/rstudio
# finish
exit # in boot2docker window, to quit boot2docker
from rocker.
Right, this looks like what I would expect; on the container /data is owned by root by default because it's below the level of the user's home directory (e.g. /home/rstudio). On the linux side I've simply resorted to mounting data in the user's home directory instead, which seems more natural than editing permissions. Can you just replace /data with /home/rstudio/data, e.g.
docker run -v /home/rstudio/data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-data
?
from rocker.
Thanks for the suggestion, it would be good if it could be that simple :)
I get unable to connect to service
when I try http://192.168.59.103:8787/ (so no RStudio server at all) and Windows cannot access \\192.168.59.103\home\rstudio\data
or any variant of that path when I try to connect on the host side. Seems that this might not be how windows is expecting the share to operate, I don't have any specific ideas to move ahead, but will keep exploring (suggestions welcome!).
from rocker.
Hmm, well, you're using the volume syntax to specify only the location on the container (e.g. no colon). Perhaps we could make Windows browser part happy by doing:
docker run -v /data:/home/rstudio/data --name my-data busybox true
maybe?
If that works and you can access: 192.168.59.103\data
then maybe we can debug why you can't connect to RStudio server next... (that one might be due to an old version of rocker/rstudio, lemme know what hash your image has)
from rocker.
(Or perhaps we might need to also run the busybox container as a non-root user to avoid the rstudio unable to connect to service
issue)
from rocker.
Okay, haven't tested on Windows yet but I think this will work:
docker run -v /data:/home/docker/data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-data
docker run -itdp 8787:8787 -e USER=docker --volumes-from my-data rocker/rstudio # connect this container to shared folder
And login as user/pw set docker/rstudio.
I think I see why this isn't working when user is rstudio
instead of docker
, testing a fix now.
from rocker.
Okay, just pushed a fix so it will work with the rstudio user or any custom user. Also noticed a typo in the command I gave earlier (forgot to use /data:/home/rstudio/data
as I had suggested). So with the new version on sandbox (not yet merged) I'm thinking this should work:
docker run -v /data:/home/rstudio/data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-data
docker run -itdp 8787:8787 --volumes-from my-data rocker/rstudio
Test with changing the /home/rstudio
to /home/docker
and using -e USER=docker
on the last call until we merge sandbox...
from rocker.
p.s. is the samba method still the most robust option for windows? From the boot2docker docs, it looks like VirtualBox Guest Additions is standard with boot2docker now, https://github.com/boot2docker/boot2docker#folder-sharing, so that you should just be able to link directly with something like:
docker run -d -p 8787:8787 -v /c/Users/foobar:/home/rstudio/foobar rocker/rstudio
Sounds more streamlined than samba, but no idea how well it works.
from rocker.
Thanks very much for all those suggestions, I've given them all a thorough workout and it looks like the winner is your last one using Guest Additions. I also tried the method here, but no luck, and it's more complex, so I'm happy to ignore that in favour of the GA method. I'll do some editing of the wiki to indicate the GA method as the way to go, though would you mind to look at the message I'm getting in the console (see bottom)?
Notes on testing:
docker run -v /home/docker/data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-data
docker run -itdp 8787:8787 -e USER=docker --volumes-from my-data rocker/rstudio # connect this container to shared folder
And login as user/pw set docker/rstudio.
- can connect to RStudio and see /data folder, but cannot access it (save to)
######################
docker run -v /data:/home/docker/data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-data
docker run -itdp 8787:8787 --volumes-from my-data -e USER=docker rocker/rstudio
And login as user/pw set docker/rstudio.
- can connect to RStudio and see /data folder, but cannot access it (save to)
- I get 'no such file or directory'
#######################
docker run -d -p 8787:8787 -v /c/Users/marwick:/home/rstudio/marwick rocker/rstudio
- log in with rstudio/rstudio
- can connect to RStudio and see contents of my host folders, and can save a new file to them from RStudio
- I can edit and save files already on the host in RStudio
- I get this message about 50 times in my RStudio console each time open the shared folder or navigate to or from the top level, which is rather distracting
05 Nov 2014 18:36:57 [rsession-rstudio] ERROR system error 71 (Protocol error) [path=/home/rstudio/marwick/Templates]; OCCURRED AT: bool core::FilePath::exists() const /home/ubuntu/rstudio/src/cpp/core/FilePath.cpp:307; LOGGED FROM: bool core::FilePath::exists() const /home/ubuntu/rstudio/src/cpp/core/FilePath.cpp:307
from rocker.
In case 2 (with -v data:/home/docker/data), on the container (that is, from
within RStudio) you say that you can see but cannot access the /data
folder. Are you referring to /home/docker/data
that you can see but
cannot write to, or to /data
? (The rstudio file browser should open up
in /home/docker and show the folder /data).
If you can't write to /home/docker/data
, can you check that you have the
very latest rocker/rstudio
(e.g. docker pull rocker rstudio
)? Can you
also run in RStudio:
system("ls -l /home/docker/")
and maybe:
system("ls -l /home")
and paste the results here? (e.g. does user 'docker' own these
directories?)
otherwise, if /home/docker/data doesn't exist, that's a different issue.
Glad the Guest Additions pretty much works out of the box with one command.
Very weird regarding that error message. what's with the 'ubuntu' user
appearing in that path anyway? (none of the containers you use are based
on ubuntu...) Can you try logging in setting the custom user name to
ubuntu instead of defaulting to rstudio and see what happens?
On Wed, Nov 5, 2014 at 11:50 PM, Ben Marwick [email protected]
wrote:
Thanks very much for all those suggestions, I've given them all a thorough
workout and it looks like the winner is your last one using Guest
Additions. I also tried the method here
http://blog.tutum.co/2014/11/05/how-to-use-docker-on-windows/, but no
luck, and it's more complex, so I'm happy to ignore that in favour of the
GA method.Notes on testing:
docker run -v /home/docker/data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-datadocker run -itdp 8787:8787 -e USER=docker --volumes-from my-data rocker/rstudio # connect this container to shared folder
And login as user/pw set docker/rstudio.
- can connect to RStudio and see /data folder, but cannot access it
(save to)######################
docker run -v /data:/home/docker/data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-data
docker run -itdp 8787:8787 --volumes-from my-data -e USER=docker rocker/rstudio
And login as user/pw set docker/rstudio.
- can connect to RStudio and see /data folder, but cannot access it
(save to)- I get 'no such file or directory'
#######################
docker run -d -p 8787:8787 -v /c/Users/marwick:/home/rstudio/marwick rocker/rstudio
- log in with rstudio/rstudio
- can connect to RStudio and see contents of my host folders, and can
save a new file to them from RStudio- I can edit and save files already on the host in RStudio
- I get this message about 50 times in my RStudio console each time
open the shared folder or navigate to or from the top level, which is
rather distracting05 Nov 2014 18:36:57 [rsession-rstudio] ERROR system error 71 (Protocol error) [path=/home/rstudio/marwick/Templates]; OCCURRED AT: bool core::FilePath::exists() const /home/ubuntu/rstudio/src/cpp/core/FilePath.cpp:307; LOGGED FROM: bool core::FilePath::exists() const /home/ubuntu/rstudio/src/cpp/core/FilePath.cpp:307
—
Reply to this email directly or view it on GitHub
#74 (comment).
Carl Boettiger
UC Santa Cruz
http://carlboettiger.info/
from rocker.
bump. @benmarwick did we ever resolve this?
from rocker.
Thanks for the reminder, now that we've got folder sharing basically working I've been enjoying using rocker as my regular R environment, and neglected the tests, sorry about that. Here's a bit of follow-up:
For this setup:
docker run -v /data:/home/docker/data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-data
docker run -itdp 8787:8787 --volumes-from my-data -e USER=docker rocker/rstudio
I get:
> system("ls -l /home/docker/")
total 0
drwxr-xr-x 2 root root 40 Nov 23 22:14 data
> system("ls -l /home")
total 4
drwxr-xr-x 4 docker docker 4096 Nov 23 22:29 docker
And if remove -e USER=docker
:
docker run -v /data:/home/docker/data --name my-data busybox true
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba my-data
docker run -itdp 8787:8787 --volumes-from my-data rocker/rstudio
I get
> system("ls -l /home/docker/")
total 0
drwxr-xr-x 2 root root 40 Nov 23 22:14 data
> system("ls -l /home")
total 8
drwxr-xr-x 3 rstudio rstudio 4096 Nov 23 22:26 docker
drwxr-xr-x 3 rstudio rstudio 4096 Nov 23 22:27 rstudio
For comparison, by not invoking sharing at all with docker run -dp 8787:8787 -e ROOT=TRUE rocker/rstudio
I get:
> system("ls -l /home/docker/")
total 0
> system("ls -l /home")
total 8
drwxr-xr-x 2 rstudio rstudio 4096 Nov 21 03:15 docker
drwxr-xr-x 3 rstudio rstudio 4096 Nov 23 22:02 rstudio
And using my current favourite: docker run -dp 8787:8787 -e ROOT=TRUE -v /c/Users/marwick:/home/rstudio/ rocker/hadleyverse
I get:
> system("ls -l /home/docker/")
total 0
> system("ls -l /home")
total 24
drwxr-xr-x 2 rstudio rstudio 4096 Nov 21 03:15 docker
drwxrwxrwx 1 rstudio staff 20480 Nov 27 2014 rstudio
as well as access to the main parts of my windows filesystem and this message periodically scrolling down the console, seems like it's related to file access operations:
23 Nov 2014 22:09:04 [rsession-rstudio] ERROR system error 71 (Protocol error) [path=/home/rstudio/Templates]; OCCURRED AT: bool core::FilePath::exists() const /home/ubuntu/rstudio/src/cpp/core/FilePath.cpp:307; LOGGED FROM: bool core::FilePath::exists() const /home/ubuntu/rstudio/src/cpp/core/FilePath.cpp:307
A similar error has been reported to RStudio support here, I've posted mine, so let's see if they have any ideas.
from rocker.
Okay, thanks a lot for the report. So it looks like the samba sharing makes the directory it shares (e.g. /home/$USER/data
) owned by root no matter what, and we loose write permissions. Of course now that you can enable root inside RStudio, one could chown -R $USER:$USER /home/$USER
to fix things. But it looks like in general we should instruct people to link directly with the -v /c/Users/...
and not samba, and things work as expected (and with fewer commands. win). Looks like you've also listed this as the prefered method on the wiki, so I think we're all set -- nice work!
Think we can close this out then. Thanks again for all the testing.
from rocker.
Related Issues (20)
- Migrating from Dockerhub? HOT 3
- Container for RStudio (not server) HOT 1
- MRAN repository snapshots were deleted - Failing jobs because packages can't be found anymore HOT 5
- docker v18 & packages repositories & rocker's images HOT 5
- unable to load shared object
- Update results in deletion of all install package...
- installation testing fails HOT 1
- r: error while loading shared libraries: libR.so: cannot open shared object file: No such file or directory
- Docker image r-base:4.2.1 not reproducible HOT 4
- Installing unstable apt packages? HOT 4
- Local Zotero Integration? HOT 3
- R session terminated HOT 1
- latest r-base breaks install of `curl` package HOT 8
- xz-utils backdoor HOT 4
- Potential high vulnerability in r-base:4.3.1 HOT 3
- Installing packages fails due to broken openssl package in debian testing HOT 2
- Alternative Methods of Extending Images? HOT 5
- update 'latest' version to 4.4.0 RDS security flaw HOT 2
- New `usrmerge` issues. HOT 5
- 502 Returned by Rstudio Server when executing 'client_init' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rocker.