Mounting data_dir volume loses data about docker-geoserver

[srstsavage]

@orlade Not sure which problem you're running into. If you map a volume to the container's /opt/geoserver/data_dir, the intention that the volume should completely replace the GEOSERVER_DATA_DIR inside the container. Using an empty directory for the mapped volume on the host (e.g. $HOME/geoserver_data) works fine. Are you still having issues?

[alexgleith]

I agree with @shane-axiom, in that using -v source:target completely replaces the volume (the geoserver data dir). This means that you should store all your config on the host, which is appropriate, as that way you can back up this config, and start/stop/rebuild your docker instance at will.

[justb4]

I tend to agree with @orlade : I had similar problems: starting without the volume-mapped dir on the host, on initial startup the directory was empty or sometimes even half-empty, e.g. only some GS dirs appeared like /styles and /gwc. IMHO this seems specific to the Tomcat/GeoServer data-dir initialization, since with other Docker -v volume mounts like /usr/local/tomcat/logs to non-existing host dirs volume mapping works fine.

A strange issue, after some struggles, I came up with a solution, maybe not so clean but it works: on first run, when the GS data dir is non-existing on the host, I start the GS container without the data dir volume mapping and copy a the initial data dir from the container to the host using docker cp and then restart the GS container with full mapping. Any changes are then not lost. Effectively this is like having a local copy on the host.
I have automated this in a bash startup script here:
https://github.com/Geonovum/smartemission/blob/master/services/geoserver/run-geoserver.sh

[alexgleith]

I run this in a production environment with a -v mapping with existing data_dir all the time and it works flawlessly.

It would be good if you could provide steps to reproduce the problem, @justb4?

[justb4]

On 01-06-16 15:26, Alex Leith wrote:

I run this in a production environment with a -v mapping with existing
data_dir all the time and it works flawlessly.

It would be good if you could provide steps to reproduce the problem,
@justb4 https://github.com/justb4?
If you map the GS data_dir from the container to an existing data_dir
containing GS config files on your host, there is no problem. The
problem I found arose with the following or similar steps:

• data_dir on host does not exist or is empty e.g. /etc/geoserver
• start the GS container with docker -v
  /etc/geoserver:/opt/geoserver/data_dir
• ls /etc/geoserver shows empty or partially filled directory

Host and container run user root so no create permission issues.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAjj5i7mWfRR1tQAwLbqnBJoopw2BzIEks5qHYicgaJpZM4GWTCL.

[srstsavage]

I can't replicate this problem as reported. The following script is re-runnable and runs a GeoServer 2.8.0 Docker container with an empty directory mapped to the data directory.

#!/bin/bash

#exit on error
set -e

TESTDIR=/tmp/docker_gs_test_data
if [ -d "$TESTDIR" ]; then
  echo "Deleting old GeoServer data directory ($TESTDIR)"
  rm -rf "$TESTDIR"
fi

docker run -it --rm --name gs_data_dir_test -p 8777:8080 \
  -v "$TESTDIR":/opt/geoserver/data_dir kartoza/geoserver:2.8.0

You can use watch ls -l /tmp/docker_gs_test_data to monitor the data directory. On startup GeoServer will create a few files/directories:

drwxr-xr-x 3 root root 4096 Jun  1 11:31 gwc
-rw-r--r-- 1 root root 1577 Jun  1 11:31 gwc-gs.xml
drwxr-xr-x 2 root root 4096 Jun  1 11:31 gwc-layers
drwxr-xr-x 8 root root 4096 Jun  1 11:31 security
drwxr-xr-x 2 root root 4096 Jun  1 11:31 styles

And then will create others (workspaces, etc) as they are created/needed. This is normal GeoServer behavior. I went through the steps of adding a workspace, datastore, and layer, and the layer rendered normally.

@orlade @justb4 Were you expecting different behavior? If you start GeoServer with an empty data directory it's not going to contain the example data normally shipped with GeoServer, that's a given.

[justb4]

Thanks for diving into this! Yes this/above is the behaviour I see: only
those few dirs created on host. But the dir /opt/geoserver/data_dir
within the container initially contains the full default workspaces,
logging and data (the stuff I usually remove first). Given that
/opt/geoserver/data_dir is for the GeoServer webapp pointed as
$GEOSERVER_DATA_DIRI would think that this would/should happen:

• GS uses the full /opt/geoserver/data_dir (including workspaces, logs
  etc) as this is what was in the .war data dir (copied during Docker
  build to /opt/geoserver/data_dir).
• Docker would map that full /opt/geoserver/data_dir to
  host:/tmp/docker_gs_test_data (just like is happening with the Tomcat
  Log dir or kartoza/postgisdata and logs which I store on host:
  https://github.com/Geonovum/smartemission/blob/master/services/postgis/run-postgis.sh)

But the opposite seems to be happening. Maybe I am missing something in
the subtle interaction GS/Docker. At least to my knowledge GS refuses to
start with an empty data dir. Does GeoServer function ok in your case?
Things like logging to files are e.g. disabled, metadata missing etc.

[alexgleith]

Hey @justb4, this is the default behaviour for GeoServer when it encounters an empty data directory. It will populate it with default stuff.

I don't think anything that you're experiencing is unexpected!

[srstsavage]

@justb4 I think the confusion is coming from Docker's normal behavior and has nothing to do with GeoServer. When mounting a host directory as a volume in a Docker container, the host directory's contents replace the container directory's contents (treated like a unix mount command). If the container is run again without the host volume mounted, the original contents of the container's directory are available again.

moby/moby#4361
moby/moby#10947

https://docs.docker.com/engine/userguide/containers/dockervolumes/#mount-a-host-directory-as-a-data-volume

docker run -d -P --name web -v /src/webapp:/opt/webapp training/webapp python app.py
This command mounts the host directory, /src/webapp, into the container at /opt/webapp. If the path /opt/webapp already exists inside the container’s image, the /src/webapp mount overlays but does not remove the pre-existing content. Once the mount is removed, the content is accessible again. This is consistent with the expected behavior of the mount command.

Note that this only happens when you specify a host directory to mount to the container volume. If you used a Docker named volume Docker will manage the storage of your named GeoServer data directory on disk, and you can mount it from other containers to access it, delete it using Docker commands, etc.

In any case, everything I'm seeing is consistent with expected Docker and GeoServer behavior. GeoServer runs well when initialized with an empty data directory.

[justb4]

Thanks @shane-axiom @alexgleith! Yes I was confused by the combination of 2 things (see next), but you have cleared this all up. This issue can be closed AFAIAC. For the record: the two things that caused my confusion:

1. what happens when mounting empty host dir in Docker container when container-dir is non-empty
2. GeoServer creating minimal config when data dir is empty

Ad 1) I thought the host dir would get the initial contents of the container dir when empty. My example/"proof" was: kartoza/postgis: mounting empty host dir to /var/lib/postgresql rendered a completely populated dir on host. But now I see that this happens explicitly in the start-postgis.sh (if [ ! -d $DATADIR ]; then and then initdb) script present the kartoza/postgis Docker image! Sorry for stirring up, am relatively new to Docker.

Maybe my script https://github.com/Geonovum/smartemission/blob/master/services/geoserver/run-geoserver.sh remains useful, possibly triggered by option (ARG) for docker run from Dockerfile, as it allows the same experience (sample workspaces/data and logging dir) as deploying a GeoServer .war 'the old way'.

[srstsavage]

@timlinux Ready for closing.

[timlinux]

Thanks @shane-axiom for your investigating!