Coder Social home page Coder Social logo

otm-docker's Introduction

Opentopomap server

This is a docker setup for a "quick" install of a server like opentopomap.org.

The config files and scripts in this image are directly taken from:

https://github.com/der-stefan/OpenTopoMap

Requirements

A multi processor server with at least 16 GB RAM and a lot of HD space.

A full worldwide server needs about 3 TB of HD to host all tiles up to zoom level 17.

You need docker and docker-compose to run this image.

NOTE

You don't need this repo locally if you use the prepared Docker Image:

jhassler/otm-docker:latest

You can find this image on Dockerhub: https://hub.docker.com/jhassler/otm-docker

You only need this repo if you want to build your own image variant. See below: Build your own image.

Setup your Opentopomap server

Notes

With this image you can setup an OSM tile server with the Opentopomap style.

You can choose which part of the world you want to cover. Everything is possible, from a single country up to the whole world.

Step 1: Prepare your server

Create a directory and put the docker-compose.yml file into it. This file is a copy of the docker-compose.yml.dist file in this repository.

Create the directory data inside your project directory with these subdirectories:

mkdir -p data/data
mkdir -p data/db
mkdir -p data/letsencrypt

Then edit your docker-compose.yml and change the following variables:

  • LETSENCRYPT=0: Change this to 1 if you want to obtain SSL certificates through Let's Encrypt. Use this on a production server.
  • EMAIL=[email protected]: Change this to a valid E-Mail address. This will be used to setup the Let's Encrypt SSL certificates. You can leave it if you use the image only locally.
  • DOMAIN=localhost: Change this to the domain (URL) of your OTM server. This will be used inside the Apache configuration.
  • WHITELIST=127.0.0.1: You can add an IP here that will not be affected by the mod_file throtteling.

This image will expose the HTTP ports 80 and 443 to the outside. Make sure they're not used by some other services on your server. If you want to host the OTM server on a different port, change the outside port section, e.g. change "80:80" to "8080:80" - then your server will be available on port 8080 from the outside world.

Step 2: Choose and download OSM data

You need a PBF file of your favourite region. This can be the full 48 GB planet file (https://planet.openstreetmap.org) or some regional extract. You can find a good list of servers in: https://wiki.openstreetmap.org/wiki/Planet.osm

Download your PBF file and put it into your project data/data directory with the name osmdata.pbf. The script expects the data to have this name.

Step 3: Download elevation data

Download some elevation data files and put the into the local directory data/data/srtm. You have to create the srtm directory.

Here are some locations for elevation data:

The files should be in HGT format compressed as ZIP files. The files will be extracted and converted in the script 04_dem_hillshade.sh.

Make sure the SRTM data covers the region you chose in step 2.

Easiest way: For worldwide SRTM data use:

http://viewfinderpanoramas.org/dem3/M31.zip
http://viewfinderpanoramas.org/dem3/M32.zip
http://viewfinderpanoramas.org/dem3/M33.zip
http://viewfinderpanoramas.org/dem3/L31.zip
http://viewfinderpanoramas.org/dem3/L32.zip

Step 4: Check your data & start the container!

Your directory structure should look something like this:

docker-compose.yml
data/data/osmdata.pbf
data/data/srtm/some-zip-file1.zip
data/data/srtm/some-zip-file2.zip
data/db
data/letsencrypt

Then start the thing:

docker-compose up -d

Check output with docker logs otm-docker -f

This will create your postgres database and start some services. They won't really come up, but the container should be running.

Step 5: Enter the container and start some scripts

This is only needed once. You have to start some scripts to import your data.

docker exec -ti otm-docker bash

Inside start a screen session. Then you can start the long running scripts and don't stop them when you log out.

screen

Once you've started some scripts, press CTRL+a d to detach from the screen session. Then logout of your container with exit. If you come back later and login to your container again, you access the session with screen -r.

Make some adjustments to the scripts depending on your available memory:

  • /scripts/import_osm_data.sh: Change MEMORY=12000 to the MBs of memory you have available inside the Docker session

Execute the following scripts in the given order:

cd /scripts
sh 00_setup_database.sh
sh 01_download_water_polys.sh
sh 02_import_osm_data.sh
sh 03_dem_hillshade.sh
sh 04_preprocess_osm_data.sh
sh 05_dem_contours1.sh
sh 06_dem_contours2.sh

Step 2 (import OSM data) takes the most time, depending on the size of your PBF file. A full planet can even take DAYS to import.

Step 5 (generating the contours data) can also take a long time and needs a lot of RAM, min. 16 GB for the world SRTM. If you get a memory fault when executing, then try it with lower resolution or on a box with more RAM.

Step 6: Almost done...

Once everything has run through, you log out and restart your container:

docker-compose down
docker-compose up -d

Step 7: See your map!

Now you should see your map by accessing your server's mapdemo directory, e.g.: http://localhost/mapdemo

The tile layer is available with the following URL http://your-server/otm/{z}/{x}/{y}.png. You can include it in your favourite mapping applications.

Optional stuff

Rendering your tiles on demand usually takes too much time. It's better to pre-render them in the background. You can do that with Tirex. To render e.g. all tiles in Switzerland for zoom levels 1-16, this command can be used:

tirex-batch -p 5 -d map=opentopomap bbox=6.0,45.78,10.44,47.83 z=1-16

mod_tile considers a tile as "old" if it's older than 3 days and triggers a re-render. If you want to prevent all re-renders and manually re-render the tiles by controlling Tirex, you can start the container with the following env var:

MOD_TILE_PREVENT_EXPIRATION=1

Build your own image

DOCKER_BUILDKIT=1 docker build -t otm-docker:latest .

otm-docker's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

otm-docker's Issues

typo in the instructions

Nothing major, in step 5 the docker command is referencing the otm-server container instead of the otm-docker container.

Container crashes when calling docker-compose up after docker-compose down

The container holds /etc/postgres files locally. After an docker-compose down the current state of these files is lost. The subdir 'conf.d' does not exist, which is referenced in the postgres.conf file. Also other files are missing.

As a workaround you can move the data directory out of the way, create a new data/data and restart the container. The container will rebuild a new postges configuration. Then stop the container, reset the data directory to the one with the real data und restart the container with docker-compose start

docker exec error

Hi,

docker image installation

docker image installation


docker log result

docker log result


docker exec result

docker exec result


docker ps result

docker ps

Thanks.

Hillshade & relief offset from the map

Thanks for creating this docker image, it simplifies setup A LOT!

But for me the resulting tile server has the relief and hillshade offset a few kilometers north east.
I went through all the configs, but can't find anything that could be the reason for this. I checked the scripts against https://github.com/der-stefan/OpenTopoMap/blob/master/mapnik/README.md, but again, nothing seems wrong.

Note the white band to the left. It's not snow (:

Screenshot 2022-08-05 at 15 47 21

Contour lines seem to be at the right place, but the relief is offset.

Screenshot 2022-08-05 at 15 48 30

This is the same location in opentopomap.org

Screenshot 2022-08-05 at 16 13 07

Both the contour lines and the hillshade & reliefe are derived from the same SRTM data via http://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm

I first tried with the complete South America pbf and SRTM for full coverage, which took days, then again from scratch with just Uruguay and only the Uruguay SRTM files, being the smallest filesize and quickest to redo. Both times the same result.
Is this a misconfiguration of the map & server or an issue with rendering?

Missing /etc/postgresql/10/main/conf.d directory

Hello,

First try using this wonderful work, but when following the doc, after the first stop and start, I get :

Creating otm-docker ... done
Attaching to otm-docker
otm-docker    |  * Starting PostgreSQL 10 database server
otm-docker    |  * Removed stale pid file.
otm-docker    | Error: /usr/lib/postgresql/10/bin/pg_ctl /usr/lib/postgresql/10/bin/pg_ctl start -D /var/lib/postgresql/10/main -l /var/log/postgresql/postgresql-10-main.log -s -o  -c config_file="/etc/postgresql/10/main/postgresql.conf"  exited with status 1: No such file or directory
otm-docker    | 2021-01-05 14:57:13.827 GMT [21] LOG:  could not open configuration directory "/etc/postgresql/10/main/conf.d": No such file or directory
otm-docker    | 2021-01-05 14:57:13.827 GMT [21] FATAL:  configuration file "/etc/postgresql/10/main/postgresql.conf" contains errors
otm-docker    | pg_ctl: could not start server
otm-docker    | Examine the log output.
otm-docker    |    ...fail!
otm-docker exited with code 1

and indeed, I find no conf;d directory in main dir.

python-gdal not included

Hi,

thank you for this project! I had to install python-gdal for script 03. Also typo in the README: not all leading 01_, 02_ etc. are given.

apt-get install python-gdal

or

apt-get install gdal-bin

Andreas

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.