maproulette / maproulette2-docker Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 13.0 134 KB

Deployment through docker for MapRoulette.

License: Apache License 2.0

Shell 80.55% Dockerfile 19.45%

maproulette2-docker's People

Stargazers

Watchers

Forkers

cstaylor mgcuthbert mvexel nvkelso simonpoole zverik nrotstan zerebubuth gaoxm kosmitldz reichg ljdelight

maproulette2-docker's Issues

Nginx routing rules cause HTTP 404 for favicons

The nginx configuration needs a tweak to fix favicons for all browsers. This needs some care because the favicons are in the frontend container and the /assets/ route is needed for swagger docs in the api container.

dbPort argument, when set, is passed to docker in the wrong order

The deployment dbPort argument's intended use is to set the host's local listening port to a custom value. The param is passed to docker in the reverse order: it sets the host's 5432 port to forward to the containers <dbPort> port. The syntax is docker ... -p <host-port>:<container-port>.

Also I think it would be valuable to change the default to allow the container to bind to localhost (-p 127.0.0.1:5432:5432) to simplify db connections via pgadmin.

MIME type compression is not done for js responses

Frontend `customLayers.json` needs documented and a template file

The customLayers.json is an optional file to provide during the docker build process, but users must know (1) that this is possible, (2) know to modify the Dockerfile to add the file. These modifications make it difficult to fetch the latest updates of the repository, and it would help if this were simplified.

The frontend customLayers.json needs documented here in the docker project and a template file created so that a user's local layer additions are not within the scope of a 'git diff'. If it's possible, it is preferred to have docker runtime overrides for optional content vs requiring a user to include a file during the docker build process.

During the yarn build of the frontend, these lines are in the output and may be helpful.

$ NODE_ENV=production node scripts/update_layers.js
Fetching latest layer data
Extracting default layers
Creating stub for src/customLayers.json -- you can add any custom layers to it

Support a deployment flag to disable the use of the container build cache

At present, our deployment scripts leverage the image build cache during the build process. This approach, while efficient, can sometimes lead to confusion and frustration, particularly in instances where the cache is utilized unexpectedly. To address this, we propose introducing a deployment flag that would provide the option to disable the cache. This addition would offer greater control and transparency in the build process, helping to alleviate any uncertainties related to cache usage.

Support a '--dry-run' flag to show what repos, images, etc would be attempted

retire or update https://hub.docker.com/r/matthieun/maproulette/

Matthieu N uploaded https://hub.docker.com/r/matthieun/maproulette/ to docker hub when he first created it. We should probably (ask him to) remove it or else update it based on more recent work.

running `docker.sh` still wipes DB

I just updated maproulette.org (see log) and the DB got wiped again.

DOCKER_USER ?

Is DOCKER_USER a variable that the user should set? It is not set by docker on my machine. Running export DOCKER_USER=mvexel && ./run_docker.sh works.

Swap deprecated mdillon/postgis image with postgis/postgis

The mdillon/postgis image (PostgreSQL 11.2, postgis 2.5) hasn't been updated in a couple of years. The mdillon/postgis docker hub now references the postgis/postgis image.

https://hub.docker.com/r/mdillon/postgis/
https://hub.docker.com/r/postgis/postgis
https://github.com/postgis/docker-postgis

Include a version file within the docker images to easily identify what git repo and tag was used for the image

After an image is built, it's not super easy to determine what it was based on. It is helpful to easily identify the source version without depending on the image tags.

It could look like this:

# cat /.version
repo=git:maproulette/maproulette2
tag=release/v4.2.0

MapRoulette cannot connect to the database

Tested the connection between docker containers with python, and it works. Did a successful psycopg2.connect(). But for some reason, maproulette cannot connect:

INFO - Creating Pool for datasource 'default'
Oops, cannot start the server.
Configuration error: Configuration error[Cannot connect to database [default]]
	at play.api.Configuration$.configError(Configuration.scala:156)
	at play.api.Configuration.reportError(Configuration.scala:990)
	at play.api.db.DefaultDBApi.$anonfun$connect$1(DefaultDBApi.scala:48)
	at play.api.db.DefaultDBApi.$anonfun$connect$1$adapted(DefaultDBApi.scala:42)
	at scala.collection.immutable.List.foreach(List.scala:389)
	at play.api.db.DefaultDBApi.connect(DefaultDBApi.scala:42)
	at play.api.db.DBApiProvider.get$lzycompute(DBModule.scala:86)
	at play.api.db.DBApiProvider.get(DBModule.scala:75)
	at play.api.db.DBApiProvider.get(DBModule.scala:56)
	at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81)
	at com.google.inject.internal.BoundProviderFactory.provision(BoundProviderFactory.java:72)
	at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61)
	at com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:62)
	at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:54)
	at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:132)
	at com.google.inject.internal.MembersInjectorImpl$1.call(MembersInjectorImpl.java:93)
	at com.google.inject.internal.MembersInjectorImpl$1.call(MembersInjectorImpl.java:80)
	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
	at com.google.inject.internal.MembersInjectorImpl.injectAndNotify(MembersInjectorImpl.java:80)
	at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:62)
	at com.google.inject.internal.InjectorImpl.injectMembers(InjectorImpl.java:987)
	at com.google.inject.util.Providers$GuicifiedProviderWithDependencies.initialize(Providers.java:149)
	at com.google.inject.util.Providers$GuicifiedProviderWithDependencies$$FastClassByGuice$$2a7177aa.invoke(<generated>)
	at com.google.inject.internal.SingleMethodInjector$1.invoke(SingleMethodInjector.java:54)
	at com.google.inject.internal.SingleMethodInjector.inject(SingleMethodInjector.java:89)
	at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:132)
	at com.google.inject.internal.MembersInjectorImpl$1.call(MembersInjectorImpl.java:93)
	at com.google.inject.internal.MembersInjectorImpl$1.call(MembersInjectorImpl.java:80)
	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085)
	at com.google.inject.internal.MembersInjectorImpl.injectAndNotify(MembersInjectorImpl.java:80)
	at com.google.inject.internal.Initializer$InjectableReference.get(Initializer.java:223)
	at com.google.inject.internal.Initializer.injectAll(Initializer.java:132)
	at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:174)
	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)
	at com.google.inject.Guice.createInjector(Guice.java:99)
	at com.google.inject.Guice.createInjector(Guice.java:84)
	at play.api.inject.guice.GuiceBuilder.injector(GuiceInjectorBuilder.scala:185)
	at play.api.inject.guice.GuiceApplicationBuilder.build(GuiceApplicationBuilder.scala:137)
	at play.api.inject.guice.GuiceApplicationLoader.load(GuiceApplicationLoader.scala:21)
	at play.core.server.ProdServerStart$.start(ProdServerStart.scala:51)
	at play.core.server.ProdServerStart$.main(ProdServerStart.scala:25)
	at play.core.server.ProdServerStart.main(ProdServerStart.scala)
Caused by: Configuration error: Configuration error[Failed to initialize pool: This ResultSet is closed.]
	at play.api.Configuration$.configError(Configuration.scala:156)
	at play.api.Configuration.reportError(Configuration.scala:990)
	at play.api.db.HikariCPConnectionPool.create(HikariCPModule.scala:63)
	at play.api.db.PooledDatabase.createDataSource(Databases.scala:205)
	at play.api.db.DefaultDatabase.dataSource$lzycompute(Databases.scala:123)
	at play.api.db.DefaultDatabase.dataSource(Databases.scala:121)
	at play.api.db.DefaultDatabase.getConnection(Databases.scala:142)
	at play.api.db.DefaultDatabase.getConnection(Databases.scala:138)
	at play.api.db.DefaultDBApi.$anonfun$connect$1(DefaultDBApi.scala:44)
	... 39 more
Caused by: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: This ResultSet is closed.
	at com.zaxxer.hikari.pool.HikariPool.throwPoolInitializationException(HikariPool.java:569)
	at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:548)
	at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
	at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
	at play.api.db.HikariCPConnectionPool.$anonfun$create$1(HikariCPModule.scala:51)
	at scala.util.Try$.apply(Try.scala:209)
	at play.api.db.HikariCPConnectionPool.create(HikariCPModule.scala:47)
	... 45 more
Caused by: org.postgresql.util.PSQLException: This ResultSet is closed.
	at org.postgresql.jdbc2.AbstractJdbc2ResultSet.checkClosed(AbstractJdbc2ResultSet.java:2654)
	at org.postgresql.jdbc2.AbstractJdbc2ResultSet.setFetchSize(AbstractJdbc2ResultSet.java:1771)
	at org.postgresql.jdbc4.Jdbc4Statement.createResultSet(Jdbc4Statement.java:39)
	at org.postgresql.jdbc2.AbstractJdbc2Statement$StatementResultHandler.handleResultRows(AbstractJdbc2Statement.java:211)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1773)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374)
	at org.postgresql.jdbc2.AbstractJdbc2Connection.execSQLUpdate(AbstractJdbc2Connection.java:263)
	at org.postgresql.jdbc2.AbstractJdbc2Connection.getTransactionIsolation(AbstractJdbc2Connection.java:775)
	at com.zaxxer.hikari.pool.PoolBase.checkDriverSupport(PoolBase.java:457)
	at com.zaxxer.hikari.pool.PoolBase.setupConnection(PoolBase.java:412)
	at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:370)
	at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:194)
	at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:460)
	at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:534)
	... 50 more

Might be related to this issue, although I have no idea.

`frontend/.env.production` should be a template file

The frontend/.env.production contains many private keys and secrets which should not be made private. To avoid accidents, the file should be a template file that an operator must copy and modify and ignored within .gitignore. The deployment should fail if the file does not exist.

UI container image build: "caniuse-lite is outdated. Please run next command `yarn upgrade`"

When the front-end container builds, it prints many warning messages:

$ /maproulette-frontend/node_modules/.bin/extract-messages -l=en-US -o src/lang/ -d en-US --flat -f json 'src/**/!(*.test).js'
Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db
$ NODE_ENV=production node scripts/update_layers.js
Fetching latest layer data
Extracting default layers
$ postcss src/styles/index.css -o src/index.css
Browserslist: caniuse-lite is outdated. Please run next command `yarn upgrade`
Creating an optimized production build...
Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db
Browserslist: caniuse-lite is outdated. Please run next command `yarn upgrade`
Browserslist: caniuse-lite is outdated. Please run next command `yarn upgrade`

Does this need accounted for during the build process or should it be a UI yarn improvement?

Remove docker `--privileged` argument?

The docker run commands include the --privileged flag, but this doesn't appear to be necessary. The ask here is to see if anything bad happens without the flag and remove if possible.

Issues when deploying branches that are not 'LATEST|^[0-9v]'

Performing a deployment with ./deploy.sh --api ljdelight/test will actually deploy LATEST and reports no error. This should be an error or (preferred) fetch the requested branch during the deploy.

The specific check is [[ "$2" = "LATEST" ]] || [[ "$2" =~ ^[0-9v] ]]. This applies to both the API and FE deployments.

Separate out configuration in `deploy.sh` to avoid git conflicts

The configuration variables within deploy.sh don't exactly work for most users, and when a user changes the variable there's a git diff that will complicate future 'git pull's. Those vars should be isolated as much as possible to avoid git conflicts.

`api/docker.conf` should be a template file

The api/docker.conf contains many private keys and secrets which should not be made private. To avoid accidents, the file should be a template file that an operator must copy and modify and ignored within .gitignore. The deployment should fail if the file does not exist.

API container logs are written within the container and are difficult to obtain

The maproulette-api container's bootstrap.sh starts the server, redirecting its output to a file within the container (./setupServer.sh > setupServer.log 2>&1).
This is a pain point because when the container is redeployed from the image, the log history is destroyed, so debugging takes more effort and there's great complexity centralizing logs.

Use a volume or bind mount for the mr-postgis database container's persistent data

It's not possible to change database versions/containers without destroying all of the data. A nice enhancement to the deployment scripts is to use a volume or bind mount for the database container's data. The script's wipeDB logic would need to be updated to account for a mount.

The larger problem without having a mount is the data is lost when the container need to be entirely recreated, such as when changing docker -p port mappings and other container variables.

Deployment script failures doesn't cause the scripts to stop executing

The scripts keep going after a failure occurs. Minor issue and and easily fixed with set -eu in a few places.

maproulette / maproulette2-docker Goto Github PK

maproulette2-docker's People

Stargazers

Watchers

Forkers

maproulette2-docker's Issues

Recommend Projects

Recommend Topics

Recommend Org