lgtm-migrator / ajapaik-web

Crowdsourcing of data for historic imagery (geotagging, face tagging, rephotography)

Join Ajapaik Slack!

This is the open-sourced Django project code for https://ajapaik.ee/

Running locally

docker pull laurielias/ajapaik-web:python-3.8.10
cp ajapaik/ajapaik/settings/local.py.example ajapaik/ajapaik/settings/local.py
docker-compose up -d

Build it yourself and launch

python-3.8.10-dlib is just python:3.8.10 with dlib and its dependencies installed. (compiling takes many minutes otherwise)

docker pull laurielias/python-3.8.10-dlib
docker-compose up --build

Multi-Device Testing

We use BrowserStack to test on a variety of devices and operating systems.

Restore data from a dump

CREATE DATABASE rephoto_production_20190511;
CREATE USER rephoto WITH ENCRYPTED PASSWORD '';
GRANT ALL PRIVILEGES ON DATABASE rephoto_production_20190511 TO rephoto;

psql -d rephoto_production_20190511 -U postgres < rephoto_20210426.schema.dump
# Data only, no integrity checks while loading it in, no privileges
pg_restore rephoto_20210426.sql -d rephoto_production_20190511 -a -x --disable-triggers -U postgres

Push new image

docker push laurielias/ajapaik-web:python-3.8.10

Debug the container

If need be override the entrypoint in docker-compose.yml to tail -f /dev/null or the like. (in case the current entrypoint crashes, for example)

docker exec -it ajapaik bash

Starting with a fresh DB, add a Django superuser

In the container:

python manage.py createsuperuser

Deploy on our server - currently still using old school supervisord + uwsgis

Make sure you have local.py (mostly secret Django settings) and client_secrets.json (Google credentials) in your project root. They will be mounted into the container on startup. Make sure the nginx on the host knows how to proxy traffic to this container. Also symlink the media directory (the one with all the photos) into your project root, same for the Postgres data directory. Push/pull images again to update.

docker-compose up -f docker-compose.dev.yml

Update Juks' Vanalinnad data

wget -r --no-parent -A empty.json,layers.xml http://vanalinnad.mooo.com/vector/places/
wget -r --no-parent -A jpg --reject-regex "(.*)\?(.*)" http://vanalinnad.mooo.com/raster/places/

Misc. running instructions

On your local machine cp local.py.example local.py to get a quick start.

Fix for 'django.contrib.gis.geos.error.GEOSException: Could not parse version info string "3.6.2-CAPI-1.10.2 4d2925d6"': https://stackoverflow.com/questions/18643998/geodjango-geosexception-error

Installing Postgres: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-18-04

May be of help: ALTER USER ajapaik WITH PASSWORD 'seekrit'; GRANT ALL PRIVILEGES ON DATABASE ajapaik TO ajapaik;

Ajapaik depends on Postgres PostGIS functionality, with a fresh-enough Postgres, installation should be easy: http://trac.osgeo.org/postgis/wiki/UsersWikiPostGIS23UbuntuPGSQL96Apt http://trac.osgeo.org/postgis/wiki/UsersWikiPostGIS24UbuntuPGSQL10Apt

You'll need your own local settings in ajapaik/settings/local.py. You should at least override or specify the following keys:

• ADMINS
• MANAGERS
• DATABASES
• SECRET_KEY
• LOGGING
• GOOGLE_API_KEY
• GOOGLE_MAPS_API_KEY
• GOOGLE_ANALYTICS_KEY
• ALLOWED_HOSTS

Running tests

source venv/bin/activate
python manage.py test --settings=ajapaik.settings.test --nomigrations --keepdb

To-do list

• TODO: command for regular stats exports
• TODO: fix core dump ageitgey/face_recognition#11
• TODO: automate stats queries or at least document them better (should be possible with a Google Sheets API key?)
• TODO: try if integrating Solr tighter will help search (the current solution where everything that matches 'Tartu' is retrieved into an array of IDs no longer performs)
• TODO: implement this answer for keeps https://stackoverflow.com/questions/18643998/geodjango-geosexception-error
• TODO: automate regular DB, media/uploads, media/videos backups
• TODO: fix # noqa as much as possible (some Django quirks will always annoy flake8 though)
• TODO: replace face_recognition with something else since it requires the horrendous dlib
• TODO: don't install test/lint requirements in production Docker