opendataservices / epds

EPDS

Development Guide

Postgis required, docker option below on how to install this easily.

With docker-compose

To execute things in the data getter container, run:

sudo docker-compose up -d
sudo docker-compose exec getter /bin/bash

Inside the getter container you can then use psql or run the python scripts, eg:

psql postgresql://epds:epds@db/epds

python scrapers/planit.py

See Loading data below for more.

Without docker-compose

This will create a local postgres instance binding to 5432 port on the local computer with user, database and password all epds

sudo docker run --name epds-postgis -e POSTGRES_USER=epds -e POSTGRES_PASSWORD=epds -d -p 5432:5432 postgis/postgis

To connect using local psql client use:

psql postgresql://epds:epds@localhost:5432/epds

The connection string postgresql://epds:epds@localhost:5432/epds can be used to connect from other applications.

Loading data

Use the included SQL to load regional data into the PostGIS store.

psql postgresql://epds:epds@localhost:5432/epds -f data/local_nature.sql
psql postgresql://epds:epds@localhost:5432/epds -f data/national_nature.sql
psql postgresql://epds:epds@localhost:5432/epds -f data/iba.sql
psql postgresql://epds:epds@localhost:5432/epds -f data/sssi.sql

Planit data

If you're not using docker-compose, don't forget to create the output directory.

python scrapers/planit.py
python scrapers/planit_load.py
psql -f scrapers/planit_load.sql

Importing new datasources

From shapefile

Install postgis package locally, but no need to run postgres locally, as it includes shp2pgsql.

Put all data in the 4326 SRID, this is the lat/long system that most point data is in.

Run shp2pgsql using a command like:

   shp2pgsql -s 27700:4326 data/National_Nature_Reserves_England.shp > data/nationl_nature.sql

• 27700 is the SRID code (projection) of the shape file. Need to look at documentation of the data to find that.
• 4326 is the target SRID (always 4326)
• data/National_Nature_Reserves_England.shp is the shapefile
• data/nationl_nature.sql output SQL that can be executed to import the data.

Try and document licence with attrubution in data/data_licences.txt

epds's Issues

Possible issue for 'solar' applications in SSSIs.

I used an application "Land At Fiddlers Green Carland Cross Windfarm St Newlyn East Cornwall" as a test case which is located inside an SSSI. Today, that application was not identified in our EPDS search or in my notebook.

It may be that there are two separate issues.

1. Generally, the problem for my notebook seems to be that today, planit only returns 300 cases when searching for 'solar' regardless of date range. Normally, planit returns 1000+ solar cases for the last three months. Going forward, we might ask is planit reliable?
2. I don't think that planit should affect our database. Possibly, our database is not returning any pins for solar applications and is only showing the SSSI boundaries?

The coordinates for the application are 50.346715,-5.043649 but here is png of our map and planit map search.