nickbearman / foss4guk2022local

FOSS4G:UK 2022 Local

Using the OS Data Hub to automate Open Data ETL to open source databases

Introduction

OS OpenData Downloads is a set of free, downloadable digital maps and comprehensive data of Great Britain, available for anyone to use, for any purpose.

The OS Downloads API is a service that lets you automate the discovery and download of OS OpenData and data packages.

The scripts included in this repository will allow you to:

1. Connect to the OS Downloads API and return a listing of the available Open Data products.
2. Download a product (in the format) of your choosing.
3. Extract and load the data into a spatially-enabled database.

Requirements

• Jupyter Notebook => Tool for interactively developing and presenting data science projects.
• PostgreSQL => An open-source relational database management system (RDBMS).
• PostGIS => Spatial database extender for PostgreSQL object-relational database.
• Geospatial Data Abstraction Library (GDAL) => Translator library released by the Open Source Geospatial Foundation for reading/writing raster and vector geospatial data formats.

Licence

The contents of this repository are licensed under the Open Government Licence 3.0.