Coder Social home page Coder Social logo

vitaly-t / niche-api Goto Github PK

View Code? Open in Web Editor NEW

This project forked from scottsfarley93/niche-api

0.0 3.0 0.0 51.21 MB

API to serve climate data for points and polygons through time

Shell 0.42% Ruby 32.88% JavaScript 21.51% HTML 44.90% Python 0.29%

niche-api's Introduction

Niche API

Scott Farley

UW Madison

Creates a web-based data service that serves global climate model data at a specific point in space and time for a given set of climate variables.

Gridded climate model output is great if you want to examine the spatial patterns of climate in a single time slice, but is difficult to use if you want to track the trajectory of a specific point in space (x, y) in climate space through time. I know of no way to programmatically and efficiently get the value of climatic variables (maximum temperature, minimum temperature, precipitation, etc) at a single grid cell at a time slice using standard web-based repositories. My goal is to create a data service that allows a client to query a space-time point (e.g., 37N, -122W, 1250 years ago) for a specific climate layer and return the information in a consumable format (json). I’d like whatever I build to be web-based, and have the heavy lifting done on the server side, so that the user doesn’t need to download the whole data file (e.g., netcdf) which would be a lot of data transfer and probably contain a lot of unnecessary information. I’d also like to build out a REST interface so that the querying can be efficiently done by any client. The service would serve downscaled CCSM3 climate model output for North America between 22,000 years ago and the present, as well as, potentially, other climate models and/or other gridded datasets (soils, land use) though both of these seem like they’d be much more complicated.

Working Demo

Click a point on the map to get a 22,000 year time series of temperature for that place on earth.

Here
<iframe src="http://grad.geography.wisc.edu/cds/map.html" id='demo'></iframe>

Technical Configuration

Database: Run a Postgres database with PostGIS spatial extensions. Store each variable (e.g., January precipitation, January maximum temperature,…) as a separate postgres raster table and each time slice (e.g, 22000 years ago, 21900 years ago, 21800 years ago…) as a separate raster band. Yields tables with several hundred to thousands of bands depending on temporal resolution of climate data. Store a pointer to that table in a central index table that keeps track of the layer’s metdata, as well as a pointer to then interpretation of the bands (100 year resolution, 500 year resolution, variable resolution,...). When a client issues a query, search the index table to find the row(s) that meet the user’s query. Go to the raster tables that are referred by each of these rows and do a point-in-raster value extraction. Send back the values at those points in those tables.

Server/Service: Run a Node.js server interface. Using the Express.js and Postgres (pg) bindings for node, parse client requests and send them to the database. When the database responds, package up the json and send the response back to the client.

Libraries:

  • PostGIS: Management of raster spatial data, handles geometry conversion on incoming calls
  • Swagger (swagger-express-mw): Ensures proper API returns by validating against a schema. Documents the API.
  • pg-promise: Simplified connection to postgres database.
  • express: Web framework for Node.js
  • uuid: Node library for generating universally unique identifiers. Used to assign IDs to client requests.
  • cors: Allow cross-origin requests from this server.
  • compression: Return all responses from the application as gzip compressed files.

Projects Using the Niche API:

Example Calls

Get a list of all climate variables in the database:

http://grad.geography.wisc.edu:8080/variables?

Get a list of all the data sources in the database:

http://grad.geography.wisc.edu:8080/sources?

Get a 22,000 year time series of January temperatures from Berkeley, CA (37.88N, -122.26W):

http://grad.geography.wisc.edu:8080/timeseries?latitude=37.88&longitude=-122.6&sourceID=6&variableID=28

Get the summer (July) maximum temperature in New York City (40.71N, 74.01W) 15,200 years ago:

http://grad.geography.wisc.edu:8080/data?latitude=40.71&longitude=-74.01&sourceID=6&variableID=28&yearsBP=15200

Get the summer (July) maximum temperature in the following locations:

Latitude Longitude Years BP
42.12 -90.74 964
50.3 -121.34 11230
60.92 -118.56 19290

Multiple points are requested using a POST request. Here's a cURL command to issue such a request.

curl -X POST -H "Content-Type: application/json" -H "Cache-Control: no-cache" -H "Postman-Token: 4cd7aa0d-3650-fe96-d822-8c4d1a321553" -d '{	"variableID": 34,
	"sourceID": 6,
	"points": [
		{
			"latitude": 42.12,
			"longitude": -90.74,
			"year": 964
		},
		{
			"latitude": 50.3,
			"longitude":-121.34,
			"year": 11230
		},
		{
			"latitude": 60.92,
			"longitude": -118.56,
			"year":19290
		}
	]
}' "http://grad.geography.wisc.edu:8080/data"

Project Homepage

To see the project homepage and any recent updates, go here.

Documentation

To see all the docs, go here.

niche-api's People

Contributors

scottsfarley93 avatar vitaly-t avatar

Watchers

 avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.