openskope / skopeui

SKOPE user interface and visualization app

Synthesizing Knowledge of Past Environments

This codebase provides a frontend only user interface for the NSF supported Synthesizing Knowledge of Past Environments project.

The current UI uses Nuxt 2, VueJS, Leaflet, and Plotly JS.

The application has a basic Makefile for configuration and deployment. A recent version of Docker and docker compose is recommended to set up a local development environment.

Running make for the first time will generate a default config.mk file that can be further customized:

• DEPLOY_ENVIRONMENT can be set to dev, staging, or prod (use dev for local development)
• SKOPE_API_HOST_URL should point at a SKOPE backend API to serve metadata and timeseries data (e.g., a deployed instance of https://github.com/openskope/skope-api/)
• SKOPE_GEOSERVER_HOST_URL should point at a GeoServer instance with SKOPE datasets and geotiff layers preloaded (see https://github.com/openskope/skope-datasets for more details)

Run make deploy to redeploy the application with whatever settings are currently defined in config.mk

With DEPLOY_ENVIRONMENT=dev a hot-reloading development server should spin up at http://localhost:3000

Contributors

• Allen Lee @alee
• Kyle Bocinsky @bocinsky
• Calvin Pritchard @cpritcha
• Christine Nguyễn @chrstngyn

skopeui's Issues

layers appear offset incorrectly when zoomed out

Zooming way out when a layer is loaded makes the layer skip around a little bit on the map with incorrect borders.

As the simplest possible solution, we could disable zooming out too far

Or look into if we can improve / correct the translation from the EPSG4326 geoserver layers mapped onto the web mercator base map tiles

Or load in a EPSG4326 base map to start with?

circle geometry does not average correctly

• circles do not appear to be supported by the backend processing via GDAL generating incorrect average values
• look into converting circles into approximate polygons

get skope prototype ready for production

• switch from dev server to production deployment: https://nuxtjs.org/guide/commands/
• add favicon #21
• fix search #20
• fix plotly chart download #19
• improve landing page UI
• add header banner (https://www.openskope.org/wp-content/uploads/2018/10/cropped-Header-New-3.png) ?
• change "Draw a marker" label to "Select a point"
• hide detailed information in a "More info" collapsable div
• add go to beginning & end buttons to year control

resize geometry does not trigger timeseries update

steps to reproduce:

1. create geometry (circle)
2. select a layer
3. resize circle
4. timeseries does not update

Moving the geometry does trigger an update though

New basemap in SKOPE UI?

This is just to start a discussion about the default basemap and selection of baseman's in the SKOPE UI. Some considerations:

• Esri.WorldTopoMap is nice in the sense that it is very easy to identify your area of interest quickly
• A monochrome basemap, though, may help interpretation by not interacting (color-wise) with the datasets that are overplayed (and usually semi-transparent)
• We could also build our own basemap using a combination of web map services with vector tiles, but each additional layer increases load time (though marginally)
• it might also be nice to have tribal land represented, as on Esri.WorldTopoMap

The Montana Mesonet uses one of these custom basemaps, but it doesn't have park service units or tribal land demarcated: https://mesonet.climate.umt.edu/map. This uses hillshades from MapTiler and Stamen toner-lines and toner-labels.

develop handcrafted UI for provenance information

work with @bocinsky and the science team to establish a first cut at what a useful visualization of the provenance for Paleocar (and other datasets) would look like on the UI

improve polygon management

users should be able to upload polygons as GeoJSON to define the study area / area of interest

look into keeping the GeoJSON object persistent in memory for the duration of their session so they can apply the polygon to different datasets without having to reupload every time

downloading drawn polygons as GeoJSON would be useful as well

flesh out landing page

• redesign landing page with mockups and a few alternative layouts (reduce whitespace etc)
• needs text describing the application and how to use it
• create some mockups for landing page layouts so we can figure out how much space is available etc

display data pixel boundaries on layer

When zoomed in on the map it would be useful to display the pixel boundaries on the layer so people can figure out if the chart data is sensible or what it's taking into account.

To solve this we could:

1. modify the backend https://github.com/openskope/timeseries-service to return additional geo data demarcating the data pixels around the input boundary
2. always render the pixel boundaries in the client ala https://leafletjs.com/examples/choropleth/ (do we pull this from geoserver?)

Please feel free to chime in if I misunderstood anything @tmcphillips @bocinsky ! Thanks 👍

add basic stats in addition to timeseries chart

Mean, sample std, median for both selected temporal range and whole available temporal range

backend: user accounts with persistent data

set up socialauth user accounts on the backend (orcid / github), let the user:

• save analytical boundaries
• additional things here

upgrade to vuetify 2.0

https://github.com/vuetifyjs/vuetify/releases/tag/v2.0.0#user-content-upgrade-guide

this is a major change, as there are quite a few backwards-incompatible vuetify components that we will need to update - after upgrading the site is completely unusable / degraded in its current form of markup. We should consider if we want to undergo this pain now or kick the can down the road.

As the application is currently not very large I think it's probably best to take the pain now.

timeseries chart issues

Keith and I witnessed some strange behavior on the chart that we will try to reproduce where the x-axis year ticks were being displayed as 1.9, 2.0, 2.1 etc., and the y-axis data ranges were off. Could have been a 1-off anomaly or an error in the underlying data cube, but @kintigh will try to figure out steps to reproduce.

• temporal constraints have been added by @cpritcha as of fa53616 👍
• edit polygon doesn't update timeseries chart properly when passing through pixel boundaries. fixed by d129e9c

push geometry persistence into store

We should move the user-specified geometry into the Vuex store like @cpritcha did in 5e4ae2d - this is then responsible for persistence / restoration in local storage etc.

We deliberately keep just a single global geometry in local storage - represents the "area of interest" and management is delegated to the user who can download and manage them as geoJSON files locally.

Transform graph to Z-score relative to period from X to Y

Three options for computing the z-score:

1. moving interval (fine as is)
2. selected temporal interval (without years)
3. fixed temporal interval: include a year range, compute the mean + stdev for that year range and compute the z-score based on that mean/stdev for that given year range, not the selected temporal interval

take mean and stdev over the entire series or a window in the series and recast / replot the series in reference

stretch goal: sliding Z-score metric, how is your experience this year related to the cumulative experience across some user-defined window in the series?

add smoothing functions

• Trailing running average of N years
• Centered running average of N years
• Spline fit to timeseries
• Kernel density smoothing (lower priority)

Issues:

• moving averages may have incomplete windows of data, response should always have a slot for alerts / messages from the server that could be displayed (e.g., @tako2428 suggests an alert to the user saying "Portions of this window have missing data")

download timeseries chart as png generates an error

clicking on the Download Chart button works but the download chart as png icon in plotly causes an error (perhaps we just disable it?)

refine dataset detail and data visualization UI

https://plotly.com/dash/

We don't have the resources to switch to dash now (it was a little too rough around the edges when we were initially evaluating frameworks) but we can still look to their showcases list for inspiration on how to design a better UI for displaying the map + stats. We should select a few exemplar interfaces that we think could be used for the dataset detail page UI and then refactor the SKOPE UI to match.

Showcases: https://dash-gallery.plotly.host/Portal/

Dashboards with maps:

• https://dash-gallery.plotly.host/dash-opioid-epidemic/
• https://dash-gallery.plotly.host/dash-uber-rides-demo/

data population for frontend

Current plan is to set up the frontend using the static JSON files for metadata

https://github.com/openskope/Datasets/tree/master/datasets

Include prism, lbda, paleocar-noaa, and srtm

display error message responses from timeseries service on UI

When we receive a 400 response from the timeseries service we should be displaying the error message on the dashboard page:

{"timestamp":1601655630007,"status":400,"error":"Bad Request","exception":"org.openskope.timeseries.controller.InvalidArgumentException","message":"The selected area does not overlap the region covered by the dataset","path":"/timeseries-service/api/v1/timeseries/paleocar_2/growing_degree_days"}

renew app.openskope.org ssl cert

need to renew ssl certs for *.openskope.org - cert expired on 12/8/2019

app.openskope.orgis causing ERR_CERT_DATE_INVALID errors on skope.comses.net

enable basemap switching

add control to switch basemaps

• https://leaflet-extras.github.io/leaflet-providers/preview/
• Stamen.TonerLite (no color)
• Esri.WorldTerrain
• CartoDB.Positron
• Esri.WorldTopoMap

timeseries chart off by one

It appears to be an off-by-one on the X axis labels in the chart, as a range of 520-550 generates a chart that ranges from 519-549

sidebar functionality

• time span slider doesn't display lower-bound numerical value when using the slider
• checkbox list returns error when boxes are checked

make variable filter consistent with colored pills on the dataset summary display

variables on the left dataset filter pane should be visually consistent with their display in the block view

consider switching to nuxt/typescript

https://nuxtjs.org/guide/typescript

add timeseries service

Existing Plotly code:

https://github.com/openskope/skope-interface/blob/bcbe42fd166be48451d473b9f06763a8a5c45f00/meteor-app/imports/ui/pages/dynamic-workspace/suites/dataset/AnalyticsChart.js

Example request URLs:

https://app.openskope.org/timeseries-service/api/v1/timeseries/lbda-v2/palmer_modified_drought_index?start=0001&end=2017&timeResolution=year&timeZero=0&

with request payload:

{"boundaryGeometry":{"type":"Point","coordinates":[-115.751953125,39.287109375]}}

geoserver integration

Need to check if Leaflet can do what we want it to do with the map layers currently embedded in our geoserver instance.

https://leafletjs.com/examples/wms/wms.html

https://travishorn.com/interactive-maps-with-vue-leaflet-5430527353c8

https://github.com/KoRiGaN/Vue2Leaflet/blob/master/examples/src/components/WMSLayers.vue

OpenStreetMap WMS servers: https://wiki.openstreetmap.org/wiki/WMS#OSM_WMS_Servers

Tim's Jupyter Notebook demo of GeoServer layers: https://nbviewer.jupyter.org/github/openskope/skope-notebooks/blob/master/Display_map_tiles.ipynb

Adding WMS legends to leaflet: https://gis.stackexchange.com/questions/182770/how-to-add-wms-legend-in-leaflet-from-geoserver

verify treatment of nodata in averaging

add dataset opacity slider

slider to control the opacity of the selected layer

set up skope favicon

geometry persistence is broken for points

cannot place a point marker due to the geometry persistence

set up wms layer animations

https://github.com/danwild/leaflet-wms-animator/blob/master/leaflet-wms-animator.js

support circles in persisted geojson

https://medium.com/geoman-blog/how-to-handle-circles-in-geojson-d04dcd6cb2e6

add timeseries service timeout

add a timeout and notify the user to "Select a point or smaller area" when the request to the timeseries service times out

improve timeout handling

• add a setInterval when issuing a query that informs the user that things are taking too long, allow them to reset / extend the timeout until it reaches a maximum
• clear timeout error messages when the query succeeds

paleocar water year legend has extra red X

Look into GeoServer GetLegendGraphic:

https://docs.geoserver.org/latest/en/user/services/wms/get_legend_graphic/index.html

frontend: replace default leaflet layers control with a custom variable selection control

Right now the leaflet control layer is convenient but has some warts w.r.t. programmatically keeping track of the selected variable and which layer is currently being displayed.

Things would be cleaner if we create a custom variable control that lets us select a variable - then we are in charge of proxying to leaflet to display the appropriate WMS layer or make the right time series service calls. Should look into this and see if it's worth the trouble as it should clean up our code so we don't have to dig into leaflet component internals and do string comparisons against our vue component data.

WMS tile layers not rendering

the WMS layers don't appear to rendering correctly after grouping them in another layers control

Search function broken

When searching, returns a message that says handler.apply is not a function.

landing page year range onblur is updating too soon

Steps to reproduce:

1. enter start year 530
2. tab to end year, enter end year 630

as soon as you enter "6" in the end year, it modifies the start year as well

looks like an issue with the input events

add google analytics to nuxt app

https://nuxtjs.org/faq/ga/

make variable overlays radios like the basemap layer

https://gis.stackexchange.com/questions/267605/leaflet-api-select-one-overlay-at-a-time-like-base-layers

link redirection on datasets

• error returns when title of dataset is clicked
• "Cannot read property 'find' of undefined"

citable figures w/ minimal provenance

Figure out a way to restore all user activities / parameterizations that generated a given set of data.

Two options to consider:

1. server side (short UUIDs mapped to parameterization)
2. client side only (download / upload metadata file that restores the stored data)

Going to implement option 2 for the time being as it eases the complexity of adding additional server side logic and persistence.

stored data:

• geometry
• zoom level
• selected basemap
• temporal range
• smoothing / transformation options
• selected variable

frontend: detail page fixes

• date boundaries have strange behavior (example: enter 1000-1400)
• add step forward / backwards to animation
• update the variables to not look so clickable
• add help button somewhere on the top
• add additional variable description / metadata

Revert variable styling

Use styling from f7386ae

migrate production apps from TACC to Compute Canada

We will need to migrate everything from TACC to Compute Canada in the next few months. Coordinate with @tmcphillips to provision VMs and storage for a staging and production server.

add universal input control for all dataset variables

In the new UI after selecting a dataset we should display a variable input control on all pages that allow the user to select the variable layer being displayed on the map or being rendered out with the timeseries graph

add atemporal boolean to datasets

atemporal boolean to signify that a dataset has a date range and can be filtered or should be unaffected by the date range filter