bcgov / wps Goto Github PK

Wildfire Predictive Services to support decision making in prevention, preparedness, response and recovery

License: Apache License 2.0

Shell 2.63% Dockerfile 0.41% Makefile 0.28% Python 56.69% Mako 0.04% Gherkin 2.25% HTML 0.39% TypeScript 37.10% JavaScript 0.14% CSS 0.09%

flnr wildfire bcws python fastapi react javascript typescript weather postgis

wps's Introduction

Wildfire Predictive Services

Description

Wildfire Predictive Services to support decision making in prevention, preparedness, response and recovery.

Getting Started

Dependencies

Installing

Running the application locally in docker:

Create .env file in web using web/.env.example as a sample.
Create .env.docker file in api/app using api/app/.env.example as a sample.
Run docker compose build and then docker compose up
Open http://localhost:8080 to view the front end served up from a static folder by the python api.
Open http://localhost:3000 to view the front end served up in developer mode by node.

Developing the application in a dev container, using vscode:

Open up the project: Remote-Containers: Open Folder in Container, select docker-compose.vscode.yml
Sometimes VSCode doesn't pick up you've changed the docker container: Remote-Containers: Rebuild Container
Install extensions into the container, as needed.
You can point the API database to: host.docker.internal
You can start up other services outside of vscode, e.g.: docker compose up db and docker compose up redis

Running the api alone

Refer to api/README.md.

Running the front end alone

Refer to web/README.md

Documentation

License

Apache 2.0 License - see the LICENSE.md

Contributing

PRs

Your Github PR is required to pass all our CI checks, including our test coverage threshold via CodeCov: https://docs.codecov.com/docs/about-code-coverage

Resources

Issues
ZenHub Board
PEP8 and PEP20 coding conventions, but with 110 character line breaks
Code of Conduct

Acknowledgments

wps's People

Contributors

Stargazers

Watchers

Forkers

abschwenker kyubinhan drroad code-ql-testing jmetev1 vanislekahuna

wps's Issues

Displaying Results

As a consultant I want to see calculated 90% weather values based on my given parameters for the following indices so that I can use them in technical reports to reflect potential fire severity and impacts to communities and support funding applications.

FFMC (Fine Fuel Moisture Code)
ISI (Initial Spread Index)
BUI (Build Up Index)

As a consultant I want an output showing calculated 90% Weather values for FFMC, ISI and BUI as well as documentation of input parameters and how the values were calculated in order to accurately document it in my application.

As a BCWS employee I want to be able to review the parameters that were used in calculating 90% weather in funding applications.
I want to have consistent values so that I am able to prioritize applications and make advice on the allocation of funding.

[STORY] Weather visualization

data we collect + global weather forecasts

can we leverage existing tools and apps such as Windy (SP?), IBM (weather company) and their work?
https://www.windy.com/
https://windy.app
https://www.ibm.com/weather

can we customize these outputs into our tools for map layers

Disclaimers

As the Provincial Governement (BCWS) we want to have a disclaimer on the front end of the calculator so that we are protected from misues and mis-interpretation of the information we are providing. This should provide context around what 90% Weather values are used for, how the values are calculated and what the end values actually represent.

This same disclaimer should be written into the code so that if and when it is re-created it will have the same context that we are providing.

The goal with 90% weather is to provide a consistent process to to come up with the same values regardless of where the calculations take place.

Weather Station Network Management

We need to build a system that allows us to manage the physical inventory and maintenance of all our weather stations. This would include station and sensor inventory and meta data, maintenance and work plan tracking. There are linkages to the IITD radio shop (these are the technicians that perform the maintenance and oversee inventory) as well as to the manufaturer (FTS). I have had discussions with FTS on this and they are considering developping this functionality into FTS 360, their SAAS product for weather data collection and management. IITD has also requested that we include radio inventory into the scope of this development. By doing this, we would increase the value added from BCWS to all NRS sector.

Request Limited Data Set for 90% Weather

Get a copy of all daily weather data.

Acceptance Criteria

Ticket is in IITD JIRA to provide data export to us in the form of a CSV
List of station IDS and Station names
Verified that the request includes all data elements required to create 90% Calculator

Additional context

There are approximately 3.1 million records.
The data resides in a MSSQL database.
The copy of the data will be used to pre-calculate FWI percentiles etc. until such time as the API becomes available.
Pre-calculating all the values, should be sufficient until the end of the current 2020 fire season, or until such time as data in the existing database is corrected.
Records are ordered by weather station and date, ascending.
All records are present

[QUESTION] How we prioritize

Describe the task
A clear and concise description of what the task is.

Expected behaviour
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your request.

Additional context
Add any other context about the task here.

[FEATURE] 90% Weather Calculator

90% Weather is one of the factors that is used in funding applications through the Community resilience Investment Program (CRIP).
https://intranet.gov.bc.ca/bcws/provincial-programs/prevention/crip

90% Weather Calculator will ensure a standard method for calculating this value to ensure consistent and accurate evaluation of proposals.

Community communication initiatives (?)
Simulating effects of wildfires near communities
Uses weather stations (290) to calculate indexes
Massaging numbers, looking for work in communities

Build a calculator:

Pull from weather database
Look at indexes
Calculate - consultant "90% weather statuses, calculate consistent numbers"
Open, transparent
Current weather db ~400 MB, 1954 onward
Analysis for: climate change, decades in varying timeframes
Narrow down to certain attributes
Consultants, current staff; on current website?
Towards allocating millions of dollars (~12.5M?) in funding

Indexes:

fine fuel moisture (will a fire start?)
initial spread index (is it going to start and spread?)
build up index (if starts, will it consume and produce heat energy)
more available

Data:

13:00 readings report, 17:00 calculated 90% daily, 24 daily indexes/blocks
relative humidity, dew point, elevation
analysis on core season (May 15 - Sept 15?, June 1 - Aug 31?)
start collecting when snow-free, end late October

=> Use this work as a proof of utility/viability!

[email protected]
[email protected]

BCWS Notes:

works at a different pace than we might be used to
days vs weeks

[STORY] API Disclaimer

As Provincial Government I want a Disclaimer at the API layer of the Application so that we are protected against any lawsuits based on the information presented by the application.

Acceptance Criteria:

When anyone looks at the API layer of the 90th percentile calculator application there is a disclaimer showing the provided content.

Assumptions:

Same text for the web disclaimer can be used for the API disclaimer.

[DEVOPS] Plan and Develop Pipeline Strategy

Develop plan to take to devs. Gather feedback and iterate. This will be used in initial deployment. So far this has been split into NOW and LATER goals, which are a little less tenable now.

UPDATE: BCDK has been deprecated and a replacement is on the way. Rollout blocked.

[STORY] Fire Weather Index visualization

Describe the task
A clear and concise description of what the task is.

Expected behaviour
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your request.

Additional context
Add any other context about the task here.

Basic Uptime Tracking

Describe the task
Basic uptime tracking for our first deliverable.

Expected behaviour
Probably a quick monitoring and rights setup of Uptime Robot or a similar service.

Acceptance Criteria

Sybrand and Derek are notified by uptime robot when the site is down
anyone on the team can check the uptime robot site to see uptime info

Select Pipeline Tools

Investigate/Choose:

[TASK] GitHub Issues and ZenHub Demo and Migration

Describe the task
Setup ZenHub, move Trello tasks and add new Issues as necessary.

Expected behavior
Set up a potential alternative to Trello, but with simpler GitHub integration, better reporting and in a form more palatable to outside collaboration.

Additional context
This is currently only an alternative, not yet approved to replace Trello.

[STORY] Mobile access to determine danger rating based on location for industry

BC Hydro and Fortis are looking at developing an app that can assist with work planning based on danger rating and Wildfire Regulations

[TASK] Prepare GitHub Repos

Describe the task
Prepare and meet with team, setup GitHub repo(s), licenses and basic README.md files.

Expected behavior
Clearly defined work submission areas and placeholder documentation.

Plan base repo(s)
Attach to a consolidated issue tracker
Provide extremely basic README.md files for each repo
Invite team and any collaborators

MoreCast MVP

*formally 10-14 day Fire Weather Index (FWI) forecasts
** formally Weather Forecast and Validation Tool

Dependencies

Epic

90th Percentile Calculator

We need to establish and or confirm the equation that is used to calculate 90% weather values.
Assumptions:

Use daily FWI values
Used pre-determined time range that represents core fire season
Use selected time range (years of historical data) for the MVP this will be present to 10 years in the past.

Dependencies

Epic

[TASK] Weather data API integration

Explore opportunity to get access to the Weather Phase 3 API so that we can get access to weather data to begin working on applications such as 90% Weather Calculator.

Setup meeting between Dev team and Selkirk to go over API integration into Predictive Services

WF1 Phase 3 API Access

Describe the task
Learn how to gain access to the API

Acceptance Criteria

Align with Selkirk on how to get access from our servers to their servers
Determine how authentication to their servers works
make tickets for each step required for authentication to required servers

Additional context

who is our main contact with Selkirk? Brady to confirm

Request Team Email Address

Request team email address for support and other general issues, like service accounts.

Blocked by missing iStore details and some devops planning and decision making.

[TASK] Provision GitHub Repository

Describe the task
Request GitHub Repo(s)

[TASK] Add PIA/STRA to GitHub Repo

Example
bcgov/agri-nmp#441 (comment)

TL;DR

Your repo is missing a compliance audit file so I've created this PR with a template that you can update with the correct PIA and STRA status (status options in the table below). If you'd like me to do this for you, skip to the commands section below.

Compliance

Projects in our organization (bcgov) need to complete a Privacy Impact Assessment (PIA) and Security Threat & Risk Assessment (STRA) before they go live in production. Since every ministry has their own way of doing both the STRA and PIA we don't enforce that projects do them, only that they report on the current status.

To help with reporting, I've added a compliance audit file as part of this pull request. Please checkout this branch and edit update status as needed. Here is a table of possible states:

Status Description
TBD If you're surprised by this news, use this state. I'll let you talk to your MISO and check back later.
in-progress Use this state when your assessment(s) are underway.
completed Use this state when your assessment(s) are completed.
exempt The PIA or STRA isn't applicable to your project / repo.
Here is what a completed audit file might look like:

name: compliance
description: |
  This document is used to track a projects PIA and STRA
  compliance.
spec:
  - name: PIA
    status: in-progress
    last-updated: '2019-11-22T00:03:52.138Z'
  - name: STRA
    status: completed
    last-updated: '2019-11-22T00:03:52.138Z'

For more information check out the BC Policy Framework for GitHub.

Pro Tip

If you're not sure what to do add a comment below with the command @repo-mountie help in it; a real-live-person will reply back to help you out.

Commands

I can update the status of the PIA and STRA for you; you'll just need to merge the PR when I'm done. You can find the available status values in the table above. Below are some commands I understand:

Command Description
@repo-mountie help You're freaking out and want to talk to a person.
@repo-mountie update-pia STATUS You want me to update the PIA status.
@repo-mountie update-stra STATUS You want me to update the STRA status.

Examples

@repo-mountie update-pia completed
@repo-mountie update-stra in-progress

Weather Station Dropdown

As a Consultant or BCWS Staff member I want to select 1 to 3 weather stations so that I can get data relevant to the location I am surveying/application I am reviewing

Acceptance Criteria:

when the user selects the weather stations they are displayed as the user uses different parts of the application
If the user tries to calculate without selecting weather stations they are informed they must select at least 1 weather station
user can see the station name and code in the dropdown.

Time Range Selection Tool

The calculator will need to select and apply a pre-determined date range filter based on the station that is selected. This date range needs to be assigned to each weather station based on its geographic location in the province and the weather zone that it resides in.

Date Ranges are: (These will need to be confirmed
North (Mackenzie and north): May 15-Aug 15-31?
Central (South of Mackenzie, North of 100 Mile House): May 1-Aug 31
South (South of 100 Mile House): May 1-Sept 15

Confirm geographic separators (Dana Hicks)
Confirm date ranges for each geographic area

[TASK] Migrate and build BCWS Predictive Services Rocket Chat

Build BCWS Rocket Chat space and Migrate Predictive Services Slack work space into Rocket Chat

[STORY] Fire behaviour Prediction Outputs visualization

Describe the task
A clear and concise description of what the task is.

Expected behaviour
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your request.

Additional context
Add any other context about the task here.

Historical weather referencing

Describe the task
A clear and concise description of what the task is.

Expected behaviour
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your request.

Additional context
Add any other context about the task here.

[STORY] Display Web Disclaimer

As Provincial Government I want a Disclaimer on the Web side of the Application so that we are protected against any lawsuits based on the information in the application and its use.

Acceptance Criteria:

When anyone opens or navigates to the 90th percentile calculator application they are presented with a disclaimer with some messaging about the precision of the calculator and its use
when someone closes the disclaimer by clicking on an accept button, they are presented with the 90th percentile calculator
Displaying a draft version of the disclaimer is acceptable in order to complete this story
Access to the calculator is blocked until the user clicks on the accept button.
Disclaimer is shown every time a user opens the page. (The intention of this criteria is to simplify the development by removing any concern to managing state).

Related

Content for disclaimer to be provided

[TASK] Create and Populate GitHub Team

Describe the task
Request and populate GitHub Team.

Expected behavior
One big happy team!

Fire Weather Climatology

Describe the task
A clear and concise description of what the task is.

Expected behaviour
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your request.

Additional context
Add any other context about the task here.

Weather Station Map Link

As a Consultant I want to be able to look at a map of where the weather stations are so that I can ensure I select the correct ones

Acceptance Criteria:

I can navigate to a map with the weather station which I can reference to select the correct ones from the weather station selection tool
I can clearly find where to click to navigate to the map
Weather station map is located at: https://governmentofbc.maps.arcgis.com/apps/webappviewer/index.html?id=c36baf74b74a46978cf517579a9ee332

UX Tests:
Given a consultant wants to locate weather stations When they click on the map icon/button Then they will be navigated to the weather station map website

[TASK] Pick Languages and Techologies

Describe the task
Team decision making for languages, technologies and other things important to this project's groundwork.

Expected behavior
We can start rolling!

Percentile Selection Tool

[ADMIN] PIA and STRA

Complete PIA and STRA for Predicitive Services project including Weather Phase 3
Narrow scope to just weather calculator

Invite Pawan to Alignment Session
Fill out PIA Template (Word Document)
Fill out STRA Intake (PDF)
Identify technical advisor for Predicitive Services (Agile Team)
Touch base with Kevin H, Tech adv for WxPh 3)
Review WxPh3 PIA from 2016

Front End / User Interface

As a consultant I need a user interface / portal where I can go to access the 90% Weather calculator

This needs to be linked off of the BCWS Public Web page https://www2.gov.bc.ca/gov/content/safety/wildfire-status/prevention/vegetation-and-fuel-management/fire-fuel-management/fuel-management
This needs to be linked off of the BCWS Intranet page. Decision will need to be made as to where this lives (Prevention, Predictive Services or both?)

For MVP we will be building this as open in order to simplify as we prove concept and work through privacy / security needs.

Decsion on security and permission requirements; Open, BCeID open, BCeID with permission

[TASK] Create templates

FWI Calculator

As a consultant I want to be able to select the representative weather stations around a community so that I can calculate 90% weather values using data that is representative of the location I am surveying.

We are thinking that this will typically be up to 3 weather stations.

As a consultant I want to be able to have access to a map where I can visualize the location, as well as access station site meta data (elevation, aspect, coordinates) so that I can select appropriate representative stations for the location I am surveying.

As a BCWS employee, I want to be able to select weather stations referenced in a report or application in order to calculate 90% weather so that I can validate the information in the report / application to help with funding application reviews.

[DEVOPS] Provision OpenShift Project

BCDevOps/devops-requests#137

HFI Calculator (Prep Tools)

Operations teams need to make daily decisions on resourcing levels due to forecast FWI (Fire Weather Indices) and FBP (Fire Behaviour Predictions). Develop a calculator for FWI and FBP outputs.

Additional Context:

Probability for success. ie resources required to put out a new fire - then match to appropriate value. Currently Ops officers are copy/pasting weather station forecasts into spreadsheet calculators to determine Prep levels and resourcing requirements
Ability to analyse appropriate prep levels for Zones. We have standard thresholds. Are they appropriate? Are we over or under resourcing?
Provincial phone call to deliver Fire Centre prep levels that are written by hand
Advanced Planner and Fire Centre Ops need to issue Fire Behaviour Advisories for elevated HFI based on forecasted dailies. Currently the process involves going to every fire station forecast and using REDapp and the Redbook to calculate values for 1300.

See Miro board for sessions with Andrea Rainey and Brad Litke: https://miro.com/app/board/o9J_ksTQpME=/
Prak's user work flow research: https://miro.com/app/board/o9J_ku7DTZQ=/

Starting point can be Head Fire Intensity Calculator - similar to the weather calculator > pull in weather data , perform some calculations and display an output value

Summarise daily prep levels by Fire Centre for the Province to review every Monday and Thursday.

Users:
Zone Fire Staff, RWCO, PWCO, Forecasters

User Groups:
Zone Wildfire Coordination Officer (ZWCO)

MVP notes:

determine base Fire Center for MVP
all users can add/edit curing rates
some RWCO and ZWCO will need to be able to edit Prep level (indicate that a prep level has been edited) and require rationale for the change
additional Fire Center needs can be added as features to the base.
summarize prep details into a report
Save Report as PDF
reports are reviewable from a PWCO without needing to go into each fire centres details (list of prep levels and rationales).
Matomo integrated to help measure success

Dependencies

Epic

[TASK] (Re?)Assess Meeting and Collaboration Tools

Meeting - Skype, GoToMeeting, Hangouts
Visualization - Miro
Planning - ?
Board - Trello, GitHub Issues, ZenHub
UI/UX -
Wiki/Sharing - Slack, Toilet.Chat, GitBook, MiscWiki, SharePain
Privacy - Privacy, Private Data
Team Space - Slack, Wiki, GitHub
Security/Quality - SonarQube, Automo, BDDStack

Fire behaviour and weather advisories

Describe the task
A clear and concise description of what the task is.

Expected behaviour
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your request.

Additional context
Add any other context about the task here.

[TASK] Capture dependencies for weather database

Describe the task
A clear and concise description of what the task is.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your request.

Additional context
Add any other context about the task here.

Anomaly charts and calculations

Describe the task
A clear and concise description of what the task is.

Expected behaviour
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your request.

Additional context
Add any other context about the task here.

License

@DerekRoberts - I think we're supposed to be using Apache License 2.0 - not GPL.

[TASK] Data sharing requests

How to automate and improve the process for responding to and managing data requests. This will need to be evaluated with in Wildfire 1 team to determine who will handle this?

Integration of the Canadian Forest Service (CFS) Fire Occurrence Prediction system

Canadian Forest Services

how can the take the work that they have done and use it in our toolset?

the way that they have built it there is a spacial output, color scale based on number of fires started in an area

have a better linkage/working relationship with the research institutes, how can we be faster at consuming their products to be used in operations, and provide feedback to these institutes.

Bare Bones Dev Pipeline

Describe the task
Build a basic pipeline, documenting steps and collaborating with team to ensure knowledge-sharing.

Acceptance Criteria

Basic pipeline capable of deploying a Hello World-level app.
Needs to hit any Dev environment
Needs to be transparent and usable by WPS dev team
Templates created

Time Range Displayed on Calculator

As a Consultant I want to see the time range the calculator is using (present - 10 years) so that I can accurately document it

Acceptance Criteria:

the Time Range being used in that calculation is present on screen prior to the calculation being completed
the Time Range being used is displayed alongside the results of the calculation

UX tests:
Given a consultant wants to calculate 90th percentile When they want to see the time range Then they view the input field

[TASK] Predictive Services Intranet page

The Predictive Services Intranet page is the space that will be used to communicate predictive services products to BCWS staff and other clients. It is also the point where we can link to all of the products and services that we will be building. This is our gateway linking BCWS to the virtual world we will be creating.

We need to build this page in a way that we have 100% control over updating the content so that we can ensure accurate intelligence and functionality without being dependent on a third party to do the work for us.

Live Map

Describe the task
A clear and concise description of what the task is.

Expected behaviour
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your request.

Additional context
Add any other context about the task here.

bcgov / wps Goto Github PK

wps's Introduction

Wildfire Predictive Services

Description

Getting Started

Dependencies

Installing

Running the application locally in docker:

Developing the application in a dev container, using vscode:

Running the api alone

Running the front end alone

Documentation

License

Contributing

PRs

Resources

Acknowledgments

wps's People

Contributors

Stargazers

Watchers

Forkers

wps's Issues

Dependencies

Epic

Dependencies

Epic

TL;DR

Compliance

Pro Tip

Commands

Examples

Dependencies

Epic

Recommend Projects

Recommend Topics

Recommend Org