wswcwaterdataexchange / westdaat

Managing the code of the Western States Water Data Access and Analysis Tool (WestDAAT)

WaDE

Getting Started

1. Clone this repo.
2. Make sure you have the .NET 6 SDK installed.
3. Start SQL Server (Docker or Host machine)
   • If you already have a SQL Server container running, just use that as the Host machine no need for docker.
   • To spin up a new SQL server docker container:
     • docker-compose file supplied in src/API/BackendServices/WesternStatesWater.WaDE.Docker/docker-compose.dev.yml
     • docker-compose up -d
4. Run DbUp project from WaDE2 repo
   • Note: No need to run DbUp if you imported bacpac files for the DBs
   • Need two databases WaDE2 and WaDE2Test
     • WaDE2Test is for the unit tests
   • If not using the default SQL server, update the connection string in launchSettings.json in the DbUp project
   • May need to run this twice if using docker
5. Setup Function App
   • Open Solution: path to file WestDAAT\src\API\WesternStatesWater.WestDaat.sln
   • Right-click on Client.Functions project and choose Set as Startup Project
   • create local.settings.json file in the Client.Functions project
   • If necessary, create personal.settings.json to override any developer specific settings. As an example( e.g. Insert: {“Database”:{“ConnectionString”: “Server = ‘Insert server name here’ ; Initial Catalog = ‘Insert local db name here’ ; Integrated Security = true;”}}) However this may vary.
   • Set the listening port to 5001
     • Right-click on Client.Functions project
     • Choose Properties
     • Select Debug -> General -> Open debug launch profiles UI
     • In the command line arguments enter the following
       • host start --port 5001
     • Run the Function App
6. Setup React App
   • install NPM packages.
     • From the DashboardUI directory: npm install
   • Requires Node version 16+
   • Requires NPM version 8+
     • For Windows download the Node installer here: https://nodejs.org/en/download/
       • this installer includes NPM update
   • Start React App
     • From the DashboardUI directory: npm run start

CORS errors

• CORS configuration is set in the local.settings.json file of the function app

Update Mapbox Vector Tileset

Only do this if you need to update the vector tiles in maptiler

1. Run MapboxTilesetCreate console app located in /tools
2. Optional - Add personal.settings.json for connection string settings
3. Allocations.Points.geojson and Allocations.Polygons.geojson files will be generated in geojson dir
4. Run tippecanoe -zg -o waterRights.mbtiles --read-parallel --drop-densest-as-needed --extend-zooms-if-still-dropping --generate-ids --force -L points:"Allocations.Points.geojson" -L polygons:"Allocations.Polygons.geojson" This will take about 5 minutes
5. Upload to mapbox studio

Deployment

• Setting secrets for deployment: https://github.com/Azure/actions-workflow-samples/blob/master/assets/create-secrets-for-GitHub-workflows.md
• To set secret with Azure Credientials, there should already be app registrations for the github actions, so just need to renew secret and put the new secret in the json (need at least Application Administrator role)

westdaat's Issues

Enforce user accounts for all types of "download" functions

For consistency in user experience and to allow us to report use metrics to our funders (one of the deliverables), let's enforce the need to have an account and user login before downloading anything (data, maps, statistics etc.) in our dashboard. So we allow open access with no barriers to view the data and query it. The download action indicate an advanced level of engagement with out data that deserves reporting. Hopefully this strikes a balance between being restrictive or completely open. Our dashboard analytics still will report user hits etc. to the general visitors.

Action 1: To all "download" buttons in our mockup, let's show the "login" window.
Action 2: Update the signup dialog box to include supporting "Google" type of sign in similar to OpenET. We should discuss other the type of info we're asking for in the sign up such as State, Organization, and Country. Perhaps we should make these optional because the sign up through Google bypassed them. We need to have names and emails at minimum which we could follow up with and potentially terminate if they miss use the app.

OpenET dashboard (login to everyone)
https://explore.etdata.org/

USGS Dashboard (open to everyone)
https://dashboard.waterdata.usgs.gov/

Move River Basin Filter up + assign primary beneficial use

Doubled check Idaho water rights count

It caught my eye that Idaho shares this summary of points of diversions (different than the count of water rights). In comparison, WestDAAT QA shows ~336,000 rights. Not sure what's going on, but worth investigating.

https://data-idwr.hub.arcgis.com/

Update water rights variable for AZ data

Should the variable here be: Allocation? It seems that's the case for all of AZ's data

https://westdaatqa.azureedge.net/details/right/1626514

BTY, this water right is very interesting!

OpenET integration mockup

1. After the OpenET/WaDE "heat map" map is generated, how could the user toggle between different "reporting area types", models, or variables? I assume they'd change the input parameters and click again at the "generate heat map" button. We need to make sure that WestDAAT "navigation bar" on the left stays the same and does not scroll up after the first heat map is generated. OpenET dashboard has those parameters at the top (see below). Perhaps we should offer some basic parameters at the top like that such as switching units. Let's think about that.

2. Would you prepare example output data/map that will be generated from the "download map" button? I think the download should include data for all OpenET models (6+ ensemble) and data : ET, NetET, ReferenceET, Perception for the selected OpenET model + the permitted water rights for each "report area type". There is a good chance users want to post-process these data and it would be tedious if we ask them to click download 7 times if the download is for one model at-a-time. Users still need to click download again for another reporting area type.

Co-creation session: support regulatory overlays in WestDAAT

Here is an idea of how we may want to support filtering regulatory overlays that generalizes the existing river basin filter to so many other basins and boundaries but not through a drop-down list.

Motivational use case: show me all the local (within states) and regional (interstate) regulatory overlays (polygons). Then I want to click on a polygon and get a summary card that has a hyperlink to the regulatory overlay landing page. The user also wants a button on that card that filters all water rights (PODs or POUs) to that polygon, i.e., only showing water rights within this selected polygon (E.g., Phenix Active Management Area) . Can a user select multiple polygons? I'm not sure how multi-select may work and if we want to limit it to one polygon at a time.

Potential filters/questions

1. Toggle button (probably next to POD, POUs) to display regulatory overlay polygons. Something to discuss is how much this affects the app load time. Example: https://public.tableau.com/app/profile/wswc/viz/SWvsGWRegulatoryAreaDashboard/SWvsGWRegulatoryAreaDashboard
2. Toggle button between "water source type" that applies to the regulatory overlay. Regulatory overlays may be surface or groundwater (perhaps both) in nature. For example, Texas, California, and Nebraska have regulatory overlays that regulate groundwater. There could be surface water rights within those areas.
3. Toggle(?) button that filters those regulatory overlays when multiple are on top of each other (big river basin with multiple local smaller polygons under it) based on key metadata: RegulatoryOverlayTypeCV (e.g., state, interstate, federal).
4. How do we color-code those polygons? based on groundwater or surface water regulation type? or RegulatoryOverlayTypeCV? either one based on the user selection? Would we have a second legend for them in addition to the existing legend we have for water rights sites?

Design related points

• We probably want to add a geometry field to the RegulatoryOverlay_dim table.
  https://schema.westernstateswater.org/diagrams/4_RegulatoryInfo.html
• We will pre-process and build the connections between the sites table and the RegulatoryOverlay_dim table so the whole process would be generalized as far as adding new regulatory overlays over time. This bridge table takes care of this connection: https://schema.westernstateswater.org/tables/Core_RegulatoryOverlayBridge_Sites_fact.html

Issue with Prod pie chart colors

See these two links and screenshots below. The issue appears when Idaho or Montana is selected (one at a time). Not sure about other states. We need to test further.

https://westdaat.westernstateswater.org/?state=N4Ig7gTiBcoM4BcCGCCmcYG0QEkAiIAugDQgAOA9gDYCeVAlgHapYkiNUAm9AYvVWgh4USGKCop6CAK6cW0RtKpVSVCowDmU2fMXLS3CKgDGCeuozQAzAZEAFCkwSWA7AF83pALZIyYkABeFBReADKoAG6oVDAArAB0AAwATABsAIzJLi4AHOnpqQCcACzFsaqSMnIwxcnxycmJhYmJOakuibHpscWq6lpV8gC0%2BYnxVi5FmcXNrbE5HkA

https://westdaat.westernstateswater.org/?state=N4Ig7gTiBcoM4BcCGCCmcYG0QEkAiIAugDQgAOA9gDYCeVAlgHapYkiNUAm9AYvVWgh4USGKCop6CAK6cW0RtKpVSVCowDmU2fMXLS3CKgDGCeuozQAzAZEAFCkwSWA7AF83pALZIyYkABeFBReADKoAG6oVDAArAB0AAwATABsAIzJLi4AHOnpqQCcACzFsaqSMnIwxcnxycmJhYmJOakuibHpscWq6lpV8gC0%2BYnxVi5FmcXNrbE5HkA

Aggregated Data Shapefile Specs

As discussed in our weekly touchpoint this morning, we have some questions regarding the processed shapefiles located in the WaDE2.0 repo:

1. The shapefiles don't include a .prj file. I've used a shapefile from a different directory of that repository - Can we get the projections files that correspond to these shapefiles? If not, will the following be sufficient? We're currently using a .prj file that contains this: GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]

2. The shapefiles' headers aren't set because of some technical limitations with their generation. This is fine, of course, but we'd like to know what the properties correspond to. For context, these are the properties that we have available at the moment:
   

3. Given these parameters and the schema for the aggregated data, which shapefile properties are necessary to query the database for a given entry?

Thanks!

Priority date filter is not fulling working

The priority date filter does not fully work. See this example as it shows water rights that should be filtered out
https://westdaatqa.westernstateswater.org/?state=N4IgtgziBcIQLgJwKbPhAtANwIw5ADQgB2ANgCYCWMopAhvJfAK7nIwAsADAHQ4CcAJgBsRUgHtiAcyat20DDi78e-HAHZ%2BRKigDGjSVGgBmbQzoAFcZWLoY6gL5EA7ohrg6ABwDiicc08bKRgQACNmQhAbXVI5AFEAD2QwT3gYADM6UghkIjA6BItESnFi%2BABPABEGeUUAVnV1AA5%2BJq52hwcgA

Selected site + popup card enhancement

1. Allow users to move the summary card away from the selected point
2. When hovering over a site, change the mouse cursor from palm to pointing finger (just like the prototype)
3. Highlight the selected point or polygon to stand out.

Explore using Netlify to deploy WestDAAT

https://www.netlify.com/

Bug: display multiple beneficial uses on the landing page

This water right has many beneficial uses and we need to accommodate listing them on the landing page in a dynamic way

https://westdaatqa.azureedge.net/details/right/2197163

JSON-LD file

Update "gsp": "http://www.opengeospatial.org/standards/geosparql/", to "gsp": "http://www.opengis.net/ont/geosparql#"

https://westdaatqa.westernstateswater.org/jsonld.json

Water rights filter location

1. Let's move the Allocation Owner filter to be next to the owner classification. It's more logical to have them next to each other. I think users care less about the type of filter (general search or drop down menu) but probably rather expect relevant filters to be next to each other. For example, I want to tell the story how WestDAAT supports searching by owner name or their classification. So it helps if both of these filters are next to each other.

2. Let's use the term "Water right" in WestDAAT not allocation. So let's make it "Water right owner name"

Map content on the water right landing page

So let's revisit this landing page behavior when the user clicks at the tabs here. The map to the right side does change/update between "site" or "water right" viewed row. Should it also update for "regulatory info"? for example show the regulatory polygon? same question applies to the "Water Source info". What if there are many water sources or many regulatory rows here and the user wants to click at each one at a time? perhaps the mockup should include more rows for regulatory and water sources info as an example of the possibility.

Question, when the user clicks "download" data, would they get data and metadata for all the "rows" in these tabs? probably yes. @rwjam would you please prepare a sample data structure of this "exported data for DPL?

Double check priory dates in California's and Oregon's water rights data

I noticed that prior to "1900" priority date, California and Oregon mostly don't show water rights which is either not right, or very interesting. The same applies to North Dakota and the lower states down to Oklahoma. I know TX and NM data is incomplete in that regard. So it is worth checking and see if that is correct as we will be showing this at our Spring 2021 meeting.

WestDAAT Site-Centric Landing Page Update & Feedback (05/31/2022) - Create Site Card, update site card, Water Right Summary Table

For the WestDAAT Site-Centric Landing Page, here are some elements we would like to see included / updated.

1. Let's move the info in the sites table up to a card (see sample image).
2. Update the sites card to also include "Coordinate Method" & "Coordinate Accuracy" fields from the Core.Sites_dim table (see sample image)
3. For the Water Rights Information Table, let's only provide an abridged summary of the water right record(s), and save the full water right info for the Water Right Record Centric Landing Page.

The Site-Centric Landing Page helps us illustrate a 1-m relationship between a site and the many water right record(s) it might have. We would like to update the site centric landing page to help empathies this 1-m relationship. We like the idea of moving the info in that site table, for the given single site, up to a card with an up-&-down scroll ability.

In addition, lets put a little more site info on that card. We would like also like to update the sites card to include the "Coordinate Method" & "Coordinate Accuracy" fields from the Core.Sites_dim table.

For the water rights information table on this landing page, we are thinking of only showing a abridged version of the full water rights information (so not everything from the Core.AllocationAmounts_fact table, just a summary). We want to keep it simiple for now here and only show the WaDE terminology. Here is the list of items we would like to see on that water rights information table on this site centric landing page, in order (with more important features appearing at the top).

• WaDEUUID for water right (coming soon) (maybe for now cheat and use the AllocationAmountID as a temp place holder)
• Priority Date
• WaDE Owner Classification
• WaDE Legal Status
• Allocation Flow (CFS)
• Allocation Volume (AF)
• WaDE Beneficial Use

Check on a POU/POD site locations in MT

Ryan,
Would you check on why these POU and POD sites are far off each other? Is that a data issue from the source? To see the POU site, zoom in to the town called Choteau. I found it by following the site coordinates and Mapbox long. and lat. values as you mouse over the map.
https://westdaatqa.azureedge.net/details/right/721008

Priority Date Filter Components.

(Ryan McManaman) I've got some questions regarding the priority date filter component. As discussed, the filter needs to be a slider that lets a user select a start/end date for the rendered allocations. Looking at the schema, the allocations have one priority date, but a priority date can have multiple allocations. An allocation can also have a null priority date value. With that in mind:

Q: What happens when an allocation has no priority date and a user selects an end/start date? Do we render all null values or none of them?

Q: How is expiration date factored into the filter? For example, if a priority date occurs before the filter's start date but hasn't expired, does it get rendered (e.g. Priority Date is Oct 1, Start Date is Oct 31st in the filter, Expiration Date is Dec 1)?

Q: Is there a scenario where the priority date falls within the selected range but shouldn't be rendered?

Support nested drop down River Basin values and RegulatoryOverlay discussion

1. For the River Basin Area filter, we need to support nesting river basins (maybe through a plus sign next to high level basins?) as in the case of the Colorado River Basin > Upper Basin, Lower Basin.
2. For the Regulatory Area Search filter, we might end up with 100s of these values which makes it too hard for a drop down menu. Should the drop down menu itself include a search filter: example: the user types: North Platte and the drop down menu filters out anything else that does not match. What if the user want all the regulatory boundries for AZ or UT ?
3. Related to Point#2 where would users narrow down the search for WaterSourceTypeCV values (e.g., show regulator areas only for surface or groundwater). Would we use the same WaterSourceTypeCV above for water rights? what would be the limitations? Where would they filter through RegulatoryOverlayTypeCV (e..g, federal)
4. What if a user want to see all the regulatory overlays in the West, then download their shapefile and metadata? how do we support that? Under a new tab?

WestDAAT Water Right Record Centric Landing Page Update & Feedback (05/31/2022) - WR Card, WaDe & Native term, update site table

For the WestDAAT Water Right Record Centric Landing Page, here are some elements we would like to see included / updated.

1. Single water right info Card that shows info from the Core.AllocationAmounts_fact table (see sample image)
2. Displaying both 1) WaDE Terminology and 2) state provided Native Terminology on the same line (see below list for examples).
3. Update the sites table to also include "Coordinate Method" & "Coordinate Accuracy" fields from the Core.Sites_dim table (see sample image)

The Water Right Record Centric Landing Page helps us illustrate a 1-m relationship between a water right record and its related site(s). We like the design of this demo landing page, and would like to see something like this developed for WestDAAT. We would like to see all informative water right record information from the Core.AllocationAmounts_fact table displayed on a card with an up-&-down scroll ability.

We need a way to show both the state provided native terms (i.e., the stuff given to us) & the WaDE terminology (i.e., the stuff we use on the filters) on the returned data. Below (see table / list) is an idea we had. We would like to try and show both the WaDE Terminology & the state provided Native Terminology on the same line, separated out by a '|' symbol. For example, say the WaDE Terminology = Agriculture Irrigation, and the state provided Native Terminology = Cranberry, the resulting item in the card would look like...

WaDE Beneficial Use	Native Beneficial Use
Agriculture Irrigation	Cranberry

Lastly, we would like to update the sites table at the bottom on the to also include the "Coordinate Method" & "Coordinate Accuracy" fields from the Core.Sites_dim table.

Here is the list of items we would like to see on that water right record information card, in order (with more important features appearing at the top).

• WaDEUUID for water right (coming soon) (maybe for now cheat and use the AllocationAmountID as a temp place holder)
• Native ID
• Owner Classification
• Owner
• Priority Date
• Expiration Date
• WaDE Legal Status | Native Legal Status
• Allocation Flow (CFS)
• Allocation Volume (AF)
• WaDE Beneficial Use | Native Beneficial Use
• Application Date
• Data Publication Date
• Data Publication DOI
• Exempt
• WaDE Power Type | Native Power Type
• Generated Power Capacity (MW)
• SDWIS Identifier
• Timeframe Start
• Timeframe End
• Legacy Allocation ID(s)
• Community Water Supply System Name
• WaDE Customer Type | Native Customer Type
• Population Served
• WaDE Crop Type | Native Crop Type
• WaDE Irrigation Method | Native Irrigation Method
• Irrigated Acreage (AF)
• Crop Duty Amount (AF)
• WaDE Primary Use Category
• Water Allocation Native URL

Check if we have more Hydro and Thermo electric + Snow beneficial uses in QA db?

I was expecting to see more Hydro and Thermo electric + snow water rights than shows up here

Remove old/unused beneficial uses from the drop down menu

Download buttons and sample data for data table and pie charts section

Let's add download buttons and prepare sample data for each of "data table" and "pie charts" exports here

Aggregate Water Data

Questions:

1. Should we let users choose the legend color? e.g., blue, green etc? or its always blue? obviously its easier for us to make it static.

2. Would you prepare example output data + metadata for the download button? I think we should export all the available years for a selected Reporting Area Type and Variable Type. Should we also download data to all beneficial uses and water sources? Perhaps in the download button, we should give the user a dialog box with boxes to check on what they want from the download. I think we should do the same to the other download buttons. That would be more user-friendly than either giving them everything or just a single piece of data (i.e., for one year).

3. Next to variable type, please add a question mark (?) that would refer the user to what is meant by these variables and how the states data map to them. These variables are so critical in this work. Perhaps, it should give a hyperlink to a page of WaDE's website for that info.

4. Let's delete the "advanced tools" from the navigation bar. Also let's move the "play growth" button right next to "select year". Just the green button with no labels next to it. It should b intuitive enough what a play button does next to a slider bar.

5. How should we support the mix and match of reporting unit types on the map? i.e., to maximize the value of data when no perfect spatial match exists for all the states? should this be a new "Reporting Area Type" value called "Mixed"? then we take care of showing the best match under this option. Or should we give users an option to check multiple Reporting Area Type (check box not radio button) and then we show the best match and figure out which layer to show when multiple exist?

6. Let's rename "beneficial use" to Category of Use". Let's also add a question mark (?) next to it which probably should refer the user to a page on WaDE website for more info.

Use distinct color theme for owner classifications

In this case, both "In-review" and Reclamation (USBOR) appear to have the same blue color

Display one label per POU

Why do we have multiple identical labels showing up for the POU polygon here? I was confused thinking there are multiple ones (hiding below) until I realized they are the same label

link to recreate: https://westdaat.westernstateswater.org/details/site/WAwr_S160080?state=N4IgtgziBcIQhgFwKYBtUEsUFoBuBOEAGnHgAcZQAvAexrABllc0YAOAOgGYB2ABgBMbLlwAsAkaP4CSqJFgCuAE2QwpHIfh7CuAgKwBGbTx6jZNAHYBzRSpjYDAvh3wDtY-FwBsX13vwAvgFAA

Animation Growth button and Priority date slider

This button would come as a surprise to many users, e.g., growth of what? It is floating on its own. Instead, we should move it right below the priority date slider or even better, next to the Allocation Priority Date label with no text. So just the green play button. If the user hovers over it, then it shows the text e.g., "animate water rights growth over time"

The priority date slider should be the first one those three bard on top. Its the most attractive one. Let's color it in green to it stands out.

Summary of range of flow/volume of water rights per state + Start and End times of water rights

I'm posting this chat here so we can keep track of it. Not sure if this repo is the right one to post this issue at.

Add a message to the download dialog box

I liked how this data service tells the user how their information is going to be used
https://connectivity.sarpdata.com/summary

We use this information to:

• get in touch with you if we discover errors in the data

• provide statistics about how this tool is being used to our funders

• better understand how this tool is being used so that we can prioritize improvements

A few features or bugs to look into

1. Add back the water rights metadata card with district labeling to both state native and WaDE terms. We need these items along with the previous ones
   WaterAllcationNativeURL
   DataPublicationDate
   PrimaryUseCategoryCV
   ExemptOfVolumeFlowPriority
   CropTypeCV
   IrrigatedAcreage
   AllocationCropDutyAmount

CustomerTypeCV
CommunityWaterSupplySystem
OwnerClassificationCV
IrrigationMethodCV
PopulationServed

GeneratedPowerCapacityMW
PowerType

AllocationTimeframeEnd
AllocationTimeframeStart

https://schema.westernstateswater.org/tables/Core_AllocationAmounts_fact.html

2. Update the Volume and Flow filter to the OR case: see here #101

3. Move the Toggle view to be right below the Toggle Point Size
   

WaterAllcationNativeURLDataPublicationDateID

Surface and groundwater regulatory and management agencies in the Western US

We need an interactive Tableau dashboard that visualizes all the Surface and groundwater regulatory and management agencies in the Western US and shows the differences in statues in SW and GW.

Here is a good starting point.
https://scholar.law.colorado.edu/cgi/viewcontent.cgi?article=1004&context=groundwater-in-west

The dashboard should have a map of the western 18 states. When the user clicks at a state, it should list the agencies that regulate and manage water and their URLs. It should also list how GW and SW are governed (e.g., connected or separate).

The purpose of the dashboard is to show the diversity in governance methods to help us identify the role of water data collection in sharing across the west.

Text to be used in the feedback/suggestions

This text was used in WaDE 1.0 and let's reuse some of it.

Welcome to the WaDE Requests Forum! This forum is for users and data providers to make suggestions for changes to the WaDE framework. Topics can range widely between database enhancements, performance suggestions, interface suggestions, including new datatypes, or new ways to search for data. Please feel free to make suggestions. Not every suggestion will be adopted, but all will be reviewed by the WIDS committee, and if accepted, they’ll be added to the WSWC’s WaDE Github Repository in their relative priority.

FORUM RULES:
Please follow the forum guidelines to help everyone get the most out of the forum:

-Before you post your question, please check if anyone has asked it already and received a reply.
-Stay on topic.
-Please be respectful and courteous to other forum members.
-If you need to refer to another post, please quote the post so other members know what you are referring to.

Posts that violate these rules may be deleted.

WaDE Terms of service

WaDE 1.0 Disclaimer
https://github.com/wadewswc/wade-postgresql

Licenses to choose from
https://choosealicense.com/

OpenET Terms of service
https://openetdata.org/terms-of-service/

---

From the OpenET team
API User Agreement
Thank you for your interest and support in OpenET. Our intent is to collaborate and improve the quality and accuracy of OpenET data.

In exchange for receiving early, unpaid access to the API, you agree to the following:

1. Given OpenET’s staff limitations to support new member onboarding, prior to seeking guidance from OpenET team members, I will:
   a. Review OpenET API Documentation:
   https://open-et.github.io/docs/build/html/index.html
   b. Watch the Training Webinar:
   https://www.youtube.com/watch?v=F8CbtKmLcDI&feature=youtu.be
   c. Review background content on methodologies, accuracy, FAQs and use cases
   available on www.openetdata.org
   d. Coordinate with any members of my organization that are using the API

2. I will provide feedback on API bugs.

3. I will not share OpenET API Access or Resources to individuals outside of the OpenET API Working Group.

4. If incorporating data from OpenET into a comparison to other data sources, water budget, or other analysis, I will share and, if requested by OpenET, discuss analysis with the OpenET team before presenting it to other parties.

5. I will cite OpenET as the source of data in any published study or analysis.

6. When data from OpenET represents a large component of a study, analysis, or academic publication, I will offer or discuss co-authorship with members of the OpenET team.

7. If using API access to conduct an analysis of the accuracy of OpenET data, I will share findings with the OpenET team.

Do you accept the terms of the agreement? *
I accept
I do not accept
Name *
Organization *
WESTERN STATES WATER COUNCIL
Email *

water right landing page: variable info + controlled vocabulary

1. For the variable metadata, let's not show this item because it can be confusing (the unit is already included in the field name):
   Maximum Amount Unit: CFS

2. We also need to show and distinguish the WaDE controlled terms next to the native terms. e.g., AllocationLegalStatusCV with a hyperlink that takes the user to the CVs webpage that contains all the CVs of that term. Note, the meeting we decided to have one hyperlink next to beneficial uses filter to this webpage http://vocabulary.westernstateswater.org/beneficialusecategory/
   We probably want to the the same to all other controlled vocabulary (CV) filters

3. We want to add most of the metadata items listed here such as CropTypeCV,CustomerTypeCV, CommunityWaterSupplySystem, AllocationCropDutyAmount, WaterAllcationNativeURL, ExemptOfVolumeFlowPriority, DataPublicationDate. See here
   https://schema.westernstateswater.org/tables/Core_AllocationAmounts_fact.html

https://westdaatqa.azureedge.net/details/right/2802286

How to handle different water rights amount search cases?

We decided to design the flow and volume range filters to be simplisitic. In reality, there are many complex cases that we need the filters to handle as summarized in this table. We need to see how to support handling the AND OR cases

Use case: show me water rights that have a flow of more than 5 CFS or a volume of more than 7,000 AF. I think the current implementation handles the AND case, not the OR. What if the user wants to limit the search to the AND case?

Case	Flow	Volume
case 1	value	value
case 2	value	null
case 3	null	value
case 4	null	null
case 5	0	0
case 6	value	0
case 7	0	value

Notes

1. Case 4 is for exempt water rights
2. Zero is a value but there is a chance the state added zero instead of null. We need to revisit the mapping of our data and probably reach out to the states to revise how we handle this null/zero issue.
3. Are there other cases out there?

SQL Query
This basic SQL query is a starting point to see the distribution of CFS and AF types of values across states

SELECT [AllocationFlow_CFS],
      [AllocationVolume_AF]
      
  FROM [Core].[AllocationAmounts_fact]

NLDI related use case: priority date order

For Frank the Farmer, Tony once mentioned this point and I love it. I'm not sure how hard it would be to code. Suppose the NLDI API returns 200 WaDE POD sites. Each site has its priority date from its parent water right. Our WestDAAT code sorts the sites based on priority date from the oldest to the newest. Then Assign he oldest one a value of 1 and the newest one a value of 200. If two priority dates are equal to the same year and month, then assign them the same order e.g., 5. WestDAAT should show the numeric value as a label. The "label" should be included as part of an export function (i.e., include it in the downloaded data)

The order from 1-99 is a normalized index across potentially many places. For now, it would be easier to have the highest order depending on being whatever the highest number of sites we have.

Update the points count to be queried from the database not from the tiles

We need to update the count to be the real total number, not just the points on the top tile.
Should we have two counts, one for POU and another for POD sites? Example

• 38353 Points of Diversions Displayed
• 22217 Points of Places of Use Displayed

WaDE and OpenET integration on our dashboard

Suggestions:
High level bold title: Interoperability with OpenET Data. Let's add a (?) info here where it shows a box of text on what OpenET is.
Second level title: Compare aggregate permitted water vs estimated consumptive water use

Let's update the labels to these:

Label: Comparison year

Label: Boundary type

Label: OpenET model

Label: OpenET variable

Aggregate Water Data landing page

In WaDE 1.0 catalog front end dashboard, we used to show a pie chart for the water budget across categories of use of a selected "area" if the state has many variables reported. I think we also showed a pie chart of water use across beneficial uses. This is a very useful insight.

1. Let's add a pie chart plot (for the many categories of use of the same year) on the landing page probably below the time series plot (USGS landing pages allow multiple plots on their landing page). We probably could make the plot "variable" to be "category of use", "water source type", and "VariableCV"
   https://waterdata.usgs.gov/monitoring-location/10142000/#parameterCode=00065&period=P7D
   

2. Let's add a polygon map similar to the water right POD landing page. This is a feature of the Geoconnex landing pages which we have a deliverable for https://geoconnex.internetofwater.dev/demo/

Tons of unknown beneficial uses

@rwjam, when you have some time, would you look into what does this "unknown". Colorado and Arizona seem to have a tone of them. It could be related to the "null" issue or to how Tseganeh (if any) map things back then.

In the CVs, we have both "Unknown" and "Unspecified" values. I think a water right in general by definition must have a beneficial use. So if the field is missing in any state's data, we probably should drop and delete the "Unknown" and keep "Unspecified".

http://vocabulary.westernstateswater.org/beneficialusecategory/

https://wade-mapbox-prototype.azureedge.net/

WestDAAT Errors 03/31/2022

Below are a few errors we caught when doing a demo of WestDAAT on March 31, 2022.

1. Seems like the Rest Filter Button is working as intended for the map, but not the filters themselves. Ideally when pressed the Rest Filter Button would also reset all the filters back to a default / launch app state.
   

2. There is a transparency issue with POU polygons around the Las Vegas area. We suspect the error is caused due to too many polygons layered on on top of one another at this location. We will verify if this is this case and determine how we can solve this issue on our end. We are also open to suggestions if we cannot solve this issue ourselves / if this is the true state of the data and cannot be changed.
   

3. There is a discrepancy between the Priority Date posted on the popup card and the Priority Date Filter inputs for the UT data.
   

4. When displaying sites by Water Source Type some sites in OR appeared as block dots. We suspect this was due to updates we had pushed to the QA, but the vector tiles themselves had not been updated to reflect those QA changes. As a whole, we suspect this will not be an issue in the long run as this should be resolved anytime the vector tiles are updated. We will either need assistance updating those vector tiles every time we push changes to the QA, or instruction on how we can update the vector tiles ourselves.
   

Change labels of NLDI to more friendly title

So the vast majority of users wouldn't know what NLDI is. Instead, let's let's brand it in a more user-friendly way to our users that tie back to the persons and use cases.

Suggestions:
High level bold title: Interoperability with USGS, EPA, and Internet of Water
Second level title: Query upstream or downstream sites and data (the info under (?) should refer to the NLDI, USGS, and IOW stuff)

The "drag and drop pin on map" needs to be more specific like:
Drag and drop the search location pin on map

The Latitude and Longitude values are auto-populated from the "pin" on the map. So should these be moved below right before the "click me" button. Or should we get rid of them? I cann't imagine our users trying to input long and lat values. We may need to support an API call for advanced users to let them pass a long and lat. values but for the dashboard, this is over kill.

Under direction: These should be a radio button because its an "OR". The API call works one at a time.

• Query sites relevant to shepherding water downstream
• Query sites and data upstream of a point.

Data Type
Update it to "Scope of upstream and downstream query"

• WaDE sites
• USGS Sites
• EPA Sites

"The Generate NLDI Map" header should be more informative like:
Display upstream or downstream sites on the map.

Bug? Los Angeles Department of Water and Power water rights not showing in WestDAAT

These two giant water rights (see the SQL script below) for the LOS ANGELES DEPARTMENT OF WATER AND POWER don't seem to show up on the map. Why?
https://westdaat.westernstateswater.org/?state=N4Ig7gTiBcoIYBsEHsDGcAuBLZA7A8mLgKZTQgAy%2BAygAQCCAcgOICiFrdAIqwAr0AlACoBZVoyG18AMVoB1ekNYCGjLrV745ykABoQAB2QIAngiwkYAbQC6%2B3AgAmWaVgQZSXTHBigEmLAwAV0diGFwgpH0UXABzQJCw6AiokGcIYlRsPABnGABmfUdvXmQLDDzoAHYAXxr9AFs4A18QAC9kZAaKYgA3YgQYKoA6AEZRgBYATgAGKYmANhmAVmWq5anV6IDg0IKF4fyFiYAmE831-LWFqui8eN2kgFpxkYAOfJOZk%2BmF0aOFvl6uAII5WkZytQsG0kiBenoQE0DMwIMgggYLLEYCAAEZBEB1IAWe need to add map coordinates to the url so the user can replicate the same exact view to a certain location.

/****** Script for SelectTopNRows command from SSMS  ******/
SELECT [AllocationAmountID]
      ,[OrganizationID]
      ,[VariableSpecificID]
      ,[MethodID]
      ,[PrimaryUseCategoryCV]
      ,[DataPublicationDateID]
      ,[DataPublicationDOI]
      ,[AllocationNativeID]
      ,[AllocationApplicationDateID]
      ,[AllocationPriorityDateID]
      ,[AllocationExpirationDateID]
      ,[AllocationOwner]
      ,[AllocationBasisCV]
      ,[AllocationLegalStatusCV]
      ,[AllocationTypeCV]
      ,[AllocationCropDutyAmount]
      ,[AllocationFlow_CFS]
      ,[AllocationVolume_AF]
      ,[PopulationServed]
      ,[GeneratedPowerCapacityMW]
      ,[IrrigatedAcreage]
      ,[AllocationCommunityWaterSupplySystem]
      ,[SDWISIdentifierCV]
      ,[AllocationAssociatedWithdrawalSiteIDs]
      ,[AllocationAssociatedConsumptiveUseSiteIDs]
      ,[AllocationChangeApplicationIndicator]
      ,[LegacyAllocationIDs]
      ,[WaterAllocationNativeURL]
      ,[CropTypeCV]
      ,[IrrigationMethodCV]
      ,[CustomerTypeCV]
      ,[CommunityWaterSupplySystem]
      ,[PowerType]
      ,[AllocationTimeframeStart]
      ,[AllocationTimeframeEnd]
      ,[ExemptOfVolumeFlowPriority]
      ,[OwnerClassificationCV]
  FROM [Core].[AllocationAmounts_fact]

  where AllocationOwner like '%LOS ANGELES DEPARTMENT OF WATER AND POWER%' aND AllocationVolume_AF!=0
  ORDER by AllocationVolume_AF ASC

Filtering NLDI WaDE sites against WaDE database

On our last call with DPL, an efficiency challenge was raised on how to filter NLDI WaDE sites (thousands) against the WaDE database sites (2 million sites). Narrowing the filtering to a boundary of interest would be more efficient which would search for a fraction of sites in the WaDE database. NLDI does return the upstream basin boundary but it does not return the downstream one.

Potential workaround to get both downstream and upstream boundaries together

1. Get the downstream NLDI mainstem features.
2. Get comid for the last feature in downstream geojson
3. Use comid form (2) to get the upstream basin boundary. This boundary will be for both the lower and upper streams (in the original question in (1)) of the river (the entire basin boundary). (see screenshot below)
4. Use the boundary in (3) to narrow the NLDI matching in the WaDE database.
5. Let's run a small test and see how this may work.
   

Move the Toggle view to be right below the Toggle Point Size?

Move the Toggle view to be right below the Toggle Point Size
At the outset of visiting the applications, the user should be able to toggle between PODs and PODs similar to the point size.
We're going to see if this matters to our focus group users or not.

WaterAllcationNativeURLDataPublicationDateID

Site specific data viz in WestDAAT

Here is one of the use case questions that we want to support

How does water use for my city compare with other cities in the region? We also need to reflect the designs we have in this RShiny app into WestDAAT Mockups. The current site-specific mockup works fine for gauge station data but less for POD-POU data.
We'd need to have co-creation sessions with DPL to go over that.
https://waterdataexchangewswc.shinyapps.io/MapSiteSpecific_ver3/

https://xd.adobe.com/view/6ac0d60f-f506-4489-82e9-438e1f8f7c65-3f93/screen/da86eb03-15f4-4fb4-9656-8e74010707ab?fullscreen

Refine our controlled vocabularies

We need to revisit our controlled vocabularies and refine them especially as we're transitioning to support data query and filtering (in the portal).

One issue here is how groundwater is defined. We probably need a new column to catch the specifics of groundwater separately.
https://docs.google.com/spreadsheets/d/1tZ3DIYDx7J-dsldHfihQjOABibMpFoGKvmSe8ReKW10/edit?usp=sharing

Analytic Popup Table Feedback & Update (05/31/2022) - Summary of site and water rights info.

For the WestDAAT popup Analytics Table -> Data Table, here are some elements we would like to see included / updated.

1. Let's only provide an abridged summary of the water right record(s) info here.

For the popup Analytics Table -> Data Table, we are thinking of only showing an abridged version of the full site or water rights information. We want to keep it simple for now here and only show some unique id, date, and mostly WaDE terminology related items (similar to what you would see on the filters). We feel we can save the more detailed versions of the site info for the site-centric landing page, and the more detailed water right information for the water right record centric landing page

Here is the list of items we would like to see on that popup Analytics Table -> Data Table, in order (with more important features appearing at the right).

• WaDEUUID for water right (coming soon) (maybe for now cheat and use the AllocationAmountID as a temp place holder)
• Priority Date
• WaDE Owner Classification
• WaDE Legal Status
• Allocation Flow (CFS)
• Allocation Volume (AF)
• WaDE Beneficial Use

Incorporate Fuzzy Matching in WestDAAT search functions

https://www.poweredsolutions.co/2019/03/26/fuzzy-matching-in-power-bi-power-query/