Coder Social home page Coder Social logo

noaa-gsl / mats Goto Github PK

View Code? Open in Web Editor NEW
6.0 5.0 0.0 71.92 MB

MATS is a quick & interactive way to view verification statistics

Home Page: https://gsl.noaa.gov/mats/

CSS 1.15% HTML 0.25% JavaScript 55.09% Shell 6.43% Python 19.08% Perl 0.01% Gherkin 17.76% Dockerfile 0.10% Go 0.13%

mats's Introduction

MATS: Model Analysis Tool Suite

MATS is a quick & versatile way to view model verification metrics. It is built using the Meteor Framework.

Development

Getting Started

To build a MATS application (MATS/apps/<appname>) you only need Docker installed. MATS can be built locally with the build.sh script in the root of the repo or like so:

$ APPNAME=aircraft
$ docker build \
    --build-arg APPNAME=${APPNAME} \
    --build-arg BUILDVER=dev \
    --build-arg COMMITBRANCH=$(git branch --show-current) \
    --build-arg COMMITSHA=$(git rev-parse HEAD) \
    -t mats/development/${APPNAME}:dev \
    .

Note that this repo uses git submodules so to get started you will need to run git submodule update --recursive after you clone the repo. You will also need to create a settings file. If you're a GSL developer, you can clone the mats-settings repo. Otherwise, contact the MATS team for a sample settings file.

To do further development you will want to install Meteor. Installation instructions can be found in Meteor's documentation.

Once installed, you can build and run a specific app with the following:

$ cd ./apps/aircraft
$ meteor npm install
$ env METEOR_PACKAGE_DIRS=../../MATScommon/meteor_packages/ \
    meteor run \
      --settings <path/to>/mats-settings/configurations/local/settings/aircraft/settings.json

You should be able to access the app from localhost:3000

Testing

The test suite for MATS consists of cucumber-style acceptance tests powered by webdriver. You will most likely need to run them with the app hooked up to an internal GSL database.

The Webdriver tests rely on the XPath so you will most likely need to run MATS with the --production flag otherwise it will inject extra <div> tags like MeteorToys and some tests will fail. Running MATS with this flag will look something like the below. Your settings path will probably be different.

$ cd apps/radar && \
    env METEOR_PACKAGE_DIRS=../../MATScommon/meteor_packages/ \
    meteor run \
    --production \
    --settings ~/git/mats-settings/configurations/local/settings/radar/settings.json

More on testing can be found in the testing README

Other Documentation

You can find a high-level overview of the project's CI process in the CI README.

Disclaimer

This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an “as is” basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.

mats's People

Contributors

matsapps avatar mollybsmith-noaa avatar randytpierce avatar ian-noaa avatar jeffhamiltonnoaa avatar kirklholub avatar weixue21 avatar dependabot[bot] avatar gopa-noaa avatar billmoninger avatar

Stargazers

 avatar Branden Collingsworth avatar Laura Melling avatar Dave Turner avatar Curtis Alexander avatar

Watchers

James Cloos avatar Kostas Georgiou avatar  avatar  avatar Curtis Alexander avatar

mats's Issues

Text on TimeSeries


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21232, https://vlab.ncep.noaa.gov/redmine/issues/21232
Original Date: 2016-08-12
Original Assignee: randy.pierce


Text is broken for TimeSeries. The times on the left only correspond to the values first curve. If more curves are added (especially curves with much less data) the difference is obvious. The data on the new curves will fill the first few rows, regardless of what time its values actually correspond to.

  1. Add more than one curve (it’s more obvious if the 2nd curve as only a few data points)
  2. Plot graph and look at text

Label incrementing is broken for custom labels and editting


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21233, https://vlab.ncep.noaa.gov/redmine/issues/21233
Original Date: 2016-08-12
Original Assignee: randy.pierce


After editing a curve the label incrementing breaks, doesn’t increment and seems to try and start over by just adding another C-0 at the end of the first. Custom labels also do not increment (they may not be meant to, but adding a -1 or something on the end might be a good idea).

  1. Add several curves
  2. Edit one of the curves (you don’t have to change anything, just click edit)
  3. Save changes and attempt to add more curves.
  4. Try removing the curves currently added and add more
    or
  5. Try adding a custom label to a curve
  6. Press 'Add Curve' twice

Text on 'hide curve' and 'hide points' buttons not updating properly


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21244, https://vlab.ncep.noaa.gov/redmine/issues/21244
Original Date: 2016-08-12
Original Assignee: randy.pierce


If 'hide curves' or 'hide points' is set to 'show' when the 'Back' button is pressed they will not update when a curve with the same label is plotted again, and will read 'show' even if the curve is already visible.

  1. Plot a curve
  2. Press ‘hide curve’
  3. Hit ‘back’ and plot the curve again
  4. Look at the ‘hide curve’ button

Editing curves not working


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21240, https://vlab.ncep.noaa.gov/redmine/issues/21240
Original Date: 2016-08-12
Original Assignee: randy.pierce


Editing curves does not seem to work at all. Curve text is not updated and curve is ultimately not changed.

  1. Plot a couple of curves
  2. Edit one to be identical to another and see if they are on the plot

Changes needed to make publication-quality plots


Author Name: bonny.strong (@bonnystrong)
Original Redmine Issue: 12197, https://vlab.ncep.noaa.gov/redmine/issues/12197
Original Date: 2015-10-01
Original Assignee: randy.pierce


From Stan's experience submitting paper to Monthly Weather Review, the following changes needed to be made to existing EMB verification plots.


Forecast lengths of 6 or 18 not working


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21248, https://vlab.ncep.noaa.gov/redmine/issues/21248
Original Date: 2016-08-12
Original Assignee: randy.pierce


Attempting to plot a curve with a forecast length of 6 or 18 will bring up an internal server error. (This is an issue in the database I think)

  1. Add any sort of curve with a forecast length of 6 or 18
  2. Attempt to plot

Diff's not plotted for 'Plot Unmatched'


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21229, https://vlab.ncep.noaa.gov/redmine/issues/21229
Original Date: 2016-08-12
Original Assignee: randy.pierce


Diff’s are not plotted when ‘Plot Unmatched’ is selected on TimeSeries and Profile. There was talk of disabling the 'Plot Unmatched' button when diff curves are added

  1. Add two different curves
  2. Check the ‘absolute diffs’ box
  3. Click ‘Plot Unmatched’

Zooming is inconsistent


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21241, https://vlab.ncep.noaa.gov/redmine/issues/21241
Original Date: 2016-08-12
Original Assignee: randy.pierce


All plots need to have a maximum value on both axis, otherwise zooming out is not consistent when one axis stops but the other zooms out to infinity.

  1. Plot a profile or time series graph
  2. Try zooming out with the scroll wheel or buttons
  3. The graph will quickly compress itself.

Region selector moves


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21236, https://vlab.ncep.noaa.gov/redmine/issues/21236
Original Date: 2016-08-12
Original Assignee: randy.pierce


Not a serious issue, but changing the value of any selector moves the ‘region’ selector to the right a few pixels.

  1. Refresh the app
  2. Change any selector while watching ‘region’

'Cloud Coverage' parameter has no effect


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21247, https://vlab.ncep.noaa.gov/redmine/issues/21247
Original Date: 2016-08-12
Original Assignee: jeffrey.a.hamilton


Changing the ‘cloud coverage’ parameter does not change the graphs at all, though this may be due to an issue with the data and not the app

  1. Plot two TimeSeries or Profile plots that are identical in every way except the selected cloud coverage parameter
  2. Compare the curves on a graph

Test issue


Author Name: bonny.strong (@bonnystrong)
Original Redmine Issue: 8853, https://vlab.ncep.noaa.gov/redmine/issues/8853
Original Date: 2015-06-22
Original Assignee: xue.wei


Created this to play around with and learn how the system works. I'm assigning to Xue initially.

Xue, when you get this, can you change the Status field to "In progress".

Jeff, I set you up as a "watcher". So I expect you should see if the status changes.

Incorrect regions displayed in selection for FIM


Author Name: bonny.strong (@bonnystrong)
Original Redmine Issue: 18749, https://vlab.ncep.noaa.gov/redmine/issues/18749
Original Date: 2016-05-13


During demo at global modeling meeting, when selected FIM first, it did not show global regions for selection (e.g. NHX, SHX, global).

Also, when Shan requested to change one parameter for plot, had to refresh whole screen and start over to select all parameters for 2 models again. In other words, did not support editing a parameter.

First curve on TimeSeries does not always work


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21234, https://vlab.ncep.noaa.gov/redmine/issues/21234
Original Date: 2016-08-12
Original Assignee: randy.pierce


Sometimes the first curve added to a TimeSeries plot will return no data. This usually happens after the page has just been loaded or refreshed, happens approximately 1-in-4 times, and is not influenced by anything that I can figure out.

  1. Refresh the page and try to plot any curve. (you may have to do this 5 or 6 times)

Issues with number spinners


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21231, https://vlab.ncep.noaa.gov/redmine/issues/21231
Original Date: 2016-08-12


Two strange things happen when number spinners are opened and their values are changed:
They will close and flicker open if the selector (the dark blue part of the button) is clicked again
They will close if another button on the page is clicked, sometimes with other behavior (if the 'Add Curve' is clicked, for example, the open
number spinner will close and no curve will be added).

To test these open any number spinner and then change its value, then try and close it or click on something else. These may be two different issues to be resolved separately.

Sites map has issues showing multiple sites selected


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21239, https://vlab.ncep.noaa.gov/redmine/issues/21239
Original Date: 2016-08-12
Original Assignee: randy.pierce


The highlighted sites on the sites map do not show when multiple sites are selected, only one site is ever highlighted at a time and choosing ‘all’ on the sites selector does not highlight anything.

  1. Open the ‘sites’ selector and the ‘sites map’
  2. Try selecting multiple sites on the map
  3. Try checking the ‘all/none’ box for sites

'Plot Matched' not working with 3+ curves


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21225, https://vlab.ncep.noaa.gov/redmine/issues/21225
Original Date: 2016-08-12
Original Assignee: randy.pierce


When more than two curves are plotted on profile and difference curves are added plot matched will not work

  1. Add 3 different curves (ideally with some mismatched data)
  2. Check the ‘absolute diffs’ box
  3. Plot matched

More than one diff curve breaks Visibility/Surface/Ceiling


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21250, https://vlab.ncep.noaa.gov/redmine/issues/21250
Original Date: 2016-08-12
Original Assignee: randy.pierce


If more than one difference curve is added when “Plot Matched” is pressed the app will throw an internal server error [500]

  1. Add at least three curves to the plot
  2. With three curves added ‘show matching diffs’ will add two diff curves and will give an error when “Plot Matched’ is pressed
  3. ‘Pairwise diffs’ will work with three curves because it will only add one diff curve. If another curve is added ‘pairwise diffs’ will add two diff
    curves and will also give an error when ‘Plot Matched’ is pressed.

Profile results do not match Bill's


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21246, https://vlab.ncep.noaa.gov/redmine/issues/21246
Original Date: 2016-08-12


The data returned by the query is sometimes a little off, sometimes very different. Graphs look similar, sometimes identical, but can be different in small ways. The query looks the same, and I’m unsure if the different numbers for the data is intentional.

  1. Plot a TimeSeries or Profile plot with both the MATS and the old RUC site
  2. Make sure both are plotting the same graph and that the RUC ‘show text’ box is checked
  3. Compare the data of the two plots. Changing any of the parameters will affect the data, however the FIM model over the RUC region with
    temperature as the variable will return obvious differences.

hide points then hide curve ends up showing points


Author Name: bonny.strong (@bonnystrong)
Original Redmine Issue: 16227, https://vlab.ncep.noaa.gov/redmine/issues/16227
Original Date: 2016-03-07


If I create a time series plot, it initially shows the curve and the points. (buttons indicate "hide curve" | "hide points")

If I do hide points, points disappear, and only curve is shown - all good (buttons: hide curve | show points)

If I then do hide curve, the curve disappears, but the points reappear - and they shouldn't. (buttons: hide curve | show points; no change in state)

If I then click show points - they disappear (buttons: hide curve | show points)

Click hide curve; both curve and points reappear (buttons: hide curve | show points)

The value display on label is not always correct


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21235, https://vlab.ncep.noaa.gov/redmine/issues/21235
Original Date: 2016-08-12
Original Assignee: randy.pierce


The value display for the ‘label’ selector does not accurately reflect the label being added to the curve when custom labels are involved.

  1. Enter some unique text into the ‘label’ selector
  2. Add the curve
  3. Add another curve without closing the label
  4. Compare the curve texts with the value in ‘label’ and the value displayed for ‘label’

All the Profile and TimeSeries plots desperately need to be refactored with the new dataset


Author Name: kevin.bruhwiler (kevin.bruhwiler)
Original Redmine Issue: 21245, https://vlab.ncep.noaa.gov/redmine/issues/21245
Original Date: 2016-08-12


I'm listing this as urgent because I think a lot of other bugs are going to go away once this is finished and it would be a very good idea to do this first. The present code is a mess, takes a long time to decipher and debug, and isn't worth the effort since it should all be changed soon anyway.

adding all models except for retros to upper air


Author Name: xue.wei (xue.wei)
Original Redmine Issue: 14577, https://vlab.ncep.noaa.gov/redmine/issues/14577
Original Date: 2015-12-30
Original Assignee: xue.wei


I have added all models to upper air, excluding those models which have stopped running and all retros.

Model names displaying in the webpages like upper-air, surface, ceiling and other verifications are not consistent ,
like in upper-air, HRRR model displays like HRRR (3km CONUS)
in surface , HRRR model displays like HRRR (0 1 2 3 6 9 12h fcsts)
I will talk to Bill to see if it is better to make all same.

Add Bill's changes for plots to MATS plots.


Author Name: bonny.strong (@bonnystrong)
Original Redmine Issue: 11834, https://vlab.ncep.noaa.gov/redmine/issues/11834
Original Date: 2015-09-18
Original Assignee: bonny.strong


Review emails to/from Bill Moninger and Stan about plotting changes needed for publication, and ask Bill about what other changes he made. Implement same plotting features in MATS.

See attached document for partial summary of Bill's emails with Stan.


Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.