equinor / webviz-config Goto Github PK

• Hide unused plot options

• Add tests

This is mainly to start a discussion on how to have interaction between different plugins.

Motivation

The main use case for interactions is data "drilldown".
E.g. narrowing data selection by having a global filter or by selecting data in one visualization which updates data in another visualization.

Here is an example of potential interaction between two webviz-subsurface plugins.
A callback has been added to the SummaryStats plugin which stores the realization of the currently clicked line. A callback in the Intersect plugin picks up the stored realization and updates the plot. This simple interactivity makes it much easier to navigate and understand the underlying data.

This type of interactivity is fairly easy to do within a single plugin and is described well in the Dash documentation, but it is currently not possible to do (in a robust way) between different Webviz plugins.

Solutions

The easiest solution is just to have it as it is today. Interactivity can be added for individual plugins without allowing any interactions between different plugins.

This will probably give rise to alot of duplicate code as the same type of visualizations would need to be combined in different plugins

We could mitigate this by introducing e.g. "nested plugins" which can be built by combining multiple plugins with some additional callbacks to share data

We could also make plugins more re-usable by separating out visualizations from control components (e.g. dropdowns) and callbacks making it easier to build new plugins using existing code.

Alternatively we can investigate having a shared app state with key, value pairs which plugins can read and write to.

When setting up a configuration file, it is currently difficult for the average user to know from the command line which plugins are installed.

• Add a subcommand e.g. webviz plugins which lists all installed plugins, a short explanation of what it does, together with link to hosted documentation (if it exists in setup.py, see also #100).

Currently install instructions are printed to the terminal, and only for Chromium.

Change e.g. to

• automatically open guide in default browser (this will ensure that the user also becomes aware of what is the default browser).
• move guide from terminal to github page.
• add for other popular browsers as well (Firefox etc).

I don't seem to be able to choose the width (and height) of a figure within the Markdown container.

When creating a figure with ![Alt text](./example_banner.png "Some caption") adding =100x200 at the end as in https://stackoverflow.com/questions/14675913/changing-image-size-in-markdown doesn't seem to work.

I also tried using pure html <img src="./example_banner.png" width="200"/> with no success.

Some containers has a title argument as input which renders a dcc.H1 element.
This should instead be handled by markdown outside the container to allow more flexibility in container placement and styling.

The portable standalone generated app uses hot reload when running on localhost. This adds unnecessary runtime since flask then loads twice initially (hot reload is only useful during interactive making of the webviz configuration file, i.e. when running in non-portable mode - not when viewing a finished app).

A user expects a default theme when starting webviz.

• Improve fonts in default theme

• Make default logo

A user want to style Plotly graphs using a theme with e.g. colors
This could be done by defining a dictionary that is passed into each container.
The dictionary could then be added to a plotly graph in a callback.
E.g. my_plot['layout'].update(self.plotly_layout)

• Add a plotly_layout dict to the default theme
• Add plotly_layoutas a SPECIAL_ARGS

When running webviz build some_config.yaml, users want the webviz application to open automatically in default browser.

Consider doing the same when running webviz build some_config.yaml --portable ./some/outputfolder.

A user wants to control how plugins are placed on a page.
Some testing is needed to find the best solution.
It is probably easiest to expose some classes that can be modified using css from a theme.

• Add a "plugin-layout" class to plugins
• Wrap plugins in a parent div with a "plugin-parent-layout" class

Move the abstract base class WebvizContainer, which all actual containers inherit, to separate module in order to remove risk of circular import (this can happen if a container author imports their own container in a non-standard way in e.g. test scripts).

The webviz-config code base has gotten new features since first draft was commited. The code is still quite modular, however, there are room for improvement and restructuring.

A user wants to render a markdown text file
Currently markdown can be added as individual lines in the config file. This gets messy for large sections of text.

• Add container that reads markdown files
• Add support for images (if possible)

A user wants to have sub pages to organize a lot of content.

• Add support for tabs in the config file

• Add tabs in the jinja templating

• Add theming of tabs

With Travis you now recently get Chrome version 74 when asking for stable Chrome.

This results in Travis CI failing due to incompatible driver. Update to e.g. this line to a more recent driver like https://chromedriver.storage.googleapis.com/74.0.3729.6/chromedriver_linux64.zip

Should we implement a webviz.Graph class that inherits dcc.Graph for use in our containers?
This would allow us to set e.g. config options such as responsive and modeBarButtonsToRemove for all graphs as default.

A user wants to view PDF files in webviz

This could be easily done by using the built in dash html components.
E.g. html.Embed(pdf_file)
But this will require changes to the CSP.

• Investigate most secure way of adding PDF files
• Add a container to view PDF files

The Dash Table API has been changed in 4.0.0

https://dash.plot.ly/datatable/interactivity

E.g. the bool sorting is replaced by a string sort_action
filtering with filter_action

• Implement Dash Table API changes in the DataTable container

...perhaps using either MathJax or KaTeX.

Depends on #27.

Currently there are ANSI sequences like '\033[92m' in different output statements. These are hard to read in the code. Collect as constants to a single class/module, and instead use those definitions.

• Bump Dash version to 1.0.0

• Rewrite tests to use new Dash pytest fixtures

When typed as an integer without e.g. "+47", yaml treats it as an integer, which then fails when bleach html sanitize it (expects a string).

Increase the cache timeout from 60 seconds to 1 hour.

Based on an idea from @perolavsvendsen. The base class for WebvizContainers could perhaps facilitate showing metainformation like:

• What is the data source?
• Who to contact for more detailed information?
• A guided tour in the more complex containers (using more text than what is practical to always show/include in e.g. a plot).

Currently we are using simple flask-caching. This is not 100% multithread compatible according to docs.

Change to e.g. redis?

• Add Markdown file to Webviz store

Support for adding container specific assets.

Before the WebvizContainer class was introduced custom containers could be added e.g like so:

1. Create a python script with a Container class. E.g.:

import dash_html_components as html


class OurCustomContainer():

    def __init__(self, title: str):
        self.title = title

    @property
    def layout(self):
        return html.Div([
                         html.H1(self.title),
                         'This is just some ordinary text'
                        ])

2. Make the scripts location available in PYTHONPATH
3. Include the Container in the config file as:
   - container: <scriptname>.OurCustomContainer

With the addition of the WebvizContainerparent class the above script should be:

import dash_html_components as html
from webviz_config.containers import WebvizContainer


class OurCustomContainer(WebvizContainer):

    def __init__(self, title: str):
        self.title = title

    @property
    def layout(self):
        return html.Div([
                         html.H1(self.title),
                         'This is just some ordinary text'
                        ])

and added to the config file in the same way.

This does not work and raises an AttributeError:

AttributeError: module 'webviz_config.containers' has no attribute 'OurCustomContainer'

Currently hot reload has mostly been tested with simple yaml configuration files, using simple plugins. It also only checks the yaml configuration file for changes - not files linked to within the configuration file.

• Extend file watch to also include input files to different plugins? Markdown files, ++
• Does caching work also with hot reload?

With debug=False it looks like Python error messages are not shown in the terminal. Investigate how to combine (if possible):

1. debug=False
2. print error messages to terminal

There is some information here and here.

E.g. Travis.

Both non-portable and portable mode of webviz build.

Currently webviz-config uses basic HTTPS authentication from dash-auth when running on localhost . This degrades the user experience due to:

• The user has to provide some previously configured webviz username+password pair each time opening a browser —> less productive.
• On a single-user (e.g. private) computer/machine it is an unnecessary barrier (the server is only accessible from localhost, and there is only one user with access locally).

An alternative approach could be to use a token 🔑 based approach instead when running on localhost. The user will be familiar with this approach from Jupyter notebooks.

• https://jupyter-notebook.readthedocs.io/en/stable/security.html
• https://neilmadden.blog/2019/01/16/can-you-ever-safely-include-credentials-in-a-url/

Depends on #37.

There are needs of getting the data used in a container/visualization. This could e.g. for using the values further in Excel sheets for specific calculations, for use in reports etc.

Add a "Download to csv"-function to the future ABC-container modebar, and let each container author provide a function which e.g. provides a dataframe structure which should be transferred to csv.

• Add more docstrings to the _certificate.py module.
• Extend lifetime, and simultaneously even further restrict DNS_NAME scope (to e.g. localhost only).

When plugins are missing a required argument, or it is of the wrong type, the user is currently provided by a decent error message. Example:

Traceback (most recent call last):
[...]
webviz_config._config_parser.ParserError: The plugin `DataTable` requires the argument `csv_file` in your configuration file.

This could perhaps be further improved by also giving a URL to hosted plugin documentation (if available from the setup.py install script and project_urls["Documentation"]).

Currently list and dict can't be used as arguments in functions given to webviz_store.

A user that is developing a container should be able to inherit some default arguments.
(e.g. plotly_layout)

• Add parent class for containers
• Rewrite existing containers to use parent class

When creating a portable instance, the different functions called can be time consuming and many.
The user can sometimes be unsure if it is still running and/or overall progress

E.g.:

• Add a dynamic "terminal spinner" to show the script is still live.
• Add overall progress (e.g. finisihed with x% of the functions to be called).

Simplification and standardization can be done by changing the app dockerfile to use an official base image. In addition to end-user simplification, we facilitate pushing out security updates if necessary in the dependency tree.

• Verify that current deployment setup works with recent changes.
• Check if manually created webviz base image can be replaced with official base image built from https://github.com/equinor/webviz-docker.
• Do changes in https://github.com/equinor/webviz-docker if necessary.
• Simplify the setup in https://github.com/equinor/webviz-config/tree/master/webviz_config/static by going from

  ARG WEBVIZ_BASE_IMAGE
  FROM $WEBVIZ_BASE_IMAGE
  

  to FROM webviz/base_image.

When running webviz build with the --portable option, no files are generated in webviz_store.

If debug in the flask server is not set to True, it looks like hot reloading of the dash application is not working. According to dash documentation, there is a flag dev_tools_hot_reload which can be set to True to enable hot reloading (without enabling the whole suite of debug tools).

Here is the line in the template setting debug/dev_tools_hot_reload and here is the dash documentation on the dev tools.

The package should be automatically built and uploaded to pypi on tag releases.