datopian / dashboard-js Goto Github PK

As a new User I want to know what this is and what it does so that I can decide whether to investigate further

This is a library and pattern for quickly creating dashboards using data package data sources. It is lightweight and easy to customize.

Requires only knowledge of HTML and CSS
Extendable: base widgets are simple react and you can build your own or extend

Acceptance criteria

• I have a section about what this library does in the README
• There is no syntax errors (you need to proof read)

Tasks

• create a draft of the section you going to add
• get it reviewed by somebody
• improve per feedback
• push to github

Is it possible to install the package via NPM? If so, can we include the instructions in the docs? If not, can we publish the package to NPM and then add to the docs? I'm happy to help with this :)

tried with both view config https://github.com/sagargg/dashboards but views are not rendering.
I've checked that file is serving from github on CKAN-NG and url endpoints is working. http://localhost:4000/dashboards/co2-emission-by-nation.
On the browser side it is throwing error Unhandled Promise Rejection: ReferenceError: Can't find variable: TransformStream

If the suffix field is missing from the figure spec definition the template displays undefined. It should display nothing instead.

var config = { 
  widgets: [
    {   
      "elementId": "testWidget",
      "view": {
        "title": "Economy",
        "legend": "Recent unemployment rate",
        "resources": [
          {
            "datasetId": "unemployment",
            "name": "unemployment-rate",
            "transform": [
              {
                "type": "aggregate",
                "fields": [
                  "date"
                ],
                "operations": [
                  "argmax"
                ]
              }
            ]
          }
        ],
        "specType": "figure",
        "spec": {
          "fieldName": "unemployment_rate"
        }
      }   
    }   
  ],  
  datasets: ["https://datahub.io/london/unemployment"]
}

Acceptance criteria

• missing suffix displays no suffix

Tasks

• debug
• fix
• deploy

@anuveyatsu commented on Thu Jun 20 2019

This creates buggy behaviour, e.g., we have case where we check if a file is FileRemote class by accessing its file.constructor.name - https://github.com/datopian/data.js/blob/master/lib/parser/csv.js#L43-L46 - however, after debugging, it appeared that all constructor function names are renamed to t in build files.

Here is a similar issue with uglifyJS lib - mishoo/UglifyJS#1429

This one is from create-react-app which we use in dashboards-js - facebook/create-react-app#1476

@anuveyatsu commented on Thu Jun 20 2019

This issue appears after compilation to ES5 so it is out of the scope of this repo. Moving to dashboard-js.

Our current transforms are implemented here and it uses old vega transform lib - https://github.com/frictionlessdata/datapackage-render-js/

A new vega-transform lib has a number of changes. We need to perform analysis if it worth to use it (is it usable, eg, as a plugin) or we should keep our old transforms and extend if needed.

Analysis

Current vega-transforms - https://github.com/vega/vega/tree/master/packages/vega-transforms

Pros:

• it looks modularized
• it has tests
• it is up-to-date and well maintained

Cons:

• it will require us to implement parsing 'transform' spec, e.g., Vega has 'parser' library here - https://github.com/vega/vega/tree/master/packages/vega-parser
• we never needed more than what we already have
• we'll need to modify existing transforms (we have 5-6 such examples but we don't know how many datasets on datahub are using it)

• Link to widget example in index.html should link to line number (index.html is big)
  • Should js code be separate from the index.html for clarity e.g. in app.js?
    • we're going to have "example" section in #4
• npm run eject - do we need this documented (do we need rest documented or could we link to create react app docs?)
  • There is a bunch of "create react app" boilerplate still in the readme.
• Explain what widgets are (a little diagram of a dashboard page)
  • this should be part of #3 where we explain what it is

When I import bundled code directly from raw github, I get the following error in the browser's console:

Refused to execute script from 'https://raw.githubusercontent.com/datopian/dashboard-js/master/build/static/js/main.js' because its MIME type ('text/plain') is not executable, and strict MIME type checking is enabled.

This is how we import:

<script type="text/javascript" src="https://github.com/datopian/dashboard-js/blob/master/build/static/js/main.js?raw=true"></script>

Acceptance criteria

• I can import the bundled code easily with 1-2 line of snippet that I can just copy-paste from the README

Tasks

• research and analysis of what would be a workaround for this
• implementation

Analysis

After quick googling found this:

We added the X-Content-Type-Options: nosniff header to our raw URL responses way back in 2011 as a first step in combating hotlinking. This has the effect of forcing the browser to treat content in accordance with the Content-Type header. That means that when we set Content-Type: text/plain for raw views of files, the browser will refuse to treat that file as JavaScript or CSS. (Read full article here - https://github.blog/2013-04-24-heads-up-nosniff-header-support-coming-to-chrome-and-firefox/)

Some workarounds from stackoverflow:

https://stackoverflow.com/a/20311329/6502239

So there is a service that can help us to serve JS files from github - https://combinatronics.com/ which we will test/use. Our URL for bundled code will become - https://combinatronics.com/datopian/dashboard-js/master/build/static/js/main.js

How would I add a Figure widget?
What's the specification?

Tasks

• add a section under 'Usage' section which is called 'Figure Widgets'
• explain how to write a Figure widget, e.g., have an example + explanation about each property

As a new User, I want to know how to use this to build my own dashboard so that I can do that …
Get started instructions or example directory …
Add <script … dashboard-js.js></script> to your HTML file (from raw GitHub)
Add a widget placeholder

Acceptance criteria

• Have an instructions section in README of how to include a dashboard-js library and how to configure widgets and data sources

• Have a demo-project with the dashboard-js.js library

• Have the link of the demo-project inside of README

• Have a link of the city-indicators project

Tasks

• Create a draft of the section with the instruction of how to configure the project with dashboard-js.js

• Have the draft reviewed

• Improve the draft per feedback

• Push it to GitHub

• Create a demo-project that uses the dashboard-js library

• Have the project reviewed

• Improve the project per feedback

• Push it to the GitHub

• Add link in dashboard.js-js README to the demo-project

• Add link in the README of the https://london.datahub.io/ as an example of a more complex project

Acceptance criteria

• Figure widget supports transforms

Tasks

• move spec up to view object so that only suffix and prefix will remain
• re-use transforms from datapackage-render-js
• change suffix to a simple text “prefix{value}suffix”
  • If not defined just render the number otherwise render that text
• remove logic showing last/first data - it should expect just 1 row and should render specific elements from that row

Analysis

Figure spec should be minimal, e.g., title, legend etc should go up into "view" object. The following properties will be kept in Figure spec:

• suffix and prefix - these are optional, see example below
• fieldName - a name of the target field. It is needed since after transformation we will have a single row as a dictionary of field: value pair

"spec": {
  "fieldName": "rate",
  "suffix": "%"
}

this will display 4.5%. You also can have "prefix" attribute which will be placed before the value so that it will display prefix4.5%.

If I try to enable a figure widget using the following configuration:

{"widget":{
  "elementId": "publicTransit",
  "view": {
    "title": "XXXXXXX Rides",
    "legend": "XXXXXXXX Total rides",
    "resources": [
      {
        "datasetId": "montreal-kpis",
        "name": "trips",
        "transform": [
          {
            "type": "aggregate",
            "fields": [
              "Valeur"
            ],
            "operations": [
              "argmax"
            ]
          }
        ]
      }
    ],
    "specType": "figure",
    "spec": {
      "fieldName": "Valeur"
    }
    }
  },
"dataset": "https://datahub.io/starsinmypockets/montreal-kpis"}

I get the following console error:

Uncaught (in promise) Error: No newline characters found in your file...
    at e.sniff (index.js:648)
    at csv.js:234
    at _ (runtime.js:62)
    at Generator._invoke (runtime.js:288)
    at Generator.e.<computed> [as next] (runtime.js:114)
    at n (asyncToGenerator.js:3)
    at s (asyncToGenerator.js:25)
    at asyncToGenerator.js:32
    at new Promise (<anonymous>)
    at asyncToGenerator.js:21

I am trying to write a new dashboard for Montreal. I am using the london dash as a model for the project. Figure fails to render with the error message:

dashboard.js:formatted:69368 Uncaught (in promise) TypeError: e.on is not a function
at dashboard.js:formatted:69368

Config: https://github.com/datopian/montreal-city-indicators/blob/master/widgets/publicTransit.json

Not sure what I am doing wrong here.

Following the http://kb.datopian.com/dashboards/#configure-vizualizations and Repo Readme
I did understand to create a widget but I didn't understand how datasets property gets the data resources from remote on config.
Can't figure out how to create a view widget using another resource instead of datahub.io resources.

Is it possible to get better source maps working? Currently console errors appear to be formatted such as:

dashboard.js:formatted:69368 Uncaught (in promise) TypeError: e.on is not a function
at dashboard.js:formatted:69368
at new Promise ()
at e.exports (dashboard.js:formatted:69367)
at dashboard.js:formatted:18424
at x (dashboard.js:formatted:26180)
at Generator._invoke (dashboard.js:formatted:26158)
at Generator.e. [as next] (dashboard.js:formatted:26195)
at n (dashboard.js:formatted:9199)
at s (dashboard.js:formatted:9213)
(anonymous) @ dashboard.js:formatted:69368
e.exports @ dashboard.js:formatted:69367
(anonymous) @ dashboard.js:formatted:18424
x @ dashboard.js:formatted:26180
(anonymous) @ dashboard.js:formatted:26158
e. @ dashboard.js:formatted:26195
n @ dashboard.js:formatted:9199
s @ dashboard.js:formatted:9213

It would be nice if the stack trace referred to the source file, rather than the build file. I believe this can be accomplished with webpack.

I think that rather than using config for widget definitions and html markup for the dashboard layout, that we can encapsulate layout and configuration by using React JSX components.

I'm thinking along the lines of:

app.js

// psuedocode 
import { Dashboard, Row, Widget } from components
import specs