heavyai / heavyai-charting

Dimensional charting built to work natively with crossfilter rendered using d3.js

heavyai

This package enables using common Python data science toolkits with HeavyDB. It brings data frame support on CPU and GPU as well as support for arrow. See the documentation for more.

Quick Install (CPU)

Packages are available on conda-forge and PyPI:

# using conda-forge
conda install -c conda-forge heavyai

# using pip
pip install heavyai

Quick Install (GPU)

We recommend creating a fresh conda 3.8 or 3.9 environment when installing heavyai with GPU capabilities.

To install heavyai for GPU Dataframe support (conda-only):

mamba create -n heavyai-gpu -c rapidsai -c nvidia -c conda-forge -c defaults \
    --no-channel-priority \
    cudf heavyai pyheavydb pytest shapely geopandas pyarrow=*=*cuda

Documentation

Further documentation for heavyai usage is available at: http://heavyai.readthedocs.io/

heavyai-charting's Issues

line chart: "blank" chart if no variance in y

General Info

• Platform: All
• Browser Name: All
• Browser Version: All
• Application Version: master dev

Issue Description

Line chart looks blank if there's no variance in the y-axis measures - the line is there, it's just drawn directly on the bottom axis.

Expected Result

Provide better indication that the line exists. Maybe draw the in the middle of the chart, or show markers (if few points), or have a labelled tick on the y.

Actual Result

1px blue line drawn on bottom axis.

Screenshots

Other Information

Lopri, won't be seen in most datasets.

review gnarly syntax in files we tell prettier to ignore

review the files listed in our .prettierignore file and see if we can refactor them so that using prettier to format them does not result in an error.

to check for errors, remove a file from the .prettierignore and then run

yarn format:check > format-check-log.txt

any filenames that have a newline following them caused an error during formatting. you should also be able to see the errors in the terminal window in which you ran this command.

Revert workaround for mapd-immerse/issues/3880

🪲 If bug

• Operating System and version:
• Browser and version:
• Steps to reproduce (including dashboard link):
• Description of issue:

🆕 If feature request

• User story:
• Specific requirements/functionality:
• Mocks (if applicable):

This is effectively reverting a workaround for a back end rendering performance issue. The original bug is here:

https://github.com/mapd/mapd-immerse/issues/3880

We have since implemented a different method of rendering symbols which no longer suffers from this problem (it's very similar to the old 'points' mark but works for all symbol shapes). Backend rendering was switched over to the new rendering method at commit aef949dd70d4abac6790ba7adeebfa04404d9ad5.

Filter Charts By Clicking Legend Item

User story

As a user, I would like to be able to click on a legend item to remove data from the chart for that legend item, so I can quickly show/hide some data series. Clicking on the same legend item again would add back the data.

Acceptance criteria

• All chart types should work the same
• Multiple data series could be toggled at the same time
• User should not be able to toggle a series when there's only one
• The line should be hidden. As a stretch goal, line should be greyed-out on legend item hover
• The y axis domain should change to the new data extent if it's not locked
• It should also work with continuous legend
• An event should be triggered that could be used to set a filter on other charts and to persist the show/hide state

Poly Raster Layer as Vega-Lite Specification

{
  transform: {
    limit?: number,
    sample?: true
  },
  mark: {
    type: "poly",
    strokeColor: ColorType,
    strokeWidth: number,
    lineJoin": "miter",
    miterLimit": number
  },
  x: {
    type: "quantitative",
    field: string | Expression
  },
  y: {
    type: "quantitative",
    field: string | Expression
  },
  color: {
    type: "quantitative",
    aggregate: string | Expression,
    domain: Array<number>,
    range: Array<number>
  }
}

contexify error on node 7.0.0 and higher

it's currently not possible to run yarn test successfully if you are using node 7.0.0 or higher/newer

Upgrade to Latest Stable node and npm Versions

We should update our repo so that it works for the latest stable versions and node and npm.

how to install mapd-charting on ubuntu 16 or other linux-like operating system?

🪲 If bug

• Operating System and version:
• Browser and version:
• Steps to reproduce (including dashboard link):
• Description of issue:

🆕 If feature request

• User story:
• Specific requirements/functionality:
• Mocks (if applicable):

Release as npm Module

Deprecate Use of Crossfilter In Favor of SQL Oriented Data Layer

I am proposing that we deprecate the use of crossfilter and adopt a more SQL oriented data layer.

The current mapd-crossfilter library falls short in three key areas:

1. expressivity
2. development velocity
3. modularity

By adopting a more expressive, relational data layer, the full potential of the mapd-core backend can finally be utilized. In addition, it opens the door to new possibilities such as leveraging the Vega runtime.

Issues With Crossfilter

Expressivity

The only way to increase crossfilter's expressivity is to decrease its maintainability.

Crossfilter works for building very simple OLAP queries, but once you start building slightly more complicated projections and aggregations, the API falls apart.

For instance, when one projects (creates a "dimension") on a time field, this field is implicitly cast but is then "uncast" with regex when queries are constructed. We see the same pattern of using an awkward projection setter and then extracting field information with regex when creating our raster chart projections.

Since the dimension constructor is meant for simple projections, one has to pretty much hack the library to express more complicated relations such as subquerying. Even using SQL expressions such as COALESCE and CASE in a clean and maintainable way is pretty much off the table.

Development Velocity

Developing the crossfilter library should be a simple matter of understanding SQL and a minimal API to build relations. It is not.

In order to actually make changes or fixes to the library, one has to first understand the peculiarities of the codebase itself.

For example, one must have knowledge of idiosyncratic details such as:

• difference among global, scoped, and "regular" filters
• difference between a group and groupAll
• the exact data structures of things like "binParams"
• why there are two ways of setting a projection

Modularity

The coupling of concerns makes the library hard to use, understand, and develop. Separate concerns are not adequately modularized, which hinders testability and readability.

One example of this coupling is in the group query methods group.all(), group.top(), group.bottom() where the query writing and the querying are bound together in one method. But despite such coupling, we see that developers naturally want to decouple these pieces.

Another example is the library itself as an intermixing of concerns such as query writing, querying, caching, data processing, and result formatting.

Advantanges of a SQL Data Layer

To replace crossfilter, I am proposing a data-layer centered around SQL and relational operations. Two good starting points from which to model an API are the Vega Data Transform API and the Calcite Algebra API

The key idea here is that since crossfilter is basically a way to build up a stack of relational transformations, one might as well orient the data-layer API to this abstraction level.

Expressivity

By using a data-layer geared toward building pipelines of relational transformations, we can make full use of SQL, and therefore, mapd-core. All SQL expressions -- even subquerying -- come right out of the box.

Furthermore, if in building these transformation stacks, the API becomes a cumbersome activity, the data-layer would be expressive and flexible enough that one could easily build a higher-level abstraction (syntactic sugar) such as crossfilter.

Development Speed

Without being burdened with codebase specific knowledge, developers would just need to know how the transformation pipelines are represented and how that translates to SQL.

Cleaner Charting Code

With a cleaner and proper implementation, data layer state setting and getting would be easier to reason about. As a result, confusing code at the charting level that works around these complications would be resolved.

New Possibilities: Vega

A declarative Vega inspired data-layer would make interoperability with the Vega runtime much more natural. One can imagine a Vega oriented API where the user specifies SQL data transformations in the same way visual encodings are specified: declarative JSON specifications.

While it would not be impossible to "refactor" crossfilter to be more SQL oriented, I would actually argue that, in the time it would take to do that, one could have already built a properly abstracted data-layer, extended it, and leveraged it to enhance the capabilities of the charting library.

In the end, it's a matter of spending a finite amount of development time -- either spending it on a sunk cost or a sustainable, forward-thinking solution.

Empty bar on rowchart.

When there's only one value (and therefore only one bar) on a rowchart and this value is negative, the chart does not fill the color of the bar and just displays it empty. The same thing does not happen when the value is positive.

Here's my setup:

chart .height(height) .width(width) .elasticX(true) .othersGrouper(false) .measureLabelsOn(true) .dimension(dim) .group(g) .autoScroll(true) .cap(150);

The same thing happens in Immerse as well - see attached image.

use mapd/legendables fork

update package.json to use use mapd/legendables fork

https://github.com/mapd/legendables

Create Example of "Overview-Detail" Histograms

Currently we have the ability to create a "overview-detail" (focus-range) relationship between two line charts. We would like an example of how to do so with histogram

how can I implement my own json data

Hello,
Thank you for creating this platform.

I want to implement my own json data which I get from a service. So I see that mapd-crossfilter does not support that as original crossfilter. So how can I make it work with my own json data?

Infinite Scrolling Row Chart

Instead of limiting the number of groups, the row chart should be made infinitely scrollable.

If not a default, could be turned on as an option.

Geojson with multi-polygon shapes are not rendering properly

🪲 If bug

• Operating System and version:
• Browser and version:
• Steps to reproduce (including dashboard link):
• Description of issue:

🆕 If feature request

• User story:
• Specific requirements/functionality:
• Mocks (if applicable):

pointmap hover triggers and stays when going off map #2029

move mouse onto pointmap
move mouse off pointmap near where a point is
hover box shows up when mouse is off
expected behavior: when mouse is off map chart, the hover box should disappear

Rapid chrome trackpad scrolling appears to delay infinite scroll fetching of new data

General Info

• Platform: All
• Browser Name: All
• Browser Version: All
• Application Version: master dev

Issue Description

Network request will not get executed because debounce is still in effect with chrome and trackpad scrolling.

Steps to reproduce

1. Using chrome and your trackpad (2 finger scrolling)
2. Try scroll to bottom of table rapidly.
3. It will get stuck in Chrome sometimes (but not firefox) because the whole page moves up (chrome specific feature) when using 2-finger scroll.

Screenshots

Make heatmap simplify queries when all items on one axis are selected

Currently when one selects a whole row or column (or multiple rows or columns) on a heatmap, DC calls crossfilter in a way that generates a filter for every cell. This can get really expensive even on our backend at scale.

See below for the query that is generated when one selects the origin state of CA on one axis of flights.

"SELECT COUNT(*) as val FROM flights_123M WHERE (origin_state = 'CA' AND dest_state = 'AK' OR origin_state = 'CA' AND dest_state = 'AR' OR origin_state = 'CA' AND dest_state = 'AZ' OR origin_state = 'CA' AND dest_state = 'CA' OR origin_state = 'CA' AND dest_state = 'CO' OR origin_state = 'CA' AND dest_state = 'CT' OR origin_state = 'CA' AND dest_state = 'FL' OR origin_state = 'CA' AND dest_state = 'GA' OR origin_state = 'CA' AND dest_state = 'HI' OR origin_state = 'CA' AND dest_state = 'IA' OR origin_state = 'CA' AND dest_state = 'ID' OR origin_state = 'CA' AND dest_state = 'IL' OR origin_state = 'CA' AND dest_state = 'IN' OR origin_state = 'CA' AND dest_state = 'KS' OR origin_state = 'CA' AND dest_state = 'KY' OR origin_state = 'CA' AND dest_state = 'LA' OR origin_state = 'CA' AND dest_state = 'MA' OR origin_state = 'CA' AND dest_state = 'MD' OR origin_state = 'CA' AND dest_state = 'ME' OR origin_state = 'CA' AND dest_state = 'MI' OR origin_state = 'CA' AND dest_state = 'MN' OR origin_state = 'CA' AND dest_state = 'MO' OR origin_state = 'CA' AND dest_state = 'MT' OR origin_state = 'CA' AND dest_state = 'NC' OR origin_state = 'CA' AND dest_state = 'ND' OR origin_state = 'CA' AND dest_state = 'NE' OR origin_state = 'CA' AND dest_state = 'NJ' OR origin_state = 'CA' AND dest_state = 'NM' OR origin_state = 'CA' AND dest_state = 'NV' OR origin_state = 'CA' AND dest_state = 'NY' OR origin_state = 'CA' AND dest_state = 'OH' OR origin_state = 'CA' AND dest_state = 'OK' OR origin_state = 'CA' AND dest_state = 'OR' OR origin_state = 'CA' AND dest_state = 'PA' OR origin_state = 'CA' AND dest_state = 'PR' OR origin_state = 'CA' AND dest_state = 'SD' OR origin_state = 'CA' AND dest_state = 'TN' OR origin_state = 'CA' AND dest_state = 'TX' OR origin_state = 'CA' AND dest_state = 'UT' OR origin_state = 'CA' AND dest_state = 'VA' OR origin_state = 'CA' AND dest_state = 'WA' OR origin_state = 'CA' AND dest_state = 'WI' OR origin_state = 'CA' AND dest_state = 'WY');"

This query could just as be easily represented as WHERE origin_state = 'CA'. Multiple columns and rows selected could simply be (row_var = 'A' or row_var = 'B' or col_var = 'C' or col_var = 'D'). Would be good to scope what it would take (likely simple) in the heatmap chart in DC to generate these simpler (and more performant) queries.

Simplify How To Use Library

Using this library should be a simple matter of installing the package and importing it (or using a bundle)

Currently, its very cumbersome to use this library. One has to import it like so:

require("script-loader!mapbox-gl/dist/mapbox-gl.js");
require("script-loader!mapbox-gl/dist/mapboxgl-overrides.js");
require("mapbox-gl/dist/mapbox-gl.css");
require("@mapd/mapdc/mapdc.css");
require("@mapd/mapdc/scss/chart.scss");

import * as dc from "@mapd/mapdc";
const d3 = dc.d3;

One must also create a webpack config file like this:

var webpack = require("webpack");
var path = require("path");

module.exports = {
  entry: {
    app: [
      "script-loader!@mapd/connector/dist/browser-connector.js",
      "./index.js"
    ]
  },
  devtool: "source-map",
  output: {
    path: path.resolve(__dirname, "assets"),
    publicPath: "/assets/",
    filename: "bundle.js"
  },
  module: {
    rules: [
      {
        test: /\.js$/,
        include: [
          path.resolve(__dirname, "../../src"),
          path.resolve(__dirname, "../../index.js"),
          path.resolve(__dirname, "index.js"),
          path.resolve(__dirname, "node_modules/@mapd/mapdc")
        ],
        loader: "babel-loader"
      },
      {
        test: /\.css$/,
        use: [{ loader: "style-loader" }, { loader: "css-loader" }]
      },
      {
        test: /\.scss$/,
        use: [{ loader: "style-loader" }, { loader: "css-loader" }, { loader: "sass-loader" }]
      }
    ]
  }

explore react-map-gl-alt as an alternative to mapd fork of mapbox-gl-js

https://github.com/AlpacaTravel/react-map-gl-alt#differences-to-react-map-gl

an idea from @clhenrick

Fix Webpack Build

Currently the webpack build script compiles the code in a way where nothing is actually exposed. This is because src/index.js only exports individual modules and does not export them under a single dc namespace. There's also some tricky stuff going on like the require()'ing of our forked version MapboxGL and D3JS v3.x.

#212 starts to address this but still needs some work.

Throw an error if redraw is called before a render

🪲 If bug

• Operating System and version: MacOS 10.12.5
• Browser and version: Chrome 61.0.3163.91
• Steps to reproduce (including dashboard link):
• Description of issue:

When i check Example1 in your examples the line chart shows a nice dot with a popup on mousemove. My line chart however gives the following error on mousemove:

my implementation is as follows:

  public draw(days?: number) {

    this.crossfilter
      .groupAll()
      .reduceMulti(this.dateExpression)
      .valuesAsync(true).then((chartBounds: IMapdChartBound) => {

        // if there is no data to display for the given time range
        // chartBounds will return null values.
        // here we manually set the chartbounds for the given time range
        if (chartBounds.minimum === null && chartBounds.maximum === null) {
          switch (days) {
            case 1:
              chartBounds.minimum = subDays(new Date(), days);
              chartBounds.maximum = new Date();
              break;
            case 7:
              chartBounds.minimum = subDays(new Date(), days);
              chartBounds.maximum = new Date();
              break;
            case 30:
              chartBounds.minimum = subDays(new Date(), days);
              chartBounds.maximum = new Date();
              break;
            case 90:
              chartBounds.minimum = subDays(new Date(), days);
              chartBounds.maximum = new Date();
              break;
            case 365:
              chartBounds.minimum = subDays(new Date(), days);
              chartBounds.maximum = new Date();
              break;
          }
        }

        // return correct timebin value so graph doesn't show to much straight lines
        const getTimeBin = (days: number) => {
          const binMap = {
            1: 'hour',
            7: 'hour',
            30: 'day',
            90: 'day',
            365: 'week',
          };

          return binMap[days];
        };

          // draw transactions chart
          const transactionsRecentChart = (window as any).dc.lineChart('.transactions__recent')
            .width(this.parentWidth)
            .height(300)
            .colors((window as any).chart_colors)
            .transitionDuration(1500)
            .brushOn(true)
            .elasticY(true)
            .enablePopup(true)
            .dimension(this.transactionsRecent)
            .group(this.transactionsRecentGroup)
            .x(
              (window as any).d3.time.scale.utc()
                .domain([chartBounds.minimum, chartBounds.maximum]),
            )
            .binParams({
              binBounds: [chartBounds.minimum, chartBounds.maximum],
              timeBin: getTimeBin(days),
            });

          transactionsRecentChart
            .xAxis()
            .scale(transactionsRecentChart.x())
            .tickFormat((window as any).dc.utils.customTimeFormat);

        (window as any).dc.redrawAllAsync();
      });

This problem occurs both with the linechart and also with the piechart. Since my linechart implementation does not differ a lot from what you do in your example 1 i'm thinking i'm might missing a dependency.

Can you point me in the right direction?

Add "X" to clear zoom to field for pointmap

General Info

• Platform: All
• Browser Name: All
• Browser Version: All
• Application Version: master dev

Issue Description

When text is populated in the Zoom To field for pointmap, an X should appear at the right of the field to allow the field contents to be cleared.

Create size legend for bubble charts

For instance https://vega.github.io/vega/examples/scatter-plot/

Pointmap/BEscatterPlot should hide empty legend

General Info

• Platform: All
• Browser Name: All
• Browser Version: All
• Application Version: master dev

Issue Description

If you create a colored pointmap but remove all the colors except the default. The legend should be hidden.

Steps To Reproduce

Link To Dashboard

Screenshots

Other Information

update main to point to bundle

do this so that we can develop mapd-charting using npm link

Cursor to indicate horizontal select action on linechart, histogram

For the cursor on histogram and line chart, it would be good to provide a cursor which indicates that the user can select a portion of the chart by doing a horizontal drag. Something like a dual arrow cursor with an arrow pointing left and an arrow pointing right.

Error when pie chart group becomes really small

🪲

• Operating System and version:
  macOS 10.12.5
• Browser and version:
  Chrome 59.0.3071.115
• Mapd versions
  mapdc tag v1.1.1
  mapd-crossfilter tag v1.1.0
  mapd-connector tag v2.0.0
• Steps to reproduce (including dashboard link):
  • go to https://github-investigation.firebaseapp.com/
  • click on the user greenkeeperio-bot
• Description of issue:
  The merged pie chart goes into the loading state indefinitely
  Console shows an error like:

": error"
TypeError: t.replace is not a function

We are seeing this on https://sgc.garvan.org.au/explore as well. For example select 0.8-0.9 in the Avg Allele Frequency row chart

"error" module imported incorrectly in base-mixin

https://github.com/mapd/mapd-charting/blob/develop/src/mixins/base-mixin.js#L31

When removing number from legend input box, it should reset to default

General Info

• Platform: All
• Browser Name: All
• Browser Version: All
• Application Version: master dev

Issue Description

Difficult to know how to reset to default values on point map cont-legend

Screenshots

1. change lower bound field to a different number.

2. Press enter:

3. Click back on input box -- and then remove value.

4. When Clicking off: Number should go back -176.65

Other Information

Refactor charting examples

Refactor charting examples:

• Use a standard webpack config
• Run libraries from source vs via copying from distribution bundles
• Update to ES6
• Split example html and js more cleanly: ideally, one html file that injects examples
• Remove script tags from example html in favor of ES6 imports
• Move large inline comment blocks into documentation files

Add `prettier` Formatting as Pre-Commit Hook

As a maintainer, I would like to have a consistent format and style across the codebase.

Prettier seems like a great way to ensure that our code conforms to a consistent style.

How to set meaningful legend texts on a multiSeries chart

On the stacked bar chart demo the series legend shows meaningless names ("series_..."), as shown below:

I have tried to glean from the code how to make the popup and/or a dc.legend display the correct names for the series, but so far a way for that doesn't seem to exist at present.

Use `master` Branch as Default Dev Branch

Exception on resizing point map...

🪲 Bug

• Operating System and version: Win 10 64 bit
• Browser and version: Chrome 62.0.3202.94
• Steps to reproduce (including dashboard link): On resizing a point map (https://mapd.github.io/mapd-charting/example/example2.html) an exception is thrown.
• Description of issue: There is an exception thrown during resizing of the browser Window. It looks like the guilty method is exiting prematurely, which I presume is unexpected behaviour. The exception is caused by an undefined 'group' when requesting 'lastFilteredSize' in raster-chart.js method:
  _chart.data(group => { if (_chart.dataCache !== null) { return _chart.dataCache } const bounds = _chart.getDataRenderBounds() _chart._updateXAndYScales(bounds) _chart._vegaSpec = genLayeredVega( _chart, group, lastFilteredSize(group.getCrossfilterId()) )
  The browser console shows:

Uncaught TypeError: Cannot read property 'getCrossfilterId' of undefined
    at Object.<anonymous> (mapdc.js:32955)
    at Object._chart.data (mapdc.js:5183)
    at Object._chart._doRender (mapdc.js:33069)
    at Object._chart.render (mapdc.js:5660)
    at reSizeAll (example2.html:412)
    at complete (lodash.js:8619)
    at delayed (lodash.js:8629)

Properly cancel hit testing on raster chart destroy

If a user fires a raster chart destroy while a hit test is active, the existing raster chart may not be properly removed.

update ecstatic version due to known security vulnerability

I notice in this repo a security alert pointing at security vulnerability in pre-2.0 version of ecstatic. This should be upgraded.

Prettier format codebase

https://github.com/prettier/prettier

Allow user to set legend title

Change the raster-layer-heatmap-mixin to use vega transforms instead of a pre-query to generate statistics for coloring the heatmap bins.

🪲 If bug

• Operating System and version:
• Browser and version:
• Steps to reproduce (including dashboard link):
• Description of issue:

🆕 If feature request

• User story:
• Specific requirements/functionality:
• Mocks (if applicable):

The following is an example SQL query currently generated by the raster-layer-heatmap-mixin:

SELECT  
  reg_hex_horiz_pixel_bin_x(conv_4326_900913_x(lon),-20037508.34003972,20037508.34003971,conv_4326_900913_y(lat),-8577530.094065012,15584629.78017406,10.012195121951219,11.56108709767519,0,0,1642,990)
    as x,  
  reg_hex_horiz_pixel_bin_y(conv_4326_900913_x(lon),-20037508.34003972,20037508.34003971,conv_4326_900913_y(lat),-8577530.094065012,15584629.78017406,10.012195121951219,11.56108709767519,0,0,1642,990)
    as y,
  count(*) as color
FROM
  tweets_nov_feb_60M
WHERE 
  ((lon >= -179.99999999999747 AND lon <= 179.99999999999739) AND
   (lat >= -60.7889803101517 AND lat <= 80.0713755865373))
GROUP BY x, y;

To color the bins generated by this query, a pre-query is run to gather statistics. That query is this:

SELECT min(c.color) as minimum, max(c.color) as maximum, avg(c.color) as mean, stddev(c.color) as deviation FROM (<above query>) as c;

This prequery is performed when the state of the color domain of the heatmap raster layer is set to auto. For example, here is a JSON sent to the heatmap layer mixin to set its state:

const config = {
    mark: "hex",
    encoding: {
      x: {
        type: "quantitative",
        field: "lon",
        size: WIDTH
      },
      y: {
        type: "quantitative",
        field: "lat",
        size: HEIGHT
      },
      color: {
        type: "quantize",
        aggregate: "count(lang)",
        scale: {
          domain: "auto",
          range: colorRange,
          default: "#0d0887",
          nullValue: "#0d0887"
        }
      },
      size: {
        type: "manual",
        value: 10
      }
    }
  }

config.encoding.color.scale.domain is set to auto which is a keyword to specify to run the above pre-query and then use the following expressions for the color domain:

config.encoding.color.scale.domain = [max(minimum, mean - 2*deviation), min(maximum, mean + 2*deviation)]

Rather than running a pre-query, we can instead use vega transforms to do the same thing. This will run the entire render at least 2x faster as we won't need to run that pre-qurey, which, depending on the data, can be relatively expensive.

We won't have to change anything from the APIs perspective. Only internally to the heatmap mixin. Instead of generating a pre-query when the "auto" color domain mode is active, we'll instead update the BE vega to include the following:

      {
        "name": "xformtable",
        "source": "heatmap_queryheat",
        "transform": [
          {
            "type": "aggregate",
            "fields": ["color", "color", "color", "color"],
            "ops":    ["min", "max", "avg", "stddev"],
            "as":     ["minimum", "maximum", "mean", "deviation"]
          },
          {
            "type": "formula",
            "expr": "max(minimum, mean-2*deviation)",
            "as": "mincolor"
          },
          {
            "type": "formula",
            "expr": "min(maximum, mean+2*deviation)",
            "as": "maxcolor"
          }
        ]
      }

Document Existing Keyboard Shortcuts for Raster Charts

Existing map controls for pan & zoom via mousepad and key shortcuts aren't documented. From customer issue asking for how to zoom out without his mousepad I found them by accident. And also endorse it would be pretty helpful to "unzoom" to the original zoom level, as you can sometimes get lost on a big map.

Contextify Error (install node-gyp rebuild)

🪲

• Operating System and version: MacOS Sierra 10.12.6
• Browser and version: Chrome 61.0.3163.100
• Npm version: 5.5.1
• Node version: v8.2.1
• Steps to reproduce (including dashboard link): Clone repository and then run "npm install"
• Description of issue: (see below)

ISSUE DESCRIPTION

When attempting to build mapd-charting so I play around with it and attempt to build something, I get the errors seen below. Any advice on how to resolve this issue would be greatly appreciated.

Solutions I've tried:

Most of the comments from this link https://github.com/nodejs/node-gyp/issues/809#issuecomment-155629365

LOGS

> [email protected] install /Users/davidlewis/Cyber/WebDashboard/mapd-charting/node_modules/contextify
> node-gyp rebuild

xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance

xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance

  CXX(target) Release/obj.target/contextify/src/contextify.o
../src/contextify.cc:131:56: warning: 'NewInstance' is deprecated [-Wdeprecated-declarations]
        Local<Object> wrapper = Nan::New(constructor)->NewInstance();
                                                       ^
/Users/davidlewis/.node-gyp/8.2.1/include/node/v8.h:3674:3: note: 'NewInstance' has been explicitly marked deprecated here
  V8_DEPRECATED("Use maybe version", Local<Object> NewInstance() const);
  ^
/Users/davidlewis/.node-gyp/8.2.1/include/node/v8config.h:332:29: note: expanded from macro 'V8_DEPRECATED'
  declarator __attribute__((deprecated))
                            ^
../src/contextify.cc:150:16: error: no member named 'SetAccessCheckCallbacks' in 'v8::ObjectTemplate'
        otmpl->SetAccessCheckCallbacks(GlobalPropertyNamedAccessCheck,
        ~~~~~  ^
../src/contextify.cc:182:51: warning: 'GetRealNamedProperty' is deprecated [-Wdeprecated-declarations]
        Local<Value> rv = Nan::New(ctx->sandbox)->GetRealNamedProperty(property);
                                                  ^
/Users/davidlewis/.node-gyp/8.2.1/include/node/v8.h:3227:3: note: 'GetRealNamedProperty' has been explicitly marked deprecated here
  V8_DEPRECATED("Use maybe version",
  ^
/Users/davidlewis/.node-gyp/8.2.1/include/node/v8config.h:332:29: note: expanded from macro 'V8_DEPRECATED'
  declarator __attribute__((deprecated))
                            ^
../src/contextify.cc:209:38: warning: 'GetRealNamedProperty' is deprecated [-Wdeprecated-declarations]
        if (!Nan::New(ctx->sandbox)->GetRealNamedProperty(property).IsEmpty() ||
                                     ^
/Users/davidlewis/.node-gyp/8.2.1/include/node/v8.h:3227:3: note: 'GetRealNamedProperty' has been explicitly marked deprecated here
  V8_DEPRECATED("Use maybe version",
  ^
/Users/davidlewis/.node-gyp/8.2.1/include/node/v8config.h:332:29: note: expanded from macro 'V8_DEPRECATED'
  declarator __attribute__((deprecated))
                            ^
../src/contextify.cc:210:42: warning: 'GetRealNamedProperty' is deprecated [-Wdeprecated-declarations]
            !Nan::New(ctx->proxyGlobal)->GetRealNamedProperty(property).IsEmpty()) {
                                         ^
/Users/davidlewis/.node-gyp/8.2.1/include/node/v8.h:3227:3: note: 'GetRealNamedProperty' has been explicitly marked deprecated here
  V8_DEPRECATED("Use maybe version",
  ^
/Users/davidlewis/.node-gyp/8.2.1/include/node/v8config.h:332:29: note: expanded from macro 'V8_DEPRECATED'
  declarator __attribute__((deprecated))
                            ^
4 warnings and 1 error generated.
make: *** [Release/obj.target/contextify/src/contextify.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack     at emitTwo (events.js:125:13)
gyp ERR! stack     at ChildProcess.emit (events.js:213:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Darwin 16.7.0
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/davidlewis/Cyber/WebDashboard/mapd-charting/node_modules/contextify
gyp ERR! node -v v8.2.1
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok 
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/davidlewis/.npm/_logs/2017-10-31T16_14_40_881Z-debug.log

gitignore dist folder

add the dist folder with built javascript and css bundles to the project .gitignore

the thinking here is mapd-charting users who want to use the latest master are sophisticated enough to

yarn install
yarn build

on their own. other users should be able to use a released version of mapd-charting from the releases page or npm.

Point Raster Layer API as Vega-Lite Specification

{
  transform?: {
    sample?: true,
    limit?: number
  },
  encoding: "point",
  x: {
    type: "quantitative",
    field: "lon"
  },
  y: {
    type: "quantitative",
    field: "lat"
  },
  size: number | "auto" | {
    type: "quantitative",
    field: string,
    domain: [number, number],
    range: [number, number]
  },
  color: HexCode | {
    type: "density",
    range: Array<HexCode>
  } | {
    type: "quantitative" | "ordinal"
    field: string,
    domain: [number, number] | Array<string>,
    range: Array<HexCode>
  }
}

destroyChart() on Map/Scatter seems to be incomplete

• Operating System and version: Ubuntu 16.04
• Browser and version: Firefox 60.0.1
• Steps to reproduce:

1. Create a Map or Scatter, e.g. one of the pointMapChart + pointLayer in the examples
2. Destroy the chart with pointMapChart.destroyChart()
3. Re-create a brand new chart => Error related to chartBody being undefined, the new chart still displays properly

• Description of issue:
  It looks like destroying the chart does not clean up the chart registry and therefore the old chart is still attempted to be rendered (and fails because it was previoulsy destroyed). After some digging, it looks like a simple call to deregisterChart() in the destroyChart() method solves the problem:

  _chart.destroyChart = function() {
    deregisterChart(_chart, _chart.chartGroup())
    ...  

Also, I found out that if we switch between a Scatter and a Map (i-e: destroy the scatter, create a new map in the same <div> container), the X and Y axis of the scatter are now shown on the maps. Since the axis are elements in the container, adding the following to the destroyChart() seems to make things right:

  _chart.destroyChart = function() {
    deregisterChart(_chart, _chart.chartGroup())
    _chart
      .root()
      .attr("style", "")
      .attr("class", "")
      .html("")
    ...

I am not enough familiar with the chart rendering elements and lifecycle to know if this is actually a correct fix and whether it is the only calls to make. I hope it helps though.

How To implement only point map just like tweet map example!

Hello,
Thank you for creating such a wonderful platform.

I had a doubt regarding UI implementation of the point map as shown in the tweet map example https://mapd.github.io/mapd-charting/example/example2.html2

We want to show point map in our UI with node js backend as we don’t want to expose our DB credentials and I’m not able to find a proper way to implement point map.

I’d be really grateful if someone could guide me about it.

Raster Chart Popup Behavior Broken

https://github.com/mapd/mapd-charting/blob/develop/src/charts/raster-chart.js#L353
https://github.com/mapd/mapd-connector/blob/develop/src/mapd-con-es6.js#L905

The raster chart is using the method getResultRowForPixel incorrectly. The method has the signature (error, result) => but its being used like (result) =>.

This causes popups to be broken on the raster charts

enforce `cap` on rowChart as required property.

bug 🪲

• Operating System and version:
  OSX 10.12.5
• Browser and version:
  Chrome 59
• Steps to reproduce (including dashboard link):
  Create a rowChart without a cap but with an overridden order
• Description of issue:

dc.rowChart('#chart')
    .width(300)
    .height(300)
    .dimension(myDim)
    .cap(30) // if this is removed, the chart will no longer be ordered by 'key0'
    .group(chromGroup.order('key0'));

Table chart does not parse custom sql properly

🪲 If bug

• Operating System and version:
  OSX
• Browser and version:
  Chrome -- Version 59.0.3071.115
• Steps to reproduce (including dashboard link):

1. Create a dashboard
2. create as custom measure
3. custom measure should have "cast([columnname] as float)

• Description of issue:
  It does not render the data properly on table chart.