data2viz / data2viz

An Android, JavaFx, JS multiplatform datavisualization library with comprehensive DSL

Isomorphic dataviz

Data2viz is a data visualization toolbox for Kotlin Multiplatform.

You can pick what you need in the different modules and use them independently in the following environments: Android, JavaScript(IR and Legacy), and JavaFX (iOS to come).

Your code produces the same results and rendering on each platform.

A lot of algorithms come from d3js modules.

core

This module exposes some basics elements like Circle, Rect, Path, and also some parts of DSL (percent, angles, etc.).

color

This module provides algorithms to create, modify, and convert colors through differents spaces (RGB, HCL, HSL, Lab). It also manage linear and radial color gradients. You can easily use it outside of data2viz.

shape

Algorithms to generate paths: symbols, curves, ...

scale

Different ways to convert data to sizes.

interpolate

Functions to perform interpolation of various elements (numbers, colors, points, curves)

random

Algorithms to generate random data.

axis

Helper module to create axis for charts, using scales.

delaunay

This module is a high performant algorithm to compute the Voronoi diagram. You should use it in many use cases to find the closest point of interest to the pointer position.

ease

This module contains a collection of easing functions to manage acceleration inside animations.

dsv

Minimal way of parsing CSV, TSV files using Kotlin.

force

Use physics to animate your visualizations.

format

Different ways of formating numbers and currencies.

time

Some multiplatform classes to manage time and dates in visualizations.

time-format

Extend formatting to time and dates.

quadtree

Separation of space through the quadtree algorithm.

voronoi

Fortune's algorithm implementation of the Voronoi diagram. You should probably use Delaunay instead.

geo

A collection of mechanisms and functions to project GeoJson elements (points, lines, polygons) on a screen using different implementations of projections.

hexbin

A useful way of aggregating data using hexagons. You can represent data through the fill color or the area.

sankey

A generator for sankey charts.

hierarchy

A generator for hierarchical charts.

tests

An internal module used to simplify multiplatform testing with a higher DSL.

timer

A multiplatform implementation to manage animations through shared frames.

viz

Multiplatorm API and implementation of rendering and events management.

Data2viz allows you to develop data visualizations through a fully typed DSL. It simplifies the creation of complex visualizations via the IDE’s context-based suggestions.

Where should I start?

Documentation

All data2viz documentation is located in a distinct documentation project. You should start there and follow the first JavaFX tutorial.

Data2viz Playground

You can also play with the API without installing anything. Data2viz playground is a website where you can browse existing sample, modify them online and immediately see the result.

Using in your projects

Note that the library is experimental, and the API is subject to change.

The library is published to data2viz space repository.

Gradle

• In multiplatform projects, add a dependency to the commonMain source set dependencies

kotlin {
    sourceSets {
        commonMain {
             dependencies {
                 implementation("io.data2viz.d2v:d2v-axis:0.10.7")
                 implementation("io.data2viz.d2v:d2v-chord:0.10.7")
                 implementation("io.data2viz.d2v:d2v-color:0.10.7")
                 implementation("io.data2viz.d2v:d2v-contour:0.10.7")
                 implementation("io.data2viz.d2v:d2v-delaunay:0.10.7")
                 implementation("io.data2viz.d2v:d2v-dsv:0.10.7")
                 implementation("io.data2viz.d2v:d2v-ease:0.10.7")
                 implementation("io.data2viz.d2v:d2v-force:0.10.7")
                 implementation("io.data2viz.d2v:d2v-format:0.10.7")
                 implementation("io.data2viz.d2v:d2v-geo:0.10.7")
                 implementation("io.data2viz.d2v:d2v-hexbin:0.10.7")
                 implementation("io.data2viz.d2v:d2v-hierarchy:0.10.7")
                 implementation("io.data2viz.d2v:d2v-quadtree:0.10.7")
                 implementation("io.data2viz.d2v:d2v-random:0.10.7")
                 implementation("io.data2viz.d2v:d2v-scale:0.10.7")
                 implementation("io.data2viz.d2v:d2v-shape:0.10.7")
                 implementation("io.data2viz.d2v:d2v-tile:0.10.7")
                 implementation("io.data2viz.d2v:d2v-time:0.10.7")
                 implementation("io.data2viz.d2v:d2v-timer:0.10.7")
                 implementation("io.data2viz.d2v:d2v-viz:0.10.7")
             }
        }
    }
}

• To use the library in a single-platform project, add a dependency to the dependencies block.

dependencies {
    implementation("io.data2viz.d2v:d2v-axis:0.10.7")
    implementation("io.data2viz.d2v:d2v-chord:0.10.7")
    implementation("io.data2viz.d2v:d2v-color:0.10.7")
    implementation("io.data2viz.d2v:d2v-contour:0.10.7")
    implementation("io.data2viz.d2v:d2v-delaunay:0.10.7")
    implementation("io.data2viz.d2v:d2v-dsv:0.10.7")
    implementation("io.data2viz.d2v:d2v-ease:0.10.7")
    implementation("io.data2viz.d2v:d2v-force:0.10.7")
    implementation("io.data2viz.d2v:d2v-format:0.10.7")
    implementation("io.data2viz.d2v:d2v-geo:0.10.7")
    implementation("io.data2viz.d2v:d2v-hexbin:0.10.7")
    implementation("io.data2viz.d2v:d2v-hierarchy:0.10.7")
    implementation("io.data2viz.d2v:d2v-quadtree:0.10.7")
    implementation("io.data2viz.d2v:d2v-random:0.10.7")
    implementation("io.data2viz.d2v:d2v-scale:0.10.7")
    implementation("io.data2viz.d2v:d2v-shape:0.10.7")
    implementation("io.data2viz.d2v:d2v-tile:0.10.7")
    implementation("io.data2viz.d2v:d2v-time:0.10.7")
    implementation("io.data2viz.d2v:d2v-timer:0.10.7")
    implementation("io.data2viz.d2v:d2v-viz:0.10.7")
}

Current status and roadmap

APIs are mostly stabilized now, but there may still be some breaking changes before v1.0.

Inspirations

• d3js: a lot of modules and algorithms come from d3js.
• paperjs: another source of inspiration for viz hierarchy and simple API.
• chromajs: smart library for managing colors easily.
• delaunator: a really fast JavaScript library for Delaunay triangulation of 2D points.
• kotlinx.html: isomorphic html rendering.
• kotlintest: nice DSL for testing (partly ported in test).

data2viz's Issues

Canvas does not clean when applying a transformation

when applying a transform (rotate+translate) the canvas is partially cleaned see this example.

import io.data2viz.viz.*
import io.data2viz.color.*
import io.data2viz.scale.scales

fun main(args:Array<String>){
    viz {
        var angle = .0
        val g = group {
            rect {
                width = 50.0
                height = 50.0
                fill = colors.blue
        	}
        }             
        onFrame {
            angle += .1
            g.apply {
             	transform {
                	rotate(angle)
                	translate(.1, .1)
            	}
            }
        }
    }.bindRendererOnNewCanvas()
}

Tile Module : Harmonize tilesCount / zoom / tileSize behavior

tilesCount shouldbe an Int
the link between tilescount and zoom should be easier to understand

Can't try this out with Gradle

I'd like to try it out, but adding

compile 'io.data2viz:d2v-axis-common:0.3.1'
to build.gradle just doesn't work.

It can't download certain pom's from bintray.com, which is solved by saving security certificate in the browser and then importing it on the Security tab of the Java Control Panel:

Then there is another authentication issue downloading d2v-axis-common-0.3.1.pom from https://maven.oracle.com, which I had created an Oracle account for and added the credentials section to the build.gradle:

    maven {
         url "https://maven.oracle.com"
         credentials {
             username "****"
             password "****"
         }
     }

And now I get:

 Could not resolve all files for configuration ':compileClasspath'.
Caused by: org.gradle.internal.resolve.ModuleVersionResolveException: Could not resolve io.data2viz:d2v-axis-common:0.3.1.
Required by:
    project :
Caused by: org.gradle.internal.resolve.ModuleVersionResolveException: Could not resolve io.data2viz:d2v-axis-common:0.3.1.
Caused by: org.gradle.api.resources.ResourceException: Could not get resource 'https://maven.oracle.com/io/data2viz/d2v-axis-common/0.3.1/d2v-axis-common-0.3.1.pom'.
Caused by: org.gradle.internal.resource.transport.http.HttpRequestException: Could not GET 'https://maven.oracle.com/io/data2viz/d2v-axis-common/0.3.1/d2v-axis-common-0.3.1.pom'.
Caused by: org.apache.http.client.ClientProtocolException: null
Caused by: org.apache.http.client.CircularRedirectException: Circular redirect to 'https://www.oracle.com/content/secure/maven/content/io/data2viz/d2v-axis-common/0.3.1/d2v-axis-common-0.3.1.pom'</i>

Am I doing something wrong?

viz-js: manage HDPi screen

Move to new multiplatform model

See https://discuss.kotlinlang.org/t/kotlin-1-3-m2-new-multiplatform-projects-model/9264

This new model should clean up the modules.

The current version creates a gradle project for every platform even if there is no specific platform code. After the migration we should keep only one project per module.

Need kotlin 1.3 #35

TRANSITION: Color to LinearGradient

Implement transition from color to LinearGradient using all points of LinearGradient and the same color.

Move to kotlin 1.3

A lot of improvements:

• better performances,
• inline classes (Angle?) #38,
• multiplatform model #36

GEO : mercator projection geoPath issue

check with world-30-percent.geojson file, mercator projection fails and send an "infinity"

data2viz js not working in IExplorer - (append?)

I got a graph to work in Chrome, Firefox, Edge, but not in Iexplorer. I can see the online JS examples are also not working in IExplorer.

I think it may be because IExplorer does not accept the append method on dom elements, these calls could be probably replaced with "appendChild".

Tile Module : clean tiles out of extent

there should be an option or a described code to clean tiles out of scope for example when dragging large tilemaps around (creating tiles way out of screen)

LinearGradient: complete implementation

Current implementation is more a prototype than the definitive version of it. It generate only a linear gradient using points. It should handle direction.

Also, it can be interesting to provide nice typesafe builders.

Manage precisely vertical alignment of text on Android

see https://stackoverflow.com/questions/27631736/meaning-of-top-ascent-baseline-descent-bottom-and-leading-in-androids-font
and https://stackoverflow.com/a/42091739/231174

Internal logging API

In order to help the development, it is sometimes easier to have some logs (timer issues, ...).

This logging system shouldn't have any cost in terms of performance. Existing multiplatform solutions seems to be overkill.

Prerelase artifacts?

I know it's prerelaese at the moment, but is there any chance you provide public artifacts for prerelaese builds?

The project looks really promising, pls keep going😀

All scales initialization should be available through lambda

OK on linear

    var scale = scales.continuous.linear {
        domain = listOf(1.0, 200.0)
        range = listOf(0.0, 1.0)
    }

Not possible on pow:

    var scale = scales.continuous.pow(3.0) {
        domain = listOf(1.0, 200.0)
        range = listOf(0.0, 1.0)
    }

Current workaround:

    var scale = scales.continuous.pow(3.0).apply {
        domain = listOf(1.0, 200.0)
        range = listOf(0.0, 1.0)
    }

GEO : geoarea of a zero area polygon doesn't return the correct result in JS (OK in JVM)

investigate GeoAreaTests.geoarea_of_a_zero_area_polygon_LEGACY()

may be resolved by #39

Source and Test code in the same directory

Currently, inside each module, the source code and the test code are in same directory. This is a temporary workaround due to this bug : https://youtrack.jetbrains.com/issue/KT-16979

Test code will be separated as soon as KT-16979 will be released.

curves : add lineRadial & curveRadial

Evaluate moving Circle and Rectangle to Path

Path can manage the rectangle and circle elements. Does that make sense to give access to these concepts with two different ways?

Is there any performance difference?

SCALE : Should rename all "ranges" of Scale to another keyword as "range" is a Kotlin concept

In scales, we currently use the concepts of Domain and Range. The idea is to map the scaling as a transformation from Domain objects to Range of values.

TIME : timeWeek : selected timeWeek returns wrong day

sunday returns mondays
monday returns tuesdays...

when fixed, check for test still open in scaleTime : time_ticks_count_can_generate_1_week_ticks_LEGACY()

Give typed access to configuration depending on the platform

A configuration may have some generic parameters that apply on all platform and some that only make sense on a specific platform. It should be possible to provide access to these parameters using the same API but having different params depending on the platform.

viz {
    configure { // on common code
          resizable = false
    }
}

viz {
    configure { // on android
          minimumBatteryMinimumForAnimations = 40.pct
    }
}

Deploy js lib to npm

To facilitate the use of the library by client application (demo, samples, documentation) it should be available on npm.

Switch builds from groovy to kotlin

Now that kotlin DSL documentation exists, it should be more easy to do.

https://docs.gradle.org/nightly/userguide/java_plugin.html

Move RenderingTestSupport outside of main sources

This class contains the common renderings tests for js, jfx and android.

Before the creation of rendering-app, it was in test sources. But the integration of this new app to do the tests seems impossible with a dependency on test sources. As a temporairy workaround this class has been moved in main sources.

Provide a Style type to group all styling properties

See http://paperjs.org/reference/style/

It is an interesting way of grouping styling properties. The style can be defined at different level of the hierarchy: on a node or on on of its parents. If a style is not defined at a lower level, its parent one could be used (this is not included in this issue)

Implement all rendering tests of Path.arc

Due to the differences in arc API on each platform, all cases should be tested on all platforms with a comparison of the rendered images.

start; end; counterclocwise;
0; 0.25PI; false
0; 0.25PI; true
0; 2.0PI; true
0; 2.0PI; false
.25PI; 2.25PI; true
.25PI; 2.25PI; false
...

TILE MODULE : add wraping

tileLayout.wrap = true allows to repeat tiles in longitude

Remove Scale class in scale.kt

This was an early scaling method, check what could be kept in API and remove the class.

AXIS : tickvalues should return the computed tick values of underlying scale

actually tickvalues can just be setted to override computed tick values, and isn't used if setted to empty (default).

actually we should have something like get = setted value (if not empty) or computed value

Deploy to JCenter and/or Maven central

Currently artefact are available by adding the url of the repo:

repositories {
    maven { url "http://dl.bintray.com/data2viz/data2viz" }
}

It would be better to just indicate jcenter or maven central.

TRANSITION: combine transitions

    rect {
        width = 10.0
        height = width
        fill = colors.grey

        transitionTo {
            x = 100.0
        } thenTransitionTo {
            y = 100.0
        } thenTransitionTo {
            x = 0.0
        } thenTransitionTo {
            y = 0.0
        }
    }

Move all shape tests in common module + image rendering + image comparison scripts

Due to the new rendering engine, shape tests (JS part) are not working anymore, move them to common and put them in the new test pipeline.

Examples from README not working

The README of your repository contains two links to examples:
http://data2viz.io/dist/chart.html
http://data2viz.io/dist/svgperfs.html
both are not working for me neither in Firefox nor in Chrome.

SHAPE: Stack & Pie Layout

Simplify the stack & pie layout use, maybe using objects that are more easy to use/understand.

Provide default style with a parent/child mechanism

A default style should be provided and defined in a lambda:

viz {
    defaultStyle {
        fill = colors.red
        stroke = colors.black
        strokeWidth = 2.0
    }
}

Viz element should not use the style property (which should be internal) but instead access to style properties through direct getters and setters. It would allow to avoid the creation of a Style object for the element if only getters are called. The Style object would be created for the node only in case of setter calls. This convention would allow to reuse the same Paint instance for a group of elements sharing the same style.

Naming conventions

Why of all the class and singleton names these particular ones should start with a lowercase letter?

scales.continuous.linear { }

instead of

Scales.Continuous.linear { }

This strikes me as a very odd and arbitrary decision.

Angle as inline class

Angle appears to be a nice target for an inline class. It should provide a much more interesting API with some abstractions and no performance cost.

It should also simplify some algorithms like the various implementation of Path.arc.

API:
Number.deg
Number.rad
Angle.cos
Angle.sin
...
angle + angle
angle * Number
angle / Number

SCALE : Stick to domain + range in Scale or create a strict DomainToRange pair ?

Make travis works with android, JavaFx, and Js

Travis CI fails on android configuration:

* What went wrong:
A problem occurred evaluating project ':app'.
> The SDK directory '/Users/gaetan/Library/Android/sdk' does not exist.

See https://stackoverflow.com/questions/27644586/how-to-set-up-travis-ci-with-multiple-languages
and https://docs.travis-ci.com/user/languages/android/

Transition of transformation

Allow to use transformations in transitions:

  rect {
       x = 10.0
       y = 10.0
       width = 10.0
       height = 10.0

       transition {
           transform {
               translate(x = 10.0 * index, y = 20.0)
           }
       }
   }

Integrate platform rendering tests on the project

The prototype of testing by pixel comparison should be integrated in the project.

The javascript version is the target (the API sticks to it). Each platform project should compare its image generation with target.

On JS, use of puppeeter to copy the generated image from canvas.
On Android, save the file in the emulator and the pull it with adb.

Performance of Quadtree/Forces

Check the performance difference between d3js and data2viz (/2.5).

GEO : port "adder.js" to add more precisions to our calculation (for JS part)

and check if geoAreaTests.geoarea_of_a_tiny_polygon_LEGACY() is OK with .0 value (actually report a 4.890516e-13 value....)

GROUP element may store shared values like stroke, fill... for its inner elements

in svg a allows to pass the color parameters to its inner elements that can use it.
it's an easy way to skin multiple elements.

does this behavior should be allowed in a multiplatform library ?

Manage visibility of nodes

Provide a simple way of rendering or not elements. The property is accessible on each node of a hierarchy, from layers to leaf nodes.

If visible is set to false the node and its children are not rendered.

Target is updated two times

The original code is changing the speed on the source as well

https://github.com/d3/d3-force/blob/84d033544347dc2aa3c9e9152c0e91c2eb5e0303/src/link.js#L41-L44

data2viz/force/d2v-force-common/src/main/kotlin/io/data2viz/force/ForceLink.kt

Line 111 in 3432f93

target.velocity -= Speed(x * b, y * b)

Remove DCE of example for fast builds

It takes time during development workflow. Should only be launched for bundle task.

RadialGradient

Quite similar mechanism as LinearGradient

AREA : change API and define default x1/y1 instead of null (when setting x/y)

area creation should be more easy to understand and have a default behavior.
maybe x and y should be var instead of fun to be more easy to understand