From @ilijapuaca:

SolidPolygonLayer

In data-column.js, I can see a mention of nextPosition and noting it uses vertexOffset to get that attribute. There’s also a mention of it in polygon-tessalator.js.

I assume that once tessellation is done on raw array coming from the data set, we’d get another array of vertices that we then need to iterate over. Instead of splitting this up “start” and “end” vertices like we do for line layer, we iterate over this array of vertices, using these offsets in order to pick out the right segments.

This would obviously require putting a few things in place that we currently don’t support, like offsets, which depending on how much work it is may not be something we want to do right now. Perhaps there’s a simpler way for us to format this data so that it can be used in the same way.

The other thing we’ll need support for is loading variable size arrays in order to parse that data set. That should hopefully be pretty straight-forward, as we can reuse most of the functionality that getVectorX has in row.h

Current implementation doesn't provide functionality provided by extruded and wireframe properties

Per #112, we should add a way to avoid spamming the log with messages being logged within the render loop, just like the JS counterpart does. From @ibgreen:

Just a set recording which strings have already been logged. Could also be a map<string, timestamp> and only log if say 1 second has passed.
I believe the log function has an option.once. This is set by default on log.warn and log.error (the api is slightly different).

Several layer attributes could be specified with constants, instead of having the data generated by accessors duplicated for each row in the data table

CI is slowed down by using the absolute latest vcpkg, this also introduces the possibility of breakage as dependencies are upgraded. We should consider just locking the vcpkg version and not pulling master.

deck.gl-native could leverage the extensive data format support in GDAL

see e.g https://gdal.org/doxygen/classOGRLayer.html

Tracker task that addresses refinement of AnimationLoop API additions made in #139.

• Revisit drawingOptions property of Deck
• Revisit the way appropriate AnimationLoop implementation is picked, and its initialization in general
• Revisit usage of MTKView as the view being passed doesn't necessarily need to be a MTKView since we only interact with the underlying CAMetalLayer

OpenGL Mathematics (GLM) is a header only C++ mathematics library for graphics software based on the OpenGL Shading Language (GLSL) specifications.

https://glm.g-truc.net/0.9.9/index.html

Basic Deck/Layer/View integration (no rendering)

Ensure we have the API to drive layer diffing

Environment (2 days)

• Setup CMakefile
• Setup test framework: https://github.com/google/googletest
• Setup Travis CI
• Setup cpplint
• Setup xcode project

math library (1 day)

• initial math library for 4x4 matrices, vector3 etc. (write our own but can change later).
• Rectangle

Viewport (10 days) - @isaacbrodsky

• Port viewport.js (Viewport base class)
• Port web-mercator-viewport.js (WebMercatorViewport)
• Port web-mercator-utils.js
• Port viewport-uniforms.js
• Port test cases

Viewport/View integration (5 days) @isaacbrodsky / @ibgreen

• Port view.js
• Port view-state.js
• Port view-manager.js
• Implement View.makeViewport
• Port test cases

JSON Support (10 days)

• Integrate jsoncpp (Json::Value)
• Implement basic Component/Props system to allow props to be set from JSON.
• Decide on how to handle properties (dynamic vs strongly typed).

Layer base class (5 days)

• Implement layer class and property diffing
• Test cases for layer
• Test cases for property diffing

Deck class (5 days)

• Scaffold Deck class.
• Scaffold LayerManager class.
• Implement Deck properties and property diffing.
• Integrate layers, push layers through property diffing.

Arrow Integration (15 days)

• Implement jsoncpp (Json::Value) to Arrow (Table) converter for inline tables.
• Change Layer.data prop to Arrow Table
• Make accessors to work on rows
• If not, create Row abstraction class for table? (Layer accessors need to work on "rows")
• Investigate if Arrow C++ Table has iteration facilities.
• Create test cases that show CSV files being parsed into Arrow Tables
• Create test cases that show JSON files being parsed into Arrow Tables
• Integrate arrow vcpkg module

C++ Math Expression Parser

• Select expression parser module
• Add expression support to JSON converter (@@= strings)
• Compatibility of expression syntax across JS/C++ (OK to change JS...)

Loaders

• Add support for loading data over the network

Rendering

• Dawn integration
• Investigate memory leaks

Out of scope

Interactivity (P2)

controllers (2 days)

deck.gl

API

• Implement ScatterplotLayer
• Implement SolidPolygonLayer

Arrow

• Add implicit type casting for Vector types in Row

Memory

• Revisit dangling accessor references in LineLayer and ScatterplotLayer
• Component API such as makeComponent should use smart pointers

luma.gl

API

• Add a way to render into a texture

GArrow

• Revisit Array API in order to align it with Arrow
• Specify buffer usage enum through metadata
• Add 64-bit float support by splitting double values

Shaders

• Revisit shader organization by adding a basic mechanism that simulates modules

Memory

• Investigate and fix memory leak in render loop

Misc

• Support constant values for attributes
• Retrieve device pixel ratio and use it when calculating viewport size
• Handle window resizing
• Render loop should not sleep for 16ms, but 16ms at most

loaders.gl

• Add loading over the network

Build

• Output bundle library that contains all the modules

Platform

• Fix gcc compilation on macOS
• Investigate rendering on Linux, seems to try to use Vulkan although it's disabled

Dependencies

• Update Dawn to a more recent revision

From #124:

That being said I don't think these look the way they're supposed to look, they seem to be squares instead of circles that I'm seeing on the web. Also, when zooming in very closely, they seem to stop scaling at some point. I'm seeing weird geometry in the geometry inspector (could just be a bug in the tool, it's pretty flimsy sometimes) although all of the values seem valid, no NaNs or anything.

Changing SMOOTH_EDGE_RADIUS in geometry.glsl.h to 0 seems to completely get rid of the issue. I am not clear on the specifics about smoothedge/smoothstep, I see that it's a way to interpolate values, but I am not sure why it'd cause results we're seeing when it has the same value in JS.

Currently throws a "not implemented" exception.

We should use doxygen triple-slash style comments and keywords in C++ (///, \brief, ... ).

A lot of the current code uses JSDoc style comments. See e.g. https://github.com/UnfoldedInc/deck.gl-native/blob/master/cpp/modules/deck.gl/core/src/lib/view-manager.cpp#L61

@ilijapuaca can provide information on what constructs to use.

We should fully implement the coverage test in CI and report coverage changes.

Possible providers:

• Coveralls
• ???

Layers currently always need to be redrawn. Property diffing should be put in place by using the same mechanism that JS counterpart uses

Property lists are currently being maintained manually. In order to make maintenance easier, and consolidate these across multiple languages, there should be a way to use a single source in order to generate their definitions

Integrate a parser for mathematical expressions

A good start is probably: https://github.com/ArashPartow/math-parser-benchmark-project

Background

deck.gl JSON styles support specifying expressions as strings:
. "getPosition": "@=[x,y,z]"

We need a C++ parser for expressions. Ideally it can return both:

• an evaluation function for row wise application,
• as well as an AST (abstract syntax tree) so that we can apply the expression on columns or on GPU.

We also need to align the syntax with the JS expression parser in the @deck.gl/json module, but we could potentially make changes on the JS side as well if it makes more sense.

The last commit I saw was 2020

Currently, all the floating point input data uses arrow::float32 type. In order to provide additional precision for attributes such as positions, support should be added for arrow::float64 type, where we'd split the data into "high" and "low" components, and map them accordingly

Due to the number of different types we currently support mapping for, and somewhat complex Arrow API, the current implementation provides a balance between conciseness and performance. In order to accommodate for extremely large datasets, the performance could be improved further by minimizing the number of copies that occur

Currently, indices for both attributes and uniforms are implicitly decided based on the order they're specified in:

Attributes

• Attributes specified in attributeSchema of Model::Options take up the first n slots, maintaining their relative order
• Attributes specified in instancedAttributeSchema of Model::Options take up the next m slots, maintaining their relative order

Uniforms

• Uniform bindings are deduced based on the order in Model::Options uniforms collection

It should be possible to specify these explicitly, through an API such as:

Model.setUniformLocationMap(std::map<std::string, int>);
Model.setAttributeLocationMap(std::map<std::string, int>);

or simply by providing a similar map as part of Model::Options. We could then have an API such as model->setUniformBuffer(std::string, buffer); that assigns to these bindings based on the named, rather than an index

We need to port the SolidPolygonLayer

Compare with e.g. LineLayer and ScatterplotLayer to see how they were ported.

For initial port, ignore tesselation, just port props, basic layer functions and shaders.

• Add ability for deck.gl-native C++ code to listen deckl.gl JSON style messages arriving over a WebSocket.
• A simple web socket server (could be Node.js based)
• Boost.Beast and Networking TS: https://www.youtube.com/watch?v=7FQwAjELMek

Parsing GeoJSON data requires additional work in order to add flexibility when it comes to nested types, and the way they are being extracted by using the utility Row class. This likely also includes adding the ability to parse struct-like data into appropriate Arrow types, which seems to be somewhat straight-forward, quick sample:

auto structArray = std::static_pointer_cast<arrow::StructArray>(this->_getChunk("geometry"));
auto coordinates = structArray->GetFieldByName("coordinates");
auto meta = this->_getListArrayMetadata(coordinates, this->_chunkRowIndex);

return this->_getNestedListData(meta.value(), defaultValue);

Initial emscripten support for compiling C++/dawn to JS/WebGPU

https://github.com/emscripten-core/emscripten/pull/10218/files

Arrow dependency may or may not be suitable for transpilation may be better to use ArrowJS.

Whenever drawing occurs, all the attributes are rebuilt regardless of whether they were updated or not. AttributeManager should implement individual attribute invalidation, which should be made use of in appropriate places

• Revisit matrix accessors
• Consolidate method naming
• Go through all the methods and make sure they’re implemented or commented out
• Consolidate functionality across Vector* and Matrix* classes where applicable
• Add unit tests for core functionality
• Think about adding error checking

AnimationLoop currently sleeps for 16ms in between each frame. The actual delay should be 16ms at most

We want to be able to have multiple language bindings in this repo, but all the C++ config files are in the root, they should be inside the cpp folder, e.g.

• CMakeLists.txt, cpplint config etc files are in the root folder.

Compare with Apache Arrow repo which as a similar, fully executed multi-language structure: https://github.com/apache/arrow/tree/master/cpp

• Also, the libdeckgl.a seems to be built in the root folder, we should create a directory for artifacts.

Currently, data can only be supplied from local resources. Add the ability to load an Arrow table through network, using a library such as Boost Beast

Currently, common shader code is concatenated to appropriate layer shaders. In order to make these reusable, and easier to use, a modular approach should be used just like in JS implementation of luma.gl

Majority of the current GArrow API is aligned with the API that Arrow provides. There are a few inconsistencies that were introduced in order to reduce the complexity of the API for the need of initial release.

Arrow uses builder-style API that in order to build up arrays, where ArrayData type plays a key role in accessing the array data, and abstracting the types away completely. For simplicity sake, current Array implementation takes the underlying data as a constructor argument