The umbrella from juliocastilla

About

"A collection of functional programming libraries that can be composed together. Unlike a framework, thi.ng is a suite of instruments and you (the user) must be the composer of. Geared towards versatility, not any specific type of music." — @loganpowell via Twitter

Please visit thi.ng for additional information & topic based search of packages relevant to your use cases...

This mono-repository is home to 172+ individual TypeScript/ES6 projects (plus 115+ examples), a wide and jointly developed collection of packages, tools and general building blocks for (non-exhaustive list of topics):

Functional programming (ES6 iterators/generators, composition, memoization, transducers, multi-methods)
Reactive programming, stream / transducer based dataflow graphs / pipelines
Data structures & data transformations for wide range of use cases (maps, sets, heaps, queues, graphs etc.)
2D geometry generation, shape primitives, math, manipulation, intersections, conversions & visualizations
Canvas abstractions, pixel buffers & SVG serialization/conversion
Vector, matrix (dense & sparse), ECS implementations with optional support for strided memory layouts
Semi-declarative WebGL 1/2 abstraction layer
DSL for shader functions defined in TypeScript and cross-compilation to GLSL, JS, VEX etc.
Value-based equivalence
DSP building blocks: oscillators, noise generators, filters, 1D FFT/IFFT, muxers, rate converters
Immutable data handling, state containers, transacted state updates, Undo-Redo history
Data driven UI component toolkits (DOM-based, canvas-based, immediate-mode, multiple approaches...)
Multi-format, multi-channel pixel buffers (int & float based), conversions, dithering, Porter-Duff alpha-blending operators
Color space/format conversions, matrix based color manipulation, gradient generators, color palettes
Date-time abstraction, relative dates, iterators, formatters, math
WebWorker workflow abstractions
PEG-style functional parser combinators w/ (optional) custom grammar definition language
Forth-style pointfree DSL for functional composition and DSL development/extension
S-expression parser & runtime (interpreter) infrastructure for custom DSL creation
Low-level tooling for binary data, shared memory/WASM/WebGL interop
SIMD batch-processing of vector data
WebAssembly bridge API & data structure bindings code generators for multiple target languages
Various interpolations, math helpers, automatic differentiation (Dual numbers)
etc. (see package overview below)

This project is NOT a framework, provides no turn-key, one-size-fits-all approach and instead encourages a mix & match philosophy for various key aspects of application design (in & outside the browser). Most customization points only expect certain interfaces/type signatures rather than concrete implementations.

All packages:

are versioned independently
distributed as ESM modules (ES2020 syntax) with export maps, TypeScript typings & change logs
highly modular with largely only single function / class (only closely related functions) per file to help w/ selective imports and tree shaking
provide re-exports of all their publics for full library imports
have either none or only @thi.ng internal runtime dependencies (w/ very few exceptions!)
declare public interfaces, enums & types in an api.ts and/or constants.ts file (larger packages only)
have auto-generated online documentation at docs.thi.ng
licensed under Apache Software License 2.0

Getting started

The sheer number and varied nature & purpose of these packages makes it impossible to provide traditional "Getting started" tutorials. To compensate, this repo provides a large number of example projects, detailed readmes (at the very least for all the larger and/or more important packages) and smaller usage examples in the docstrings.

Examples & Showcase

There's a steadily growing number (115+) of standalone examples (different complexities, often combining functionality from several packages) in the examples directory.

Example screenshots	(small selection)

awesome.thi.ng

Due to other priorities still very much in its infancy & planning stage, but please help to document your own usage of these packages by contributing project information to the awesome.thi.ng repo, which will be used to build a showcase site... Thank you!

Generative art projects

Several generative art projects by Karsten Schmidt on fx(hash) have been created exclusively with libraries from this collection.

De/Frag series

Quasiflock	C-SCAPE	ASCII-SCAPE

Blog posts

How to UI in 2018
"Of umbrellas, transducers, reactive streams & mushrooms" (ongoing series):

Community, contributing, getting help

Join our little community on our Discord server or get in touch via Twitter or the issue tracker. If you'd like to contribute, please first read this document.

In general, we welcome contributions of all kinds (docs, examples, bug fixes, feature requests, financial contributions etc.). You can find a fairly detailed overview for contributors here: CONTRIBUTING.md.

Note: The default branch for this repo is develop and all PRs should be created based on this branch. This too means, the README files on this branch might refer to yet-unreleased features or packages. Please use the main branch for viewing the most recently released version(s)!.

Also please be sure to check the wiki for other project-wide information, tidbits, useful snippets etc.

Projects

Important changes (10/2021)

All packages are now only published in ESM format (ES2020 syntax) with TypeScript typings. You can find more details & rationale in issue #315.

As a result (and opportunity), many packages have undergone internal restructuring, some have been split up. All are now fully supporting deep-imports (based on declared export maps), leading to drastically smaller userland bundle sizes.

Previously we also created & published CommonJS & UMD formats, but deemed them an unnecessary burden & hindrance, since all important JS tooling/platforms do support ES modules by now.

Latest updates

As of: 2022-12-06

Status	Project	Version	Changelog
	`@thi.ng/axidraw`		changelog
	`@thi.ng/leb128`		changelog
	`@thi.ng/wasm-api`		changelog
	`@thi.ng/wasm-api-bindgen`		changelog
	`@thi.ng/wasm-api-dom`		changelog
	`@thi.ng/wasm-api-schedule`		changelog

Fundamentals

Project	Changelog	Description
`@thi.ng/args`	changelog	Declarative & functional CLI arg parsing & coercions
`@thi.ng/api`	changelog	Common types, decorators, mixins
`@thi.ng/bench`	changelog	Basic benchmarking helpers
`@thi.ng/checks`	changelog	Type & value checks
`@thi.ng/compare`	changelog	Comparators
`@thi.ng/compose`	changelog	Functional composition helpers
`@thi.ng/date`	changelog	Date/time iterators, formatters, rounding
`@thi.ng/defmulti`	changelog	Dynamic multiple dispatch
`@thi.ng/distance`	changelog	n-D distance metrics & K-nearest neighborhoods
`@thi.ng/equiv`	changelog	Deep value equivalence checking
`@thi.ng/errors`	changelog	Custom error types
`@thi.ng/expose`	changelog	Conditional global variable exposition
`@thi.ng/hex`	changelog	Hex value formatters for U4-64 words
`@thi.ng/logger`	changelog	Basis infrastructure for arbitrary logging
`@thi.ng/memoize`	changelog	Function memoization w/ customizable caching
`@thi.ng/oquery`	changelog	Pattern based query engine for JS objects
`@thi.ng/parse`	changelog	Parser combinators & AST generator/transformer
`@thi.ng/paths`	changelog	Immutable nested object accessors
`@thi.ng/strings`	changelog	Higher-order string formatting utils
`@thi.ng/system`	changelog	Minimal life cycle container for stateful app components
`@thi.ng/testament`	changelog	Minimal test runner

Maths

Project	Changelog	Description
`@thi.ng/dual-algebra`	changelog	Dual number algebra / automatic differentiation
`@thi.ng/dsp`	changelog	DSP utils, composable signal gens/processors
`@thi.ng/fuzzy`	changelog	Fuzzy logic primitives & rule inference engine
`@thi.ng/intervals`	changelog	Open/closed intervals, queries, set ops
`@thi.ng/math`	changelog	Assorted common math functions & utilities
`@thi.ng/matrices`	changelog	Matrix operations
`@thi.ng/sparse`	changelog	Sparse matrix & vector impls
`@thi.ng/vectors`	changelog	Fixed & arbitrary-length vector ops

Randomness

Project	Changelog	Description
`@thi.ng/colored-noise`	changelog	1D colored noise generators
`@thi.ng/ksuid`	changelog	K-sortable unique identifiers, binary & base-N encoded
`@thi.ng/lowdisc`	changelog	n-D Low discrepancy sequence generators
`@thi.ng/random`	changelog	Seedable PRNG implementations, distributions & utilities
`@thi.ng/random-fxhash`	changelog	PRNG impl & utilities for fxhash projects

File / file format / hardware support

Project	Changelog	Description
`@thi.ng/axidraw`	changelog	Minimal, declarative AxiDraw plotter controller
`@thi.ng/bencode`	changelog	Bencode binary format encoding
`@thi.ng/csv`	changelog	Customizable CSV parser/object mapper
`@thi.ng/dot`	changelog	Graphviz DOM & export
`@thi.ng/dsp-io-wav`	changelog	WAV file format exporter
`@thi.ng/file-io`	changelog	Assorted file I/O utils for NodeJS
`@thi.ng/geom-io-obj`	changelog	Wavefront OBJ model parser
`@thi.ng/hiccup-css`	changelog	CSS from nested JS data structures
`@thi.ng/hiccup-html`	changelog	Type-checked HTML5 element wrappers for hiccup
`@thi.ng/hiccup-markdown`	changelog	Hiccup-to-Markdown serialization
`@thi.ng/hiccup-svg`	changelog	hiccup based SVG vocab
`@thi.ng/iges`	changelog	IGES format geometry serialization
`@thi.ng/markdown-table`	changelog	Markdown table generator / formatter
`@thi.ng/mime`	changelog	File extension to MIME type mappings
`@thi.ng/pixel-io-netpbm`	changelog	1/8/16/24bit NetPBM image format reader/writer
`@thi.ng/prefixes`	changelog	Linked Data, RDF & xmlns prefixes/URLs
`@thi.ng/sax`	changelog	SAX-like XML parser / transducer
`@thi.ng/tangle`	changelog	Literate programming utilities

Iterator, stream & sequence processing

Project	Changelog	Description
`@thi.ng/csp`	changelog	Channel based async ops
`@thi.ng/fsm`	changelog	FSM / parser primitives
`@thi.ng/grid-iterators`	changelog	2D grid iterator strategies
`@thi.ng/iterators`	changelog	ES6 generators / iterators
`@thi.ng/seq`	changelog	Lisp/Clojure-style sequence abstraction
`@thi.ng/transducers`	changelog	Composable data transformations
`@thi.ng/transducers-binary`	changelog	Binary data related transducers
`@thi.ng/transducers-fsm`	changelog	Finite state transducer
`@thi.ng/transducers-hdom`	changelog	Transducer based hdom UI updates
`@thi.ng/transducers-patch`	changelog	Patch-based, array & object editing
`@thi.ng/transducers-stats`	changelog	Technical / statistical analysis

Reactive programming

Project	Changelog	Description
`@thi.ng/rstream`	changelog	Push-based, reactive event stream primitves
`@thi.ng/rstream-csp`	changelog	Adapter bridge CSP -> rstream
`@thi.ng/rstream-dot`	changelog	Graphviz visualization of rstream topologies
`@thi.ng/rstream-gestures`	changelog	Mouse & touch event stream abstraction
`@thi.ng/rstream-graph`	changelog	Declarative dataflow graph construction
`@thi.ng/rstream-log`	changelog	Hierarchical structured data logging
`@thi.ng/rstream-log-file`	changelog	Log-file output handler
`@thi.ng/rstream-query`	changelog	Triple store & query engine

Algorithms & data structures

Project	Changelog	Description
`@thi.ng/adjacency`	changelog	Adjacency matrices & graph algorithms
`@thi.ng/arrays`	changelog	Array utilities
`@thi.ng/associative`	changelog	Alt Set & Map implementations
`@thi.ng/atom`	changelog	Immutable value wrappers, views, history
`@thi.ng/bitfield`	changelog	1D/2D bit field implementations
`@thi.ng/cache`	changelog	In-memory caches / strategies
`@thi.ng/cellular`	changelog	Highly configurable 1D Cellular automata
`@thi.ng/dcons`	changelog	Doubly-linked list
`@thi.ng/diff`	changelog	Array & object diffing
`@thi.ng/dgraph`	changelog	Dependency graph
`@thi.ng/ecs`	changelog	Entity-Component System
`@thi.ng/egf`	changelog	Extensible Graph Format
`@thi.ng/gp`	changelog	Genetic programming helpers / AST gen
`@thi.ng/heaps`	changelog	Binary & d-ary heap impls
`@thi.ng/idgen`	changelog	Versioned ID generation / free-list
`@thi.ng/k-means`	changelog	K-means clustering of n-D data
`@thi.ng/ramp`	changelog	Parametric, interpolated lookup tables
`@thi.ng/quad-edge`	changelog	Quad-edge, dual-graph data structure
`@thi.ng/resolve-map`	changelog	DAG computations & value resolution
`@thi.ng/vclock`	changelog	Vector clock functions / comparators
`@thi.ng/zipper`	changelog	Immutable tree editing / navigation

Frontend / UI

Project	Changelog	Description
`@thi.ng/adapt-dpi`	changelog	HDPI canvas adaptation / styling util
`@thi.ng/dl-asset`	changelog	Asset/canvas/file download helpers
`@thi.ng/hdiff`	changelog	String diffing w/ hiccup output (includes CLI)
`@thi.ng/hdom`	changelog	Hiccup based VDOM & diffing
`@thi.ng/hdom-canvas`	changelog	hdom adapter for hiccup-canvas
`@thi.ng/hdom-components`	changelog	hdom based UI components
`@thi.ng/hdom-mock`	changelog	hdom mock implementation (testing / prototyping)
`@thi.ng/hiccup`	changelog	S-expression based HTML/XML serialization
`@thi.ng/hiccup-canvas`	changelog	hiccup interpreter for canvas api
`@thi.ng/hiccup-carbon-icons`	changelog	IBM Carbon icons in hiccup format
`@thi.ng/imgui`	changelog	Immediate mode GUI
`@thi.ng/interceptors`	changelog	Composable event handlers & processor
`@thi.ng/rdom`	changelog	Reactive, diff-less, async UI components
`@thi.ng/rdom-canvas`	changelog	rdom component wrapper for thi.ng/hiccup-canvas
`@thi.ng/rdom-components`	changelog	Unstyled, customizable component collection
`@thi.ng/router`	changelog	Customizable browser & non-browser router
`@thi.ng/text-canvas`	changelog	Text-mode canvas, drawing, tables, charts
`@thi.ng/text-format`	changelog	Color text formatting w/ ANSI & HTML presets

Geometry, image & visualization

Project	Changelog	Description
`@thi.ng/color`	changelog	Color conversions, gradients
`@thi.ng/color-palettes`	changelog	Collection of image-based color palettes
`@thi.ng/dgraph-dot`	changelog	Dependency graph -> Graphviz
`@thi.ng/distance-transform`	changelog	Image based distance field generation
`@thi.ng/fuzzy-viz`	changelog	Visualization, instrumentation for @thi.ng/fuzzy
`@thi.ng/geom`	changelog	2D only geometry types & ops
`@thi.ng/geom-accel`	changelog	Spatial indexing data structures
`@thi.ng/geom-api`	changelog	Shared types & interfaces
`@thi.ng/geom-arc`	changelog	2D elliptic arc utils
`@thi.ng/geom-clip-line`	changelog	2D line clipping
`@thi.ng/geom-clip-poly`	changelog	2D convex polygon clipping
`@thi.ng/geom-closest-point`	changelog	Closest point helpers
`@thi.ng/geom-fuzz`	changelog	Fuzzy 2D shape drawing / filling
`@thi.ng/geom-hull`	changelog	2D convex hull (Graham scan)
`@thi.ng/geom-isec`	changelog	Point & shape intersection tests
`@thi.ng/geom-isoline`	changelog	2D contour line extraction
`@thi.ng/geom-poly-utils`	changelog	2D polygon helpers
`@thi.ng/geom-resample`	changelog	nD polyline / curve resampling
`@thi.ng/geom-sdf`	changelog	2D SDF creation, conversions, operators, utilities
`@thi.ng/geom-splines`	changelog	nD cubic / quadratic spline ops
`@thi.ng/geom-subdiv-curve`	changelog	nD iterative subdivision curves
`@thi.ng/geom-tessellate`	changelog	nD convex polygon tessellators
`@thi.ng/geom-voronoi`	changelog	2D iterative delaunay/voronoi
`@thi.ng/lsys`	changelog	Extensible L-System architecture
`@thi.ng/pixel`	changelog	Multi-format pixel buffers
`@thi.ng/pixel-dither`	changelog	Image dithering w/ various algorithm presets
`@thi.ng/poisson`	changelog	nD Poisson disk sampling
`@thi.ng/porter-duff`	changelog	Alpha blending / compositing ops
`@thi.ng/rasterize`	changelog	Shape drawing, filling & rasterization
`@thi.ng/scenegraph`	changelog	Extensible 2D/3D scenegraph
`@thi.ng/simd`	changelog	WebAssembly SIMD vector batch processing
`@thi.ng/viz`	changelog	Declarative & functional data visualization toolkit

WebGL / GPGPU

Project	Changelog	Description
`@thi.ng/shader-ast`	changelog	AST DSL for x-platform shader code
`@thi.ng/shader-ast-glsl`	changelog	GLSL code generator
`@thi.ng/shader-ast-js`	changelog	JS code generator
`@thi.ng/shader-ast-optimize`	changelog	AST code optimization strategies
`@thi.ng/shader-ast-stdlib`	changelog	100+ useful AST shader functions
`@thi.ng/webgl`	changelog	WebGL 1/2 / GPGPU facilities
`@thi.ng/webgl-msdf`	changelog	MSDF font rendering
`@thi.ng/webgl-shadertoy`	changelog	Shadertoy-like WebGL setup

Low-level, binary, memory management, interop

Project	Changelog	Description
`@thi.ng/base-n`	changelog	Arbitrary base-n encoding/decoding with presets
`@thi.ng/binary`	changelog	Assorted binary / bitwise ops, utilities
`@thi.ng/bitstream`	changelog	Bitwise input / output streams
`@thi.ng/dlogic`	changelog	Digital logic ops / constructs
`@thi.ng/leb128`	changelog	WASM based LEB128 varint encoder / decoder
`@thi.ng/malloc`	changelog	Raw & typed array memory pool & allocator
`@thi.ng/morton`	changelog	Z-order-curve / Morton coding
`@thi.ng/range-coder`	changelog	Binary data Range encoder / decoder
`@thi.ng/rle-pack`	changelog	Run-length encoding data compression
`@thi.ng/soa`	changelog	Memory mapped data structures & serialization
`@thi.ng/unionstruct`	changelog	Wrapper for C-like structs / unions
`@thi.ng/vector-pools`	changelog	Data structures for memory mapped vectors
`@thi.ng/wasm-api`	changelog	Modular, extensible JS/WebAssembly bridge API
`@thi.ng/wasm-api-bindgen`	changelog	Polyglot bindings code generator for C/Zig/TS
`@thi.ng/wasm-api-dom`	changelog	WASM-side DOM manipulation
`@thi.ng/wasm-api-schedule`	changelog	WASM-side delayed code execution/scheduling

DSLs

Project	Changelog	Description
`@thi.ng/pointfree`	changelog	Stack-based DSL & functional composition
`@thi.ng/pointfree-lang`	changelog	Forth-like syntax layer for @thi.ng/pointfree
`@thi.ng/sexpr`	changelog	S-Expression parser & runtime infrastructure

Building

git clone https://github.com/thi-ng/umbrella.git
cd umbrella

yarn install
yarn build

Once the entire mono-repo has been fully built at least once before, individual packages can then be (re)built like so:

yarn workspace @thi.ng/transducers run build

# or

(cd packages/transducers && yarn build)

Building example projects

Please see the example build instructions in the wiki for further details.

Testing

(Most, but not all packages have tests)

Due to various build/config issues/complexities, we're now using our own minimal test runner @thi.ng/testament

yarn test

# or individually
yarn workspace @thi.ng/rstream run test

Documentation

Autogenerated documentation (using TypeDoc) will be saved under /packages/*/doc/ and is also available at docs.thi.ng.

yarn doc

Furthermore, an experimental tsdoc-based documentation repo provides an alternative markdown output including cross-references between all packages, but currently isn't frequently updated (only every few months):

https://github.com/thi-ng/umbrella-docs-temp

License

Contributors ✨

Thanks goes to these wonderful people (emoji key):

_{Karsten Schmidt} 💻 📖 🚧 💵	_Alberto 💻 💡 🐛 🤔 💵	_{Arthur Carabott} 💻 🤔 💡 📝 💵	_{André Wachter} 💻 🤔 🐛	_{Gavin Cannizzaro} 💻 🐛 🤔	_{Logan Powell} 📖 🐛 🤔 💵	_{Marcin Ignac} 🐛
_arcticnoah 💻	_allforabit 🐛 💻 🤔 💵	_{Yifan Wu} 🐛 📖	_stwind 💻 🐛	_evilive 💻	_{Bnaya Peretz} 💻 🐛 🤔	_oljeger 🐛
_{Nik Shevchenko} 🐛 💻	_{Matei Adriel} 💻 🐛 🤔	_{Pierre Grimaud} 📖	_{Matt Huebert} 💵	_{Raphael Saunier} 💵	_{Eric Ihli} 💵	_{David Pham} 💵
_TBD 🐛 🤔	_{Pedro Henriques dos Santos Teixeira} 💵	_{Jamie Owen} 💻 🐛	_{Robert Kesteson} 🐛 💻	_{Chancy Kennedy} 💵	_{Jarred Sumner} 🐛	_{Jamie Slome} 🐛 🛡️
_d3v53c 🐛 🛡️	_{Jannis Pohlmann} 🐛	_{Shakthi Prasad G S} 🐛 💻	_{Robin Gower} 🐛	_{Michael Latzoni} 🐛	_{Z Yin} 🐛	_{Damien Seguin} 🐛 💻
_{Rui Gil} 🐛	_Ja\|nz 💻 🚇 🚧	_{Tyler Freeman} 🐛 💻	_blackhuman 🐛	_{David Negstad} 💻	_{Muhammad Ridho} 🐛 💻	_{MarcusWagberg} 💻
_{Masashi Yoshikawa} 🐛