3mcd / harmony-ecs

A small archetypal ECS focused on compatibility and performance

harmony-ecs

A compatibility and performance-focused Entity-Component-System (ECS) for JavaScript.

Features

• Hybrid SoA and AoS storage
• Complex, scalar, and tag components
• Fast iteration and mutation [1] [2]
• Fast insert/relocate and auto-updating queries via connected archetype graph
• Compatible with third-party libraries like Three.js, Pixi, and Cannon

Installation

npm install harmony-ecs

Documentation

• Wiki
• API docs

Examples

This repo contains examples in the examples directory. You can run each project using npm run example:*, where * is the name of an example subdirectory.

Below is a sample of Harmony's API, where a TypedArray Velocity component is used to update an object Position component:

import { World, Schema, Entity, Query, Format } from "harmony-ecs"

const Vector2 = {
  x: Format.float64,
  y: Format.float64,
}
const world = World.make(1_000_000)
const Position = Schema.make(world, Vector2)
const Velocity = Schema.makeBinary(world, Vector2)
const Kinetic = [Position, Velocity] as const

for (let i = 0; i < 1_000_000; i++) {
  Entity.make(world, Kinetic)
}

const kinetics = Query.make(world, Kinetic)

for (const [entities, [p, v]] of kinetics) {
  for (let i = 0; i < entities.length; i++) {
    p[i].x += v.x[i]
    p[i].y += v.y[i]
  }
}

Harmony does not modify objects, making it highly compatible with third-party libraries. Take the following example where an entity is composed of a Three.js mesh, Cannon.js rigid body, and some proprietary TypedArray-backed data.

const Vector3 = { x: Format.float64 /* etc */ }
const Mesh = Schema.make(world, { position: Vector3 })
const Body = Schema.make(world, { position: Vector3 })
const PlayerInfo = Schema.makeBinary(world, { id: Format.uint32 })
const Player = [Mesh, Body, PlayerInfo] as const

const mesh = new Three.Mesh(new Three.SphereGeometry(), new Three.MeshBasicMaterial())
const body = new Cannon.Body({ mass: 1, shape: new Cannon.Sphere(1) })

Entity.make(world, Player, [mesh, body, { id: 123 }])

Note that we still need to define the shape of third party objects, as seen in the Mesh and Body variables. This supplies Harmony with static type information for queries and provides the ECS with important runtime information for serialization, etc.

Performance Tests

Run the performance test suite using npm run perf:node or npm perf:browser. Example output:

iterBinary
----------
iter
iterations 100
┌─────────┬────────────┐
│ (index) │   Values   │
├─────────┼────────────┤
│ average │ '12.46 ms' │
│ median  │ '12.03 ms' │
│   min   │ '11.71 ms' │
│   max   │ '18.76 ms' │
└─────────┴────────────┘

harmony-ecs's Issues

I :heart: this effort

Hey @3mcd!

First off, I recently read this blog post and super appreciate the shoutout and that you are also trying to bridge the synchronization gap between SoA and objects. I don't feel so lonely now 😅. Also, great job implementing archetypes! I still haven't had time to implement those.

Second, I was hoping that you would be willing to weigh in on an alternative way forward for the synchronization of data between SoA and objects that I drafted a short proposal for. As always, my focus is performance 🏎️. I believe this method of synchronization can bring the most performance benefits and provides an extreme level of interoperable flexibility, but I am keen to get a wide variety of opinions on this. Let me know what you think!

Cheers 🍻

Predictive Systems

Overview

I recently helped port the fantastic prediction/reconciliation algorithm CrystalOrb from Rust to JavaScript. The JS port works very well (see a very rough demo here) but CrystalOrb's current design (see this issue) works best when most game logic is located in the World.step function. Furthermore, the CrystalOrb client maintains two separate (but complete) simulations, without exposing one as the source of truth for client-only physics (e.g. particle physics).

This design makes it a bit tricky to implement ECS alongside CrystalOrb, since ECS logic is traditionally implemented as small, pure-ish functions (in that they usually only modify a small subset of entities and components), and writing non-predictive code is a bit difficult without implementing an entire command/snapshot/interpolate workflow, or writing similar command logic around the CrystalOrb "protocol".

It would be awesome if Harmony (and eventually Javelin) had support for this kind of predictive algorithm out-of-the-box. I'm dubbing this "Predictive Systems" in the meantime. Of course, currently Harmony doesn't have a concept of systems at all (only entities, components, and queries), but new structures could be introduced with this proposal which cover other best practices for system design and organization outside of netcode.

Proposal

Out-of-scope

• Peer-to-peer. I think designing the structures and functions involved within the model of the classical authoritative server <> client architecture is necessary to keep goals reasonable.
• Network transport/message protocol. User (or consuming library i.e. javelin) will need to bring their own encoder/decoder and networking stack.

In-scope

CrystalOrb does a few things that we can learn from:

• Clock synchronization of client and server using timestamps (although the author has some regrets about using wrapped i16 vs float64 -- I think the wrapped approach is cool and shaves a few bytes off each snapshot message)
• Client prediction with fast-forwarding using multiple simulations
• Commands
• Interpolation
• Client can tick at monitor refresh rate and still be accurate

Things we'll want to add

• Easily distinguish between predictive systems and states (components) and non-predictive ones
• Abstract the "current" predicted state to non-predictive systems (i.e. it seems like there is only one physics world to the client)
• Allow predictive state and non-predictive state to easily mesh

New structures and functions to consider

• Timestamps
• ClockSync
• Command
• Systems vs. predictive systems, auto-creating Archetype columns for predicted state, snapshot state, and maybe interpolated state?

API Sample

// System is defined like normal, without any notion of prediction
function input(ecs) {
    let command: InputCommand
    while (command = Command.take(ecs, CommandType.Input)) {
        const velocity = Entity.get(ecs, command.entity, Velocity)
        if (command.jump) {
          velocity.y += 10
        }
        // ...
        Entity.set(ecs, command.entity, Velocity, velocity)
    }
}

// On the client, we define a blend function that could interpolate between
// the latest predicted state and previous snapshot
function blend(queryFrom, queryTo, t) {

}

const pipeline = Pipeline.make(
    // On the client, a predictive system is configured with one or more
    // types to simulate in the future (relative to a server). Queries executed
    // within the predictive systems would somehow interact with "overlayed"
    // entity tables that shadow the original archetypes 
    System.makePredictive(input, [Prefabs.Player], [Commands.Input], blend),
    System.make(render),
)

// Some built-in commands would provide clock sync instructions and state
// snapshots to the pipeline's state for each predictive system
const commands = Command.makeCommandBuffer()

Pipeline.step(pipeline, world, commands)

// The server will need to generate snapshots at regular intervals and 
// send timestamped payloads to the client

Strict build

Enable noImplicitAny and strict in tsconfig.json and put the fires out.

`moveToArchetype` performance

Entity relocates are currently slow because most of the code was hurriedly written to get my ideas down. moveToArchetype and related functions need to be refactored.