nsstc / sim-ecs Goto Github PK

This should be transparent, so that the current API does not change - ideally. Also, a way to provide explicit types should be provided, so more optimal storage types can be used.

The translation from array-of-objects components to objects-of-arrays can happen on maintain(). Query caches should still remain an array-of-objects, though, because that's how they are used.

Maybe by putting the sorting algorithm into a wasm file, we can improve its perf, however this needs profiling!

Description

Snapshots work like a time-machine. They record every change since a certain point and allow to do or undo them (rewind to the snapshot's creation time). Since they store deltas, they are more space-efficient than saving.

Use-Case

Snapshots should be save-able (world.save()) and can be used for simple game saves (e.g. load world from Prefab and apply snapshot)

Hints

Depends on #9

The implementation should be time-efficient as well, e.g. they should not care about the exact history. If a component was changed several times, the snapshot should only record the initial and most recent value.

refer to bevy: https://github.com/bevyengine/bevy/blob/latest/examples/ecs/event.rs

In system.ts there is this code:

if (!entity.hasComponent(accessStruct[access].component) || accessStruct[access].type == EAccess.UNSET) {
    return false;
}

But should this not be:

if (accessStruct[access].type == EAccess.UNSET) {
    if (entity.hasComponent(accessStruct[access].component)) {
        return false;
    }
} else {
    if (!entity.hasComponent(accessStruct[access].component)) {
        return false;
    }
}

Or the shorter:

if (entity.hasComponent(accessStruct[access].component) == (accessStruct[access].type == EAccess.UNSET)) {
    return false;
}

Since as things stand right now it will always check if it has the component and then return false if it does not.

The bench suite needs to be current state of the art, and add more benchmarks. It's based on an old version of the rust ecs benchmark...

At the same time, sim-ecs should now be integratable into other benchmarks, which work step-by-step (which is synthetic and has nothing to do with real applications..... but whatever), so maybe sim-ecs could be part of other benchmark suites as well and get more visibility from there.

Platform: Windows_NT win32 x64 v10.0.18363
CPU: AMD Ryzen 7 3700X 8-Core Processor@3593MHz

Default Suite / Simple Insert

Ape-ECS 150 ops/s ± 1.1%
bitecs 4167 ops/s ± 1.8%
sim-ecs 221 ops/s ± 0.66%
tick-knock 543 ops/s ± 1.1%

Given a State, which is possible to create at the moment, which manages its data with a prefab, there currently is no way to clean up prefab-bound entities if the data was loaded from a save.

export class GameState extends State {
    _systems = [...];
    prefabHandle?: TPrefabHandle;

    create(actions: ITransitionActions) {
        if (actions.getResource(MenuSelection).continueLastGame) {
            // no way to fill the prefab handle here!
            loadGameFromSave(actions);
        } else {
            this.prefabHandle = createNewGameFromPrefab(actions);
        }

        actions.maintain();
    }

    destroy(actions: ITransitionActions) {
        // target: remove the `if`, we should always clean up!
        if (this.prefabHandle) {
            actions.unloadPrefab(this.prefabHandle);
        }
    }
}

One idea might be to pass a unique ID to the loadPrefab() method, so it can mark entities and find them when loading a save, but that sounds complex for users. Maybe the ID could be a hash (of the prefab) and a new method (const prefabHandle = world.linkPrefab(prefabObj)) could be used to retrieve the handle, however that sounds bad when it comes to using updated prefabs on an old save. Then there also must be a method to upgrade the prefab handle to the new prefab version.

A completely different way to handle this might be to make it explicit which data is saved, always require a prefab to be loaded, and have saved data overwrite the prefab data. This means a lot more syncing, though, since then a prefab needs to include the necessary information to do such a sync, or the ecs needs an algorithm which can bind the data, which, just like before, sounds like a problem when changing the prefab slightly for a version update.

One more way would be to chunk the data in the save to make extraction possible per chunk, where a chunk could be tied to a state, or anything a user may want, really.

I won't stall v0.3.0 for a fix for this problem, though, since progress is slow as is.

I do have some ideas where this is handy. A entity-clone method would need to clone all components, too, though, so we don't accidentally end up with several entities referencing the same component instance.

Hence, the idea is that IEntity exposes a clone() method, which returns a new entity with cloned components.

• less boilerplate
• how to manage system life-cycle? Do we even still need this now that we have actions inside runners?
• depends on centralized run-criteria

Currently, when calling world.maintain(), the whole world is re-evaluated and sorted, which means a lot of overhead in general.

A first measure would be to only work on the current state instead of everything, which would also mean that on each state change the state would have to be maintained.

Another remedy would be to add changes to a queue and work on the queue when maintain() is called, so that only real changes are worked on.

I think bringing both these changes together can bring down the maintain-cost by a lot and distribute it to several points in time, which reduces stutters and hick-ups, especially when doing lots of entity adds and removes.

When building the release version of Pong, it fails at runtime with

Uncaught (in promise) Error: Component  was already registered!
    registerComponent http://localhost:63342/sim-ecs/examples/pong/public/bundle.js:16
    withComponent http://localhost:63342/sim-ecs/examples/pong/public/bundle.js:31
    <anonymous> http://localhost:63342/sim-ecs/examples/pong/public/bundle.js:16
    <anonymous> http://localhost:63342/sim-ecs/examples/pong/public/bundle.js:16
    n http://localhost:63342/sim-ecs/examples/pong/public/bundle.js:1
    <anonymous> http://localhost:63342/sim-ecs/examples/pong/public/bundle.js:16
    n http://localhost:63342/sim-ecs/examples/pong/public/bundle.js:1
    <anonymous> http://localhost:63342/sim-ecs/examples/pong/public/bundle.js:1
    <anonymous> http://localhost:63342/sim-ecs/examples/pong/public/bundle.js:1
bundle.js:16:24187

This needs further analysis, however it should be possible to build release-versions using sim-ecs, so we need urgent fixing here.

Browsers can use WebWorkers and NodJS can use a Cluster.

Ideas:

• cache all data and logic in the workers, however that would mean that all components have to be synchronized after every batch of systems is finished
• upload the whole world to every worker whenever it is run, while simple it means a lot of overhead

-> https://github.com/Ratysz/rfcs/blob/centralized_run_criteria/rfcs/29_centralized_run_criteria.md

The iter benchmark regressed by 330% between d2de6f8 and f11e943. The logic is the same, and the only difference I can think of is switching from arrays to Sets for entity storage - which does incur a performance overhead, but I can't believe that it's that heavy! Sets are better in this case since they protect against having the same entity several times.

I am already working on this. Coverage is important to find errors early and display that the library is well tested.

All new features after getting the coverage must keep the coverage above 80%

Many built-in Prototypes might work just by themselves, however providing a default handler will stabilize the process and take away boilerplate, so that users only need to implement what matters to them.

On that note, the default one might even be helper, which lets the user register components and how to handle them, which will probably make the code a lot nicer...

While I don't think that there are a lot of ECS libraries with a similar feature set, it is important to know where we are at performance-wise. A bulk of features is useless if we perform horribly!

Problems:

• There are no stable TS-only ECS libraries
• JS ECS libs are sparse
• Comparing this implementation to a compiled one (like SPECS) is unfair, however intriguing.

Imagine having a component like this:

class MyComponent {
  health: number
  target: IEntity
}

When running the ECS and adding such components to entities, everything will work as expected. However, as soon as save() is called, the target field will be serialized as independent entity, even though it really should reference another living entity. In order to handle it correctly, we need a serializer, which knows how to serialize the reference. We already have a deserializer, which can provide logic to reverse the process :)

I wonder if an entity ID might make sense, or if I should leave it to the user to add one (via component). Not every application needs to reference entities, and for those, having an ID is overhead.

Cloned version: 0.3.0
Issue: can't build the examples due to dramatic changes in API
it looks like you got rid of Query, WithTag, and even commands in actions. A brief look into the code didn't give me answers so please, update your piece of great work!

Currently, each state defines which systems will be running
As soon as you reach 50+ systems it gets really annoying to keep your eye on both GameState and createWorld() function

I really wish there were a more versatile way of assigning systems to state, not the vice versa
for example, using decorators

@RunsInStates([ GameState, MenuState ]) export class MenuSystem extends System { readonly query = new Query({ uiItem: Write(UIItem) });

or simply a readonly array

export class MenuSystem extends System { readonly _states = [ GameState, MenuState ]

I know it can also become messy if one wants to program your world in a state-first way, not the system-first one
But then there should be an "all systems" mechanism in states, which would work by reading all the unique systems from ECS world

Madge is a tool to detect circular dependencies. It is important to find them as they may randomly break applications. sim-ecs is of a size which makes it hard to find them manually, so this tool will help and make sure the output is clean :)

The process should exit with an error code if there are errors in the JS artifacts, but only warn about errors found in the TS files.

in world.ts, we have the following code, which is in need of more cleverness. It will group most systems by themselves, as it is now, which is very bad, especially once we figure out how to go multi-threaded.

The objective of this story is to get rid of the below mentioned todos.

    // todo: improve logic which sets up the groups
    protected prepareExecutionPipeline(state: IState): Set<TSystemInfo<any>>[] {
        // todo: this could be further optimized by allowing systems with dependencies to run in parallel
        //    if all of their dependencies already ran

        // todo: also, if two systems depend on the same components, they may run in parallel
        //    if they only require READ access
        const result: Set<TSystemInfo<any>>[] = [];
        const stateSystems = Array.from(state.systems);
        let executionGroup: Set<TSystemInfo<any>> = new Set();
        let shouldRunSystem;
        let systemInfo: TSystemInfo<any>;

        if (!this.sortedSystems || this.dirty) {
            // this line is purely to satisfy my IDE
            this.sortedSystems = [];
            this.maintain();
        }

        for (systemInfo of this.sortedSystems) {
            shouldRunSystem = !!stateSystems.find(stateSys => stateSys.constructor.name === systemInfo.system.constructor.name);

            if (shouldRunSystem) {
                if (systemInfo.dependencies.size > 0) {
                    result.push(executionGroup);
                    executionGroup = new Set<any>();
                }

                executionGroup.add(systemInfo);
            }
        }

        result.push(executionGroup);
        return result;
    }

De-/serializers must be able to handle Entity fields. One common case where this becomes important is when creating a hierarchy. The usual strategy is to use a Parent component, which is a wrapper with a reference to an entity in the same world (see Amethyst and Bevy for reference). So, sim-ecs needs to store that reference and re-create it on load.

// example of a Parent component
class Parent {
  constructor(public entity: IEntity) {}
}

My current idea is to give each entity an id (could be a UUID or counter) which enables the implementation of such a feature and leave the implementation to the user, since they will likely create their custom components with entity fields.

I am open to opinions, though.

Implement a new SystemWorld type, which strips dangerous methods from World (like maintain()) and will be used for passing World into a System.

I want to have a defined object, which contains a certain state each system may use during its run. The state object should be generated before any system runs and destroyed after all systems are finished.

In order to handle the generation, a custom function may be supplied, which receives the old state object. If no handler is supplied, the state object will contain a value passed to the run() or displatch() methods (for example undefined).

The state object should be frozen, so that systems don't accidentally change it, which would lead to different behavior based on system order, which spells chaos in capital letters.

The motivation for this is to allow things like a central simulation-state, frame-based events (input), etc. to hook directly into the ECS and be synchronized.

Move scheduling to a moe decoupled module

This project looks super promising! I've been looking around for ECS implementations for the web, and so far none of them seemed to expose components first, as opposed to returning entire entity references from any given query. Excited to see where this goes.

Only feedback I had initially was that it seems like it might be a bit clunky to manually create entities. From a collaboration perspective, you'll frequently have designers with a weak technical background creating enemies or powerups for a game, and they need to be able to quickly throw together a new entity without code-heavy syntax. Granted, this feedback is just going off of example entity creation you have in your readme, so there might be some undocumented ways to create entities that I'm just missing. Just as an idea, I think you might want to have something like an in-line deserializer that lets a designer create an entity with an object literal or a piece of json.

For instance:

createEntity({
    "components": ["background", "glyph", "mineable", "name", "position_2d", "collidable", "block_light"],
    "background": { "color": [ 64, 63, 67 ] },
    "glyph": { "color": [ 217, 144, 88 ],  "id": 211, "tile_type": 0, "tiles": [ 0 ] },
    "mineable": { "amount": 50, "resource": 148, "time": 30.0 },
    "name": "Coal vein"
});

Or even something like

createEntityFromComponents({
  background: {},
  glyph: {},
  mineable: {},
  name: {},
  position_2d: {}
});

Both of these examples might be awful though, ultimately the goal is to create an API that is approachable for non-technical contributors to a game.

It would be good to have at least some Hot Module Replacement functionality. Mostly, in order to have a quick feedback loop for changes to systems. But I'd also like to include states and schedules, maybe even components and prefabs.

Ideally, the system-builder should take care of it, but that needs more research. So, the minimalistic approach could be

import {MySystem1} from './systems/my-system1.ts';
import {MySystem2} from './systems/my-system2.ts';
import {MySystem3} from './systems/my-system3.ts';

const prepWorld = ...;
const runWorld = ...;

module.hot?.accept('./systems/my-system1.ts', () => {
  runWorld.hmrReplaceSystem(MySystem1);
});

module.hot?.accept('./systems/my-system2.ts', () => {
  runWorld.hmrReplaceSystem(MySystem2);
});

module.hot?.accept('./systems/my-system3.ts', () => {
  runWorld.hmrReplaceSystem(MySystem3);
});

Maybe, and I haven't tested that yet, it would be possible to even go a little simper...

import {MySystem1} from './systems/my-system1.ts';
import {MySystem2} from './systems/my-system2.ts';
import {MySystem3} from './systems/my-system3.ts';

const prepWorld = ...;
const runWorld = ...;

runWorld.enableSystemHMR([
  [MySystem1, './systems/my-system1.ts'],
  [MySystem2, './systems/my-system2.ts'],
  [MySystem3, './systems/my-system3.ts'],
]);

// or even just - using default exports:

runWorld.enableSystemHMR([
  './systems/my-system1.ts',
  './systems/my-system2.ts',
  './systems/my-system3.ts',
]);

The implementation will be able to match the class name to replace the system inside the scheduler.

Step 1 and the highest prio are systems, since they are the ECS core and I dread most about them not having hot replacement. It would be cool to support different setups, but since I use Webpack exclusively (and it's a standard), HMR for that would come first. I plan to look into different bundlers, though, since they mostly work the same with minor differences.

These convenience methods should be part of the query interface and run whenever the query result cache is updated.

Some of them are purely for edge-cases where it makes sense to manipulate the results in a usually bad way.

filter

filter((data: PDESC) => boolean): IXXXQuery

Go over each result and decide if it should stay in the cache or not.

push

push(data: PDESC): IXXXQuery

Add a new result to the end of the results cache. This is discouraged (because the result obviously is not part of a real component)

reverse

reverse(): IXXXQuery

Reverse the results' order in cache

shift

shift(data: PDESC): IXXXQuery

Add a new result to the start of the results cache. This is discouraged (because the result obviously is not part of a real component)

sort

sort((data: PDESC) => number): IXXXQuery

Works just like Array.sort(), but on the results.

Since world.run() is the core of the whole game loop, it should support features which are expected of any good game loop executing logic, and I think one of the missing pieces is the ability to run the logic at fixed intervals. Usually, that means at 60 steps per second.

I think using setInterval() would be a good fit for this instead of coming up with custom timing logic - but we'll have to see how it works out, especially with browsers currently scaling down precision because of security concerns.

I think, making run() use fixed steps should be configured via the configuration parameter - there could be a field which sets the frequency. At the same time, logic has to be in place which makes sure that every step waits for the previous step to finish first and then execute, so that things don't go to undefined land at every hickup. Plus this has to be logged, because it might mean that the logic is too slow and developers have to optimize the code to run at the target frequency, or decrease the frequency.

The new scheduler should feature an error-handling API

The README is a little outdated and not very detailed. I want to have a book, which goes over different scenarios, but also works as a quick guide for developers to get started.

Fire events when components are created, components are added/removed and deleted.

This allows us to create a game world and then load a saved game later on, or load stuff in the background and then add it to the current running world on the fly.

There should be a way to save worlds and load them later on. This would allow a simple way to save a game, create a world in a separate tool, etc.

The last version was released over half a year ago, and a ton of new features and fixes landed, so it is more than time to publish a new version. I want to get the following things done and then release.

• Create a demo game (Pong)
  • Create a PoC to work with 5ac17ee
  • Show off sim-ecs basics (working with the Es, Cs and Ss) 5ac17ee
  • Show off states 5ac17ee
  • Show off prefabs 5ac17ee
  • Show off save/load 877a156
  • Make sure the code is in good shape to actually show off 386a208
• Fix findings from creating the demo game
  • Fix bugs 9803f66
  • Have world.run options take a constructor for the initial state 3954768
  • Polish prefab handling e99bc4a, c2097f0
  • Have the PDA take constructors for push and pop methods 1bac088
  • Throw if a system is required to run a state, but was not registered 774ceab
  • Parse component constructor parameters for required fields in prefabs and throw if they are not available (optional)
• Make sure all tests pass

There are a lot of methods which start with with and end up having long names. Especially when creating e.g. many systems, having such long names becomes a burden, so there should be short aliases available:

ISystemBuilder

IWorldBuilder

There are with() and withAll() methods, too, but it does not make sense to further shorten them imo...

The trick is to create an Object known to the system which contains an array of a certain component. Whenever sim-ecs is confronted with this type or object, it knows how to handlw it. The user API could look like this:

createWorld().withComponent(Array(Component)).build();

While there is caching for execution-pipelines (the order in which systems run, one pipeline per state), it is not used optimally. The cache is tested on every state push, but run(), for example, starts a new build without checking the cache. Also, there is no way to prepare the cache for certain states (or all of them) at a user-defined point, which means that a state-push may mean hidden, unexpected costs. I imagine a public method, which takes a state and then prepares and caches a pipeline.

Deno libraries and tsc are still unsure about a way to move forward, however more tooling is emerging to solve the at least some problems through build steps.

• Investigate https://github.com/denoland/dnt or similar tools
• Tweak code
• Tweak build pipeline
• Have sim-ecs work on Deno, Bun, Node and in the browser

Sync-Points should be

• easy to create and extend (with other sync-points) 50c66d9 2839b08
• able to have labels, which allow arbitrary insertion of systems before or after them af4e9bb
• serializable as prefabs, so a complex pipeline can be created separately, stored and loaded and have a very simple syntax 50c66d9
• safe to use - we need checks for loops! bceb421
• execute stuff on point (commands.flush(), world.maintain(), custom logic like "extract to other thread") bb7ecae

When saving a world, there should be a way to include all resources, so saving and loading a world is always restoring the whole state, not only entities.

I imagine that this can be disabled using an option, in case the resources should be freshly created.

Instead of assigning entities to a system, they can be grouped by tag. Grouped entities should have similar components. Systems should then be assigned tags which they can iterate over.

This idea is taken from Legion, which chunks entities that way. Using tags will have the benefit that doing loose queries is way faster (as they query is compared against tags instead of iterating over all entities). In addition, systems might share tags, so that adding/removing components results in fewer query-comarisons.

The iteration-overhead will be slightly increased for systems, but I'd say that's minimal.

That name is more in line with the rest of the API

See stageless discussion for bevy-ecs

At the moment, any two systems without system-dependencies may run in parallel, even when both access the same components in WRITE mode. While for simple use-cases this may not be an issue, it is a hard to debug problem.

One solution may be that before dispatching systems, they are ordered. For world.dispatch() there wouldn't be a big benefit, however pre-scheduling dependencies like that will remove one conditional from world.run() for an added benefit.

Had an idea for a potential syntax that might be able to simplify how you access components. Essentially the idea was to allow a system to process components as parameters to some sort of callback function. Hopped on the Typescript discord and they came up with this snippet. You can see the usage at the very bottom, and the stuff above that is an example of how you would implement the types. Think this or something like it might be a reasonable direction to go in instead of creating a separate data object like in the main readme example for Systems.

class Foo { name = "foo" }
class Bar { age = 5 }

type Ctor = new () => object

type ToInstances<T extends Ctor[]> = {
    [K in keyof T]: T[K] extends Ctor ? InstanceType<T[K]> : T[K]
}

function makeReadonly<T extends Ctor>(arg: T): new () => Readonly<InstanceType<T>> {
    return arg as any;
}

function createQuery<T extends Ctor[]>(...args: T) {
    return {
        execute<U>(fn: (...args: ToInstances<T>) => U) {
            return fn(...args.map(x => new x()) as any)
        }
    }
}

const query = createQuery(Foo, makeReadonly(Bar));

query.execute((foo, bar) => {
    console.log(foo.name);
    console.log(bar.age);
});

And allow groups to be sub-dispatched.

TODO:

• what if a system in a group depends on systems outside the group?