natethegreatt / bitecs Goto Github PK

removing [key: string]: any from IWorld type def will allow stricter typing on IWorld extensions

I guess this is intentional, but although I can check if a particular entity has a component (World.hasComponent) there does not seem to be a way to get the data for that component (besides using a Query, which feels utterly wrong).

Am I overlooking something obvious?

[EDIT] Moved some unrelated comments to a separate issue

import * as bitecs from 'bitecs'

const world = bitecs.createWorld()

const systemA = bitecs.defineSystem((world, parameter) => {

  console.log(parameter)

})

const systemB = bitecs.defineSystem((world, parameter) => {

  console.log(parameter)

})

const pipe = bitecs.pipe(systemA, systemB)

pipe(world, "Hello")

Expected output:

Hello
Hello

Received output:

Hello
undefined

Not sure if this is the intended way to use pipes (I understand that to get around this I could add a property to world instead) but the fact that the parameter is passed through successfully to the first system, but not any thereafter, is confusing. This led to quite a tricky bug for me (I was passing deltaTime down a pipe and was bewildered when refactoring caused it to break)

As an initial step towards making deserialization more modular, I think it would be useful to have an optional callback that would execute on deserializing entities to give access to the EIDs present in the packet.

deserialize(world, packet, eid => {
    // do cool stuff with EID here
})

Terser and UglifyJS (to name two common ones) rename functions by default when minifying, and this breaks any code that relies on comparing function names to strings.

In my case, it causes Not() to be ignored in queries. It probably affects Changed() too.

My current workaround is just to disable renaming functions that match a regexp (keep_fnames: /^Query/ in Terser minify options)

Ideally the library should survive renamed functions, as this could be a very sinister hidden bug for anyone building minified bundles.

Update: I see 385f7fe; thank you and nice work! This issue can be closed now or I will close it when I have tried out the next release.

I'm building a standard multilevel dungeon roguelike. Each level creates around 3500 entities. If the default world size is left alone, at level 27 I get a very clear error message:

bitECS - max entities of 100,000 reached, increase with setDefaultSize function.

If I increase the limit to 1,000,000 I get a different error when I reach the original default of 100,000 entities.

Uncaught TypeError: Cannot read properties of undefined (reading 'fill')

Seems like some of the internals are still limited to the default world size.

For the sake of completeness I tried setting the default to 1,000 and instead get the correct error upon reaching 1,000 entities:

bitECS - max entities of 1,000 reached, increase with setDefaultSize function.

How To Repro:

Clone project and check out the world-size-bug-repro branch of my repo here: https://github.com/luetkemj/pixitest

Install deps
npm i

Start dev server:
npm start

Go to localhost:8080 in your favorite browser.

Your character @ starts on a stair case > just press and hold the > key until you get to the 27th floor or the game blows up. Default size is set to 1,000,000 on ln18 src/index.js.

currently only one enter/exitQuery reference is allowed per query. should be allowed to have multiple references on a single query

During a bug in my own code I noticed that the Comp.attr[id] notation will also give a value, even if the entity with id is already removed. In my implementation, I had a ForceArrow component which takes two other entities and applies velocity modifications to them.

It would be nice to have something like exists(world, eid) if possible.

Is this a real intention?
https://github.com/NateTheGreatt/bitECS/blob/04f1faa/src/index.ts#L2

// Typescript port is coming soon.

I wanted to contribute to this project. I don't have skill set to contribute this project. I never developed any game. I am learning to code and I wanted to start my journey as web developer. Can I contribute to this project?

Can I add CODE_OF_CONDUCT.md and CONTRIBUTION.md to this project.

Is there a way to see if an entity exists in the world?

Is this the expected behavior?
Incidentally, based on this comment is hasComponent not recommended to use?
If so, is there another way to reuse the queried results to further filter the entities with specific tags without requerying?

Queries defined outside a system function will return all matches on the first call and only new matches on subsequent calls.

Queries defined inside a system function will always return all matches.

Created a minimal repro here: https://github.com/luetkemj/bitecs-bug-repro

I would expect that a query defined outside a system function would always return all matches. I would not expect to have to redefine it every time the system runs to get this behavior.

Hi @NateTheGreatt

That bitECS of yours is 🚀 for me.

I'm not sure I'm using it the proper way...
Anyway, I found a freaky behavior in the deserialize process, confusing old and new entities.

I narrow the mismatch to

After removing some entites, when reloading previous state, the deserialize function seems to reuse cursors that might be already present in packet to load

That, I think, is giving this weird behavior when instead of loading 2 entries, it's loading the last of them with matching previously removed entities, and it allow "another load" of the same packet with dangling entities I guess (me lost a 🧠 )

One word : WHY

I simplify the confusing demo above to an integration test to investigate :

// [...]

  it("should load 2 item after removing them", () => {
    // given
    const { assertItems, assertLength, more, less, save, load } = setup();

    // when
    // We add 2 items
    more();
    more();

    assertItems([
      [1, 100],
      [2, 101]
    ]);
    assertLength(2);

    // when
    // We save the state
    const pack = save();

    // when
    // We remove the 2 items
    less();
    less();

    // NOTE(douglasduteil): reseting the globals seems to hack the test
    // HACK(douglasduteil): use setDefaultSize to trigger a resetGlobals
    setDefaultSize(1e5);

    // when
    // We load the last state
    load(pack);

    // then
    assertItems([
      [0, 100],
      [1, 101]
    ]);
    assertLength(2);
  });


// [...]

I'm not sure about the setDefaultSize above...
It make the test pass but smells pretty wrong, I don't know ... (404 on 🧠 again)

Here is a test case sandbox to illustrate the behavior.
https://codesandbox.io/s/bitecs-0-3-34-serialize-test-b7uxt?file=/Serialize.test.js

Might be related to #67

Thanks for the reading :)
Looking forward to new versions.

🧠 🛹

Let's consider a simple collision system:

for (let i = 0; i < entities.length; i++) {
  for (let j = i + 1; j < entities.length; j++) {
    if (isColliding(entities[i], entities[j])) {
      …

How could it be implemented using bitECS? I have the feeling that something is missing. The ability to query components directly or having multiple queries per system could do the trick.

I've been trying to get 2 systems working with queries that listen to Changed on the same components, and I can't make it happen.

My understanding is that I should set clearDiff to false when evaluating the first query, however something's not right here and when I do that the first query is constantly finding entities instead of only the changed ones, and the second query is still finding nothing.

Version 0.3.16-3 btw.

I think something went wrong when publishing the package. core-ecs folder has been included into the NPM distribuable and contains many binaries (~25MB).

https://unpkg.com/browse/[email protected]/core-ecs/node_modules/uWebSockets.js/

To prevent this, you could specify the files field in package.json.

While working a benchmark suite, I figured that bitecs is not as performant as it may seem or be. I believe there is dramatically polymorphic code in some hot blocks.

Let's consider the following setup:

let world = bitECS();

world.registerComponent("A", { value: "int32" });
world.registerComponent("B", { value: "int32" });

world.registerSystem({
  name: "AB",
  components: ["A", "B"],
  update: (a, b) => (eid) => {
    a.value[eid] += b.value[eid];
  },
});

for (let i = 0; i < 1_000; i++) {
  let e = world.addEntity();
  world.addComponent("A", e, { value: 0 });
  world.addComponent("B", e, { value: 0 });
}

No surprise here, bitecs is in fact the fastest ECS implementation at doing this: looping over 1,000 entities with 2 components. world.step() can be run at 550,648 op/s!

I decided to add a second system:

world.registerSystem({
  name: "A2",
  components: ["A"],
  update: (a) => (eid) => {
    a.value[eid] *= 2;
  },
});

The second query would traverse the same number of entities, so I expected the operation count to be divided by two. Instead I got 61,925 op/s. That's a 90% performance drop. Way below what one could expect.

By rewriting System#execute, I was able to fix the performance drop. However, I must warn you, it looks bad.

let executeTemplate = (
  system,
  localEntities,
  componentManagers,
  updateFn,
  before,
  after
) => (force) => {
  if (force || system.enabled) {
    if (before) before(...componentManagers);
    if (updateFn) {
      const to = system.count;
      for (let i = to - 1; i >= 0; i--) {
        const eid = localEntities[i];
        updateFn(eid);
      }
    }
    if (after) after(...componentManagers);
  }
};

let executeFactory = Function(`return ${executeTemplate.toString()}`)();

system.execute = executeFactory(
  system,
  localEntities,
  componentManagers,
  updateFn,
  before,
  after
);

I also moved the updateFn check. It saves another 10%.

The defineQuery documentation is pretty lacking currently. I see in code that there are query modifiers for Not, Or, Changed, Any, All, and None. Some explanation of what these do or how to use them would be appreciated in the API.md file.

https://github.com/NateTheGreatt/bitECS/blob/master/docs/API.md#defineQuery

I am also willing to contribute a PR for this documentation if this is work that should be done and can be done by an outsider.

I was reading the code and there is this function in Entity.js

let resizeThreshold = () => globalSize - (globalSize / 5)

and I was thinking that wouldn't it be better if it was changed to

const getResizeThreshold = () => globalSize - (globalSize / 5)

so that it's more consistent with other functions like getEntityCursor, getRemovedEntities, getGlobalSize etc. Maybe there is reason why you did it the way it's done but while reading the code, I just thought it looked a little weird. Maybe I am wrong.

currently defining a query with the same components as a previously defined query will create an entirely new query instead of reusing the previously defined query. should instead reuse the previously defined query

I feel that the new way of defining systems as a function that returns world is anthithetical to one of my favourite things about bitECS: the lack of boilerplate code.

To this end, I suggest you bring back defineSystem, and make that add the world return instead of the user having to add it on every system themself.

Pls close if you don't agree, just my opinion!

Have a modifier similar to Changed like Deserialized or something which would track entities/components that have been updated via deserialization. I would have a use for this to update the rendering data on the client for entities which changed on the server and had the diff sent. It seems to me that as the serializer is already handling the diffing, I wouldn't want to do it a second time when I could just grab the list of entities that were deserialized and operate on those.

In my mind the implementation would be fairly trivial because the deserializer already iterates through EIDs and component data, so you'd just need to have a lookup tracking which components were iterated over, and reset that when querying.

I've never worked with TypedArrays before so I may be coming at this from the wrong direction. How would I define a component with string types? For example, suppose I want to define a "Name" component that stores the name of the entity - how would I do that?

It would be nice to have a changelog file documenting breaking changes to the API so we don't have to browse the docs to figure out what's different and why our code doesn't work anymore.

I use bitEcs in my step-lock game as game states, somehow i let the client to predict next game frame state, if the predict state does not match the server's state, then I want to rollback the state to previous state. I tried to serialize the world every frame and save the packet, then derialize the world with certain frame if the predict is wrong. But the derialization won't delete the entitis created by the prediction. How can I achieve this idea? Any suggestion would be helpful. Or is there any way I can directly repace the world's state with the packet?
Many thanks!

I've found an unexpected behaviour which I believe to be a bug or at least in need of safeguarding against somehow.

I've boiled it down to queries getting broken if you remove then add the same component during the same system call without committing removals in between.

I made a repro project here so you can see it in action: https://github.com/sebovzeoueb/bitecs-not-query-test

Just load up index.html and open the dev console to see the output from the code. That repo's README has more detail on what's going on and why I believe it to be a bug.

In the registerSystem sample with enter/exit, it seems enter/exit is defined as a function directly receiving the entity ID (eid). However, the signature is very much like update, but instead a single eid is given instead of an array (e.g. you have to do exit: (position, velocity) => eid => { do something with the entity id } instead of the current example showing exit: eid => { .... }.

Just a reminder that it would be nice to expose the Map of serializer IDs to local to better handle entity/component removal over the network with custom messages.

There are some cases where I want to simply iterate over all existing entities in a world.

Lacking a function in the API, my first thought was using an "empty" query.

Since the typings for defineQuery require an array, I passed in an empty one []

I haven't tested this thoroughly, but calling removeEntity while iterating this query has strange behavior. The first run through, it goes fine. After adding some entities back to the world, running the query again will only iterate entities that weren't deleted in the first query call. None of the new entities show up.

When calling defineQuery without the [] argument, it works much better. However, the array returned by this query will be mutated as you delete entities, so you need to clone it first if you want to get every entity (or maybe a while(entities[0]) loop works). This is different than queries with components, which defer the removal.

I'd rather avoid these unexpected behaviors entirely by having a simple function to get all my entities, and have it return an array that will not be mutated by removeEntity.

I followed the example from the docs, but using pipe with a query and a serializer is returning null for me.

See https://github.com/sebovzeoueb/bitecs-serializer-test the 3rd test is finding 2 entities in the query, but piping the query to the serializer is returning a null packet.

Not sure if I'm missing something, but when I try to call getAllEntities I get an error and looking at the getAllEntities itself returns undefined. Looking at the compiled code I could not find any mention of getAllEntities but the types still mention it.

Awesome project! I'm excited to give this a go and build something with it :)

FYI the example on the README doesn't work. Running the code exactly as is results in the following error Cannot destructure property 'time' of 'world' as it is undefined.. Seems like world isn't getting passed into the pipes correctly.

I can just run the systems manually to get around it for now.

setInterval(() => {
  movementSystem(world);
  timeSystem(world);
}, 16)

Thanks!

Repro:
installed version "bitecs": "^0.3.19-1"
Run the code from the example in the README

Actual:
Throws error Cannot destructure property 'time' of 'world' as it is undefined.

Expected:
To not throw an error

Hi!
Sorry if I made a stupid mistake. My knowledge in node.js is weak :( But I don’t understand why doesn’t see module build in your package.
I am getting an error in the console:

Node v16.5.0
bitecs v0.3.15-1

package.json:

tsconfig:

On 0.3.21-5 because -6 doesn't work at all (see my other issue), I can use the Changed modifier in a serializer, but its actually still just constantly serializing all the fields rather than just the changed ones.

EDIT: I've once again gone to my test project and found some interesting additional clues. Here's the project_ https://github.com/sebovzeoueb/bitecs-serializer-test (ignore the issues in README, those are now OK).
You'll see that I'm adding a component, serializing it with the changed modifier, changing a value and serializing again to test this. What I'm seeing is that after no value change it's still serializing a few bytes instead of the expected 0, and after a change it's serializing more. Seems like there's some base value in there getting serialized even when there's no change.

EDIT 2: Test project seems to be working just as well on -6.

EDIT 3: Existing tests work now, but I've added a test showing that repeatedly running the serializer after a change is constantly picking up the change. I also have a hunch that all non zero array items are counting as changed which would explain the massive packets in my actual game.

Hey there, cool looking project! It's really great to see people promoting data-oriented game architecture, there aren't many of us.

I've got a very similar project over at https://github.com/mreinstein/ecs

I don't know if there's significant ways that we can help each other out, but maybe there are resources/ideas we could share to make both of these ECS projects better:

• I'd love to see how mine performs on the bitECS benchmarks. I haven't done any formalized benchmark testing, but when running it in my game prototypes the time spent is way under 1% of total CPU time.
• I have a functional (but crappy looking) chrome extension that is able to read ECS world data from the page and display it in a devtools panel. Maybe this is something we could make common, or you could copy/paste into bitECS

I also added a link to bitECS at the bottom of my README, in the interests of spreading interest in data-oriented tools. :)

Anyway just wanted to say great job, and good to see these implementations not based on Object Oriented Blech-gramming.

Changed modifier used with serializers no longer works, all my packets have 0 byte length. Appears to be working fine in queries however.

I have notice the last update was last 2021. Is this package still maintained?

As far as I understand there is no way to store anything other than numbers in components.
I understand that this is part of how BitECS manages to get such fantastic performance.
When using BitECS, what is the expectation for how to attach a string to an entity?

see https://github.com/sebovzeoueb/bitecs-serializer-test for repro.

This happens when serializing an entity with a component that has a uint16 array field of 1024 elements:

    at DataView.getUint16 (<anonymous>)
    at Serialize.js:284
    at index.js:20
    at AnonymousSystem_internal (System.js:18)
    at index.js:20
    at index.js:93
    at index.js:93```

see the last example in https://github.com/sebovzeoueb/bitecs-serializer-test

Serializing using the Changed modifier on a component is causing this error:

    at Serialize.js:110
    at index.js:77
    at AnonymousSystem_internal (System.js:18)
    at index.js:22
    at index.js:95
    at index.js:95```

Using the simple code below, world.save() will throw the above error:

import World from 'bitecs'

// Create a world
export const world = World()

// Register components
world.registerComponent('transform', {
    position: [{index: 'uint8', type: 'float32', length: 3}],
    rotation: [{index: 'uint8', type: 'float32', length: 4}],
})
world.registerComponent('box', {
    size: 'float32'
})
world.registerComponent('sphere', {
    radius: 'float32'
})

console.log( world.save() )

Maybe this is related to the arrays inside the transform component?

when creating deserializers with Changed modifiers in the config, additional and unnecessary shadow states are being created which is bloating the memory consumption.

thanks @sebovzeoueb for finding this issue!

Not queries will correctly identify entities without components the first time they are run, but on subsequent runs will not accurately return components without entities.

import {createWorld, addEntity, addComponent, removeComponent, defineComponent, defineQuery, Not} from 'bitecs'

const Foo = defineComponent()

const notFooQuery = defineQuery([Not(Foo)])
const fooQuery = defineQuery([Foo])

const world = createWorld()
const eid1 = addEntity(world)

console.log('Foo:', fooQuery(world))
console.log('Not Foo:', notFooQuery(world))

addComponent(world, Foo, eid1)
console.log('Foo:', fooQuery(world))
console.log('Not Foo:', notFooQuery(world))

removeComponent(world, Foo, eid1)
console.log('Foo:', fooQuery(world))
console.log('Not Foo:', notFooQuery(world))

The fooQuery accurately returns components with or without the Foo component each time.
The notFooQuery returns the same output all three times.

The release v0.1.4 does not work on node. The reason being the addition of type inside package.json.

  "main": "./dist/index.min.js",
  "type": "module",

When using type, it indicates that main is an ES module.
Here is a backward compatible definition:

  "type": "module",
  "main": "./dist/index.min.js",
  "exports": {
    "import": "./dist/index.es.js",
    "require": "./dist/index.min.js"
  }

I have serialization working OK on 0.3.21-5 but on 0.3.21-6 it throws Offset is outside the bounds of the DataView. This is on a component with a large array property.

UPDATE: after more verification, 0.3.21-5 is also broken for me, it's possible I forgot to rebuild the client after updating. Last working version is actually 0.3.21-4b. Note that the server being on 0.3.21-5 was fine, so this implies to me the problem is happening somewhere during deserialization.

UPDATE 2: the error is being thrown from Serialize.js 241: let eid = view.getUint32(where) and sometimes other places involving where so I think this is getting incorrectly incremented sometimes.

UPDATE 3: I've added components from my actual game code to the test project (https://github.com/sebovzeoueb/bitecs-serializer-test) and you can now reproduce the error for yourself.

UPDATE 4: I've narrowed it down to tag components. As soon as there's a tag component being serialized, it breaks the deserializer.

I couldn't get arrays in arrays working. Here is an example:

"use strict";

import {
	createWorld,
	defineComponent,
	Types,
	addEntity,
	addComponent,
} from "https://cdn.skypack.dev/bitecs";

const testWorld = createWorld();

const TestComponent = defineComponent({
	// Array of floats
	array: [Types.f32, 3],
	// Array of arrays of floats
	array2: [[Types.f32, 3], 3],
});

const testEntity = addEntity(testWorld);
addComponent(testWorld, TestComponent, testEntity);

// Works
TestComponent.array[0][testEntity] = 0.1;
TestComponent.array[1][testEntity] = 0.2;
TestComponent.array[2][testEntity] = 0.3;

// Doesn't work
// TypeError: TestComponent.array2[1][0] is undefined
TestComponent.array2[0][0][testEntity] = 0.4;
TestComponent.array2[0][1][testEntity] = 0.5;
TestComponent.array2[0][2][testEntity] = 0.6;

TestComponent.array2[1][0][testEntity] = 0.7;
TestComponent.array2[1][1][testEntity] = 0.8;
TestComponent.array2[1][2][testEntity] = 0.9;

TestComponent.array2[2][0][testEntity] = 1.0;
TestComponent.array2[2][1][testEntity] = 1.1;
TestComponent.array2[2][2][testEntity] = 1.2;

I've updated my example project here to bitECS 0.3.20: https://github.com/sebovzeoueb/bitecs-serializer-test

Looks like my previous issues have been solved, but now when piping systems I'm getting some kind of error in which the world being passed in appears to be undefined

Serializing a world seems to work but deserializing throws the following error:

Uncaught TypeError: Cannot read properties of undefined (reading 'Symbol(storeBase)')
at bitecs.v0.3.34.js:472

Here's some repro code:

import { createWorld, defineSerializer, defineDeserializer } from "bitecs";

let packet;

export const save = (world) => {
  const serialize = defineSerializer(world);
  packet = serialize(world);
};

export const load = () => {
  let newWorld = createWorld();
  const deserialize = defineDeserializer(newWorld);
  deserialize(newWorld, packet);
  return newWorld;
};

I don't get an error if I use defineSerializer with same world that I serialized but I'm trying to figure out how that's useful. If I have the world already, there's no reason to deserialize.

Not sure if bug or I'm missing something...