mml-io / mml Goto Github PK

Use https://github.com/marketplace/actions/npm-publish

MMLClickEvent (dispatched on click events) should include position that matches the behaviour of MMLCollisionStartEvent as seen here:

What is your suggestion?

GLB files sometimes come with multiple animations. I would like to select which one to play. Currently, only the first animation is played.

Why is it important?

Many GLB files from SketchFab have multiple animations. Without being able to select which one to use, I cannot leverage this vast collection of 3D animation resource.

How would you implement it?

Not sure what the standard way for selection, maybe extend the 'anim' label to support selection, e.g. anim="https://my.com/duck.glb#run" to select the run animation?

Packages

No response

Extra details

No response

Describe the bug

There are some details we should consider about the CanvasText class used to draw text on behalf of the <m-label> tag:

the printAtWordWrap function:

As it is at the moment, the CanvasText function uses the printAtWordWrap method to draw the text with line-wrapping when width and height attributes were provided for the <m-label> tag.

Because printAtWordWrap immediately draws the text to the context, we cannot use line count and line height measurements performed inside this method to identify if the text will fit the provided height attribute, and do something about it in case it won't.

One alternative would be to transform this method into a computeLineWrapping method that would return the lines to be drawn and the height necessary for all the lines to fit the <m-label> with the provided font-size, width, and content attributes.

With that information, we would be able to choose between:

• expanding the <m-label> mesh height to fit all the text to be drawn considering the provided padding, font-size, and content attributes (which would have the undesirable side-effect of having a non-deterministic height for the <m-label> tag, and would also require some refactor of the Label element), or;

• creating the canvas with the necessary dimensions to draw all the text and "squish it" into the provided <m-label> mesh's height (which would have the undesirable side-effect of being distorted/squished, but already presumably better than missing text that was supposed to be drawn).

The padding usage:

We only have one padding attribute instead of separating it into horizontalPadding and verticalPadding. That reduces usage flexibility, but it also comes with some bugs.

Right now, if the user chooses a center alignment for the <m-label>, and provides a padding value greater than zero, the padding is wrongly applied to the left of the text, preventing it from being centered, and if a right alignment is chosen, the text usually gets rendered with a chunk cut out of the label given wrong padding influence.

Usage of the CanvasText class in other packages:

As I'm currently implementing a way to draw textures with text (for text chat and identification purposes) above the character on the 3d-web-client-core package of the 3d-web-experience repository, and such package already has mml-web as a dependency, we could benefit from exposing the CanvasText class on mml-web's index.ts so it can be imported and used for such purpose.

That would avoid duplicated code and improve maintainability, as the improvements delivered to this class on the mml-web package would also benefit any other package using it.

Expected behavior

The <m-label> class should offer more control over padding, and proper alignment in combination with the padding attribute(s). It should also prevent missing text when the font-size attribute combined with the content and height would cause the text to overflow the mesh's available space available to draw.

How to reproduce the bug

Using the <m-label> tag with the combination of attributes listed in the bug description can illustrate the described undesirable behavior.

Link to the code that reproduces this bug

No response

Packages

mml-web

Package versions

0.4.0

Extra details

No response

Describe the bug

• When running via the https://mml.io website using the @mml-io/[email protected] package the m-audio tags and m-video tags fail to stop the media from looping.
• Timings such as pause-time and start-time are also not handled consistently.

Expected behavior

• When using @mml-io/mml-web-runner the m-audio and m-video elements should not loop if loop="false".
• start-time and pause-time should be respected

How to reproduce the bug

• Paste <m-audio loop="false" src="https://public.mml.io/nenadsimic-jazzy-chords.mp3"></m-audio> into an example box on https://mml.io/examples.

• The audio file loops repeatedly

Link to the code that reproduces this bug

No response

Packages

@mml-io/mml-web-runner and or @mml-io/networked-dom-web-runner

Package versions

0.1.0

Extra details

No response

Describe the bug

When making changes to an MML document running using the mml-web-runner package it is possible to click on an element and trigger an error Node not visible for connection: 25.

Expected behavior

It should not be possible to trigger this error by clicking on an element that is in fact visible to the given user / connection.

How to reproduce the bug

Paste the following snippet into an example on https://mml.io/examples and click the orange label:

<m-group z="-20" y="10">
  <m-label width="22" height="3" color="orange" content="click me" font-size="200" alignment="center"></m-label>
</m-group>

You should see an error: Node not visible for connection: ${ID}

Link to the code that reproduces this bug

No response

Packages

mml-web-runner

Package versions

v0.1.1

Extra details

No response

Describe the bug

The TypeScript types specified by the libraries defined in this repository are an immediate indirection to the src directories which means that all dependent projects must parse the src with their own version of TypeScript and tsconfig rules.

This is problematic as version mismatches or increased strictness prevents the types being valid. See #33

Expected behavior

There are .d.ts files output for each library that are generated from the src without depending on the src directly.

How to reproduce the bug

Depend on a library such as mml-web and then increase the strictness / change the tsconfig.json for the dependent project. You are likely to get type errors.

Link to the code that reproduces this bug

No response

Packages

mml-web etc.

Package versions

0.1.1

Extra details

No response

This error line is intrusive, not actionable, and unlikely to hint at a genuine issue for most documents:

It should be removed and a fix for this minor inconsistency can be found in #100

When running a NetworkedDOM document without the monkeypatching applied in JSDOMRunner, there is a possibility that the diffs between states that are sent to a connected client are not consistent with the order of mutations and values that the document underwent.

This was originally indicated with an intrusive error message, but this (will be/was) removed in #101:

Original discussion can be found here: #35

Describe the bug

The script within <script>...</script> tags inside the MML document source loaded by a NetworkedDOM (or EditableNetworkedDOM) instance within a Jest test is not executed.

This appears to happen exclusively within the Jest jsdom environment. Scripts are however executed when set using an onclick attribute on elements.

RunnerIframe (within the networked-dom-web-runner package) is the class responsible for determining the runtime environment and may serve as a starting point for further investigation.

Expected behavior

Loading a document containing <script>...</script> tags should lead to the execution of the script.

For example...

const doc = new EditableNetworkedDOM(
    "http://example.com/index.html",
    IframeObservableDOMFactory,
    false,
    (logMessage) => {
        console.log('Received log message', logMessage);
    },
);

doc.load("<script>console.log('example')</script>");

...should result in Received log message example being display in jest output.

How to reproduce the bug

Create a new test suite within the test directory in the networked-dom-web-runner package, pasting in the following test code:

/**
 * @jest-environment jsdom
 */

import { EditableNetworkedDOM } from "@mml-io/networked-dom-document";
import { IframeObservableDOMFactory } from "../build/index";

test("networked-dom-web-runner jsdom script execution", async () => {
  const networkedDOMDocument = new EditableNetworkedDOM(
    "http://example.com/index.html",
    IframeObservableDOMFactory,
    false
  );

  networkedDOMDocument.load(`<script>console.log('EXAMPLE LOG');</script>`);

  networkedDOMDocument.dispose();
});

Looking at the Jest output, you will not see any mention of EXAMPLE LOG.

Link to the code that reproduces this bug

No response

Packages

networked-dom-web-runner

Package versions

0.1.1

Extra details

No response

Some early work: https://twitter.com/MarcusLongmuir/status/1673027121621393410?s=20

Define and implement the web version of an m-input tag that can be added as the direct child of m-interaction to allow requesting values from a user when submitting the enclosing interaction.

This would also benefit from introducing a label attribute to m-interaction to separate the meaning between an internal reference to the input/interaction and the presentation of that element to the user and also using those same semantics in the m-input element (e.g. name="songName" with label="Song Name").

Example code:

<m-interaction name="start" label="Start"></m-interaction>

<m-interaction name="stop" label="Stop"></m-interaction>

<m-interaction name="change track" label="Change Track">
    <m-input type="string" name="songName" label="Song Name" default="Happy Birthday" placeholder="Enter a song name..."></m-input>
    <m-input type="boolean" name="playShuffleAfterwards" label="Play Shuffle Afterwards" default="true"></m-input>
</m-interaction>

<m-interaction name="setVolume" label="Set Volume">
    <m-input type="number" name="volume" default="1.0"></m-input>
</m-interaction>

What is your suggestion?

My suggestion is to introduce strong typing for the details of CustomEvents in our developer API. This means we define & expose clear types for each event detail, improving type safety when developers are responding to these events.

Why is it important?

Strong typing, in supported environments, provides predictability, clarity, and type safety for developers, enhancing the overall developer experience.

How would you implement it?

Sketched out idea:

type MmlEventPositionEnterDetail = {
  elementRelative: ...
  documentRelative: ...
}

export type MmlEventPositionEnter = CustomEvent<MmlEventPositionEnterDetail>

new CustomEvent<MmlEventPositionEnterDetail>(positionProbeEnterEventName, {
  detail: {
    elementRelative: elementRelativePositionAndRotation,
    documentRelative: documentRelativePositionAndRotation,
  },
}),

So developers have type-safety when listening to events:

positionProbe.addEventListener("positionenter", (event: MmlEventPositionEnter) => {
    const { connectionId, elementRelative, documentRelative } = event.detail;
  });

Once such event detail types exist, they can be utilized by e.g. the React starter kit's type declarations (https://github.com/mml-io/mml-react-starter-project/blob/main/mml-document/src/declaration.d.ts) and others to provide a fully typed experience for developers. That sounds like many happy MML devs to me :)

On the implementation part, there are definitely non-trivial things to be considered, like pointed out by @MarcusLongmuir : #67 (comment)

Packages

No response

Extra details

This idea originated in the PR comments of #67

Describe the bug

Document-relative content loading was added as part of #72 / #73.

The implementation relies on being able to determine which document an element is part of to resolve the relative address into an absolute address. This is achieved by traversing the ancestors of the element when an attribute (e.g. src) is updated. In the case of snapshots, the element is not yet appended (and therefore ancestors cannot be established) when the attribute is updated.

The fix may overlap with #71 in that one strategy is to wait until the connectedCallback is fired before attempting to load content / models and then releasing them on disconnectedCallback.

Expected behavior

Document-relative content can be loaded as part of an initial snapshot

How to reproduce the bug

Specify a relative path in an element on one server (A). Load this document from another host (B). Observe that the content resolves relative to B rather than A.

Link to the code that reproduces this bug

No response

Packages

mml

Package versions

0.3.0

Extra details

No response

As suggested in #77 by @DirectiveCreator.

Support DRACO compression in the model loading using https://threejs.org/docs/#examples/en/loaders/DRACOLoader

Now tracked by much more thorough issue #104

Describe the bug

Once an M-Element is removed from the scene, its geometry remains.

Expected behavior

It would be ideal to properly dispose of removed objects' geometries and materials once their M-Element is removed from the scene.

How to reproduce the bug

This bug can be easily reproduced through the MML Playground or any other client where you have access to the renderer to check for info.

Use this MML object in your scene: wss://mml.mgz.me/examples/gliders.html

In the web package of the MML Playground, inside the update() method, add:
console.log(this.composer.renderer.info.memory.geometries) and you will see the number of geometries in memory constantly increasing, even though this MML object removes <m-cube> elements when the cell is dead.

Link to the code that reproduces this bug

No response

Packages

mml-web

Package versions

all versions

Extra details

No response

Support m-model children of m-character specifying a socket attribute which specifies which bone on the parent m-character skeleton the model is attached to (and is relatively positioned from)

What is your suggestion?

We may consider using a different material for the m-video tag to increase its color accuracy. Right now we use a double-sided MeshStandardMaterial, which by default has a shader that receives influence from various aspects of the scene (lights, shadows, dithering, fog, etc).

One possible alternative would be to use a RawShaderMaterial and selectively insert only the influences we consider a must-have.

Why is it important?

Further reasoning about the use cases may allow us to identify if the current material choice is the most adequate for them. This issue only suggests we may find out that color accuracy would be more desirable for an m-video tag rather than how much it's well blended to the scene.

How would you implement it?

Using a different material.

Packages

mml-web

Extra details

No response

Actual Behaviour

When using WebBrowserDOMRunner (document is executing in browser rather than JSDOM) event handlers on attributes such as onpositionmove are not executed.

Expected Behaviour

on attributes such as onpositionmove are executed when NetworkedDOM documents are executed in a browser and receive an event.

What is your suggestion?

Currently src attributes are interpreted as being relative to the page that the MML content has been loaded into. (e.g. an m-image with src="/foo.png" loaded on page localhost:8080 would result in requesting http://localhost:8080/foo.png)

If the m-image was an element of a remote document (e.g. an m-frame loaded from ws://example.com/) then src="/foo.png" should instead resolve to http://example.com/foo.png.

Why is it important?

This allows documents to refer to relative assets without having to determine their own address, which can be complex.

The behaviour of using the location of the document hosting the element requesting content is the same as iframe in HTML where the elements within the iframe with relative paths are loaded relative to the iframe's location.

How would you implement it?

Each remote document context has an address. This address can be used to determine the absolute path from potentially relative paths of content.

Packages

mml-web

Extra details

No response

m-frame should allow remote-loaded contents to be limited to a cube which has dimensions specified using attributes (e.g. width/limit-width)

Following from #87, the MML repo can publish each commit to the npm registry following the same approach as react: https://www.npmjs.com/package/react?activeTab=versions

The thing to note is that the 0.0.0-experimental-${commit}-${time} package versions shouldn't be tagged as latest