niklever / canvasui Goto Github PK
View Code? Open in Web Editor NEWA Three.JS WebXR UI. Enabling easy UI creation for immersive-vr sessions.
Home Page: canvas-ui-eight.vercel.app
License: GNU General Public License v3.0
A Three.JS WebXR UI. Enabling easy UI creation for immersive-vr sessions.
Home Page: canvas-ui-eight.vercel.app
License: GNU General Public License v3.0
When several meshes overlap (example: keyboard expanded in front of another panel), each mesh has its own event handler, and its own raycasters.
A click/hover/select on 2 superimposed buttons will cause 2 events instead of one (the first button hiding the second because of its offset on the Z axis).
To improve this behavior and take into account the Z order of multiple CanvasUI components, 2 ideas should be explored:
implement placeholder for input-text
const config = {
...
message: {
type: 'input-text',
placeholder: 'Click or select me',
...
},
label: {
type: 'text',
...
}
};
const content = {
message: '',
label: 'Keyboard demo'
};
CanvasUI Console: console.log wrapper with info/warn/error/debug. Of course limited to text.
( just an idea, a discussion, looking for feedbacks )
First, we currently set positions, dimensions, font, colors, for each elements using attributes, but mixed with methods in the same object.
Same thing for the config object which is a mix of different properties with differents usage (i.e element definition vs renderer, or panelSize vs an element name). This currently involves some filters when looping over object attributes in CanvasUI.
Let's take the current buttons example.
const config = {
panelSize: { width: 2, height: 0.5 },
height: 128,
info: { type: "text", position:{ left: 6, top: 6 }, width: 500, height: 58, backgroundColor: "#aaa", fontColor: "#000" },
prev: { type: "button", position:{ top: 64, left: 0 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onPrev },
stop: { type: "button", position:{ top: 64, left: 64 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onStop },
next: { type: "button", position:{ top: 64, left: 128 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onNext },
continue: { type: "button", position:{ top: 70, right: 10 }, width: 200, height: 52, fontColor: "#fff", backgroundColor: "#2659a4", hover: "#3df", onSelect: onContinue },
renderer: this.renderer
}
Below an imaginary refactor, a CSS file and a different CanvasUI setup.
CSS entries. element name are IDs. But we can use classes too. Some littles changes i.e fontColor become color to match css rules.
.button {
color:#bb0;
width:64px;
top:64px;
font-familly:"...";
background-color:#000099;
transition: background-color 0.2s;
}
.button:hover {
background-color:#ff0;
transition: background-color 1s;
}
#info {
left:10px,
top:10px,
width:492px,
height:50px,
background-color:#aaa;
color:#000
}
#continue {
top:70px;
right:70px;
width:200px;
height:50px;
color:#fff;
}
#continue:hover {
color:#0000ff;
}
.red {
background-color:red;
}
Only elements declaration, rather than a mixed object.
Note the "style" attribute to override CSS definitions.
Note the "class" attribute to link custom css properties.
Both "style" and "class" should be like HTML attributes, but parsed by CanvasUI, with properties propagation when object is created.
const elements = {
info: { type: 'text', value:'' },
prev: { type: 'button', value:'...', style:{ left: 0 }, onSelect: onPrev },
stop: { type: 'button', value:'...', style:{ left: 64 }, class:'red', onSelect: onStop },
next: { type: 'button', value:'...', style:{ left: 128 }, onSelect: onNext },
continue: { type: 'button', value:'Continue', onSelect: onContinue },
};
Everything which is not an element attribute should be moved to the config object
const config = {
panelSize: { width:2, height:0.5 } ,
height: 128,
threejs:{
renderer: this.renderer, // mandatory for XR
camera: this.camera, // mandatory when mouse controller used
scene: this.scene // mandatory for scrolling example, needed to add raycaster pointer when select
}
// this is defaults
controller:{
devices:{
mouse:{ enable:true, handler:'follow' },
xr:{ enable:true, raycasterLinesLength:10 }
},
select:{
hover: false, // select once (keyboard behavior),
// hover:true, // trigger select event while raycaster is moving (select+hover) on the element (colorpicker or slider behavior)
repeater:{
// when hover=false, we have a repeater option, delayed trigger select event + repeat.
// usefull to handle "delete" keyboard key, select will repeat until you selectend
timeoutms:100, // 0 mean immediate,
repeatms:20
}
}
},
}
new CanvasUI( elements , config );
Benefits:
There are multiples ways to read CSS values. One ot them don't need to push something in the DOM Tree, but as some logical limites.
https://stackoverflow.com/questions/324486/how-do-you-read-css-rule-values-with-javascript
I successfully tested the one from Derek Ziemba
//Inside closure so that the inner functions don't need regeneration on every call.
const getCssClasses = (function () {
function normalize(str) {
if (!str) return '';
str = String(str).replace(/\s*([>~+])\s*/g, ' $1 '); //Normalize symbol spacing.
return str.replace(/(\s+)/g, ' ').trim(); //Normalize whitespace
}
function split(str, on) { //Split, Trim, and remove empty elements
return str.split(on).map(x => x.trim()).filter(x => x);
}
function containsAny(selText, ors) {
return selText ? ors.some(x => selText.indexOf(x) >= 0) : false;
}
return function (selector) {
const logicalORs = split(normalize(selector), ',');
const sheets = Array.from(window.document.styleSheets);
const ruleArrays = sheets.map((x) => Array.from(x.rules || x.cssRules || []));
const allRules = ruleArrays.reduce((all, x) => all.concat(x), []);
return allRules.filter((x) => containsAny(normalize(x.selectorText), logicalORs));
};
})();
Current syntax, we can't really define private / public methods/getter/setter/variable : everything is public.
class foo {
constructor() {
this.bar = true;
}
}
The refactor will consist to separate public and private things, i.e
class foo {
// private variable
#foo_private= "bar_public";
// public variable
foo_public = "foo_public";
constructor() {
// modify private variable
this.#foo_private = "bar2_private";
// modify public variable
this.foo_public = "bar2_public";
// run a private method
this.#myPrivateMethod();
// run a public method
this.myPublicMethod();
}
// private method
#myPrivateMethod() {
}
// public method
myPublicMethod() {
}
}
Rather than something which is not currently used in all branches (i.e Prototype add/replace methods/variables, we should use something like this :
class CanvasText extends CanvasWidget {
}
Rather than
let property;
if (object) {
property = (typeof object.attribute != undefined) ? object.attribute : defaultValue;
}
We can (note the question mark)
const property = (typeof object?.attribute !== undefined) ? object.attribute : defaultValue;
Not only ColorPicker, Keyboard, Slider ...
We should have Text, Button, Image
Let create a dynamic left menu in js/css so we have a way to quickly switch between examples. Like threejs examples left menu without pictures (puppeteer is a way to automate screenshot and make thumbail, but it should be an improvement for later).
Examples will be defined in the js file, something like this ;
const exampleList = [
{
title: 'Buttons - Panel',
url:'../buttons/'
}
First, we currently set positions, dimensions, font, colors, for each elements using attributes, but mixed with methods in the same object. Same thing for the config object which is a mix of different properties with differents usage (i.e element definition vs renderer, or panelSize vs an element name). This currently involves some filters when looping over object attributes in CanvasUI.
Let's take the current buttons example.
const config = {
panelSize: { width: 2, height: 0.5 },
height: 128,
info: { type: "text", position:{ left: 6, top: 6 }, width: 500, height: 58, backgroundColor: "#aaa", fontColor: "#000" },
prev: { type: "button", position:{ top: 64, left: 0 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onPrev },
stop: { type: "button", position:{ top: 64, left: 64 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onStop },
next: { type: "button", position:{ top: 64, left: 128 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onNext },
continue: { type: "button", position:{ top: 70, right: 10 }, width: 200, height: 52, fontColor: "#fff", backgroundColor: "#2659a4", hover: "#3df", onSelect: onContinue },
renderer: this.renderer
}
First goal : elements objetct, only having elements declaration, rather than a mixed object.
const elements = {
'info'': { type: 'text', value:'' },
'prev': { type: 'button', value:'...', [ ... ] onSelect: onPrev },
'stop': { type: 'button', value:'...', [ ... ] onSelect: onStop },
'next': { type: 'button', value:'...', [ ... ] onSelect: onNext },
'continue': { type: 'button', value:'Continue', [ ... ] onSelect: onContinue },
};
Everything which is not an element attribute should be moved to the config object,
const config = {
/**
* CanvasUI start with a panel having a size of 1x1 meter.
* But you can define your own size.
*/
panel: {
size: {
width: 2,
height: 0.5
}
} ,
/**
* CanvasUI use a Canvas HTML element to renderer widgets into a texture.
*/
canvas: {
/**
* size: canvas size (width, height) of the offscreencanvas, default 512 (pixels)
*/
size: {
height: 128
}
},
threejs: {
/**
* threejs attribute value can be your threejs application instance, on which canvasui will look for a renderer,
* camera and scene attributes.
*/
/**
* renderer
*
* NEEDED if you want to use WebXR controllers (renderer.xr, renderer.xr.isPresenting, getControllers ...)
*/
renderer: this.renderer,
/**
* camera
*
* NEEDED if you want to use the Mouse controller, because the raycaster is attached to the camera
*/
camera: this.camera,
/**
* camera
*
* OPTIONAL if you want CanvasUI to be able to add some extra meshes (i.e raycaster intercept mesh,
* like in the panel scrolling example )
*/
scene: this.scene
}
controller: {
drivers: {
/**
* Mouse driver
* You can use mouse to interact with CanvasUI elements
* this required a camera in threejs.camera config
*/
mouse: {
/**
* enable: true | false, default true
*/
enable: true,
/**
* handler: follow, default follow ('capture' mode not yet implemented)
*
* follow mode just follow the mouse cursor.
*/
handler:'follow'
},
/**
* WebXR driver
*/
webxr: {
/**
* enable: true | false, default true
*/
enable: true,
/**
* raycasterLinesLength: in meters, default 10
*/
raycasterLinesLength:10
},
// @nik, do you see more drivers for later ? gamepad :) ?
},*
/**
* Select behavior
*
* Common settings for both WebXR driver and Mouse driver
* For WebXR driver: selectstart, select, selectend controlller events
* For Mouse driver: mouseup/mousedown/mousemove events
*/
select: {
/**
* Raycaster(s) hover behavior while element is selected
*
* hover : true | false, default false
*
* true: trigger onSelect once (keyboard like behavior)
* false: trigger onSelect while raycaster is moving (select+hover) on the element. Colorpicker or Slider like behavior.
*/
hover: false,
/**
* Select repeater behavior
*
* Only when hover = false.
* Usefull for repeating onSelect event while keeping a select WebXR button pressed or mouse down.
* In keyboard example, it simulate multiple keydown event
*/
repeater: {
* When pressing WebXR select button controller or clicking the mouse on a CanvasUI element,
* the onSelect method of the element is triggered a first time. And that's all when hover = true.
*
* But if hover = false, after the first onSelect method has been triggered, it will optionally wait
* for timeoutms milliseconds before triggering onSelect with a setInterval().
*
* Then if specified, onSelect event will be triggered every repeatms millisecond until
* the selectend or mouseup event is triggered
*
* enable or disable the repeter
* enable: true | false, default false
*
* timeoutms
* 0 mean setTimeout 0 , i.e onSelect will be triggered in the next event loop, similare to setImmediate.
* null or undefined value will trigger onSelect on the same event loop without setTimeout().
* timeoutms: default 100
*
* repeatms
* 0 mean setInterval 0 , i.e onSelect will be triggered in the next event loop, similare to setImmediate. (use with caution)
* null or undefined value will disable the repeater.
* repeatms: default 20
*/
timeoutms:100,
repeatms:20
}
}
}
}
Typical compact/minimal usage :
const config = {
panel: { size: { width: 2, height: 0.5 } } ,
canvas: { size : { height: 128 } },
threejs: {
renderer: this.renderer,
camera: this.camera
}
}
Then, instantiate like this
new CanvasUI( elements , config );
Benefits:
Parameter:
Behaviors:
Reactivity
I am trying to use a isometric camera with CanvasUI for the UI elements but i cannot set the ui closer to the camera.
Say for example we have this code:
const d = 60;
var width = window.innerWidth;
var height = window.innerHeight;
var aspect = width / height;
// add camera
const camera = new THREE.OrthographicCamera(
-d * aspect,
d * aspect,
d,
-d,
1,
1000
);
camera.position.set(22, 22, 22);
ui = new CanvasUI(
{
header: "WHY",
main: "This is the main text",
footer: "Footer",
},
{
header: {
type: "text",
position: { top: 0 },
paddingTop: 10,
height: 70,
},
main: {
type: "text",
position: { top: 70 },
height: 372, // default height is 512 so this is 512 - header height:70 - footer height:70
backgroundColor: "#bbb",
fontColor: "#000",
},
footer: {
type: "text",
position: { bottom: 0 },
paddingTop: 30,
height: 70,
},
}
);
camera.attach(ui.mesh);
ui.mesh.position.set(0, 0, -99); // it does not matter how close or far i set the camera position, it won't change at all. (Z)
ui.add(uiRef.current.mesh);
Pretty sure that i must be doing something wrong, but is what i see all the time:
Just a small rectangle.
Is there a way to use this framework with OrthographicCamera?
In all examples, we have visible scrollbar.
To fix that i purpose to add a common CSS file for all examples
<link rel="stylesheet" href="../../css/reset.css"></link>
having this
body{
margin: 0;
}
canvas {
display: block; /* fix necessary to remove space at bottom of canvas */
}
No more scrollbars
Currently, we are creating Canvas UI widgets by instantiating a Canvas class.
Just thinking to tend to something like datGUIXR.
// only one CanvasUI instance, so we can shared data between widgets (raycaster, xr controllers events ..)
const canvasUI = new CanvasUI();
// then we can add widgets like this
const panelLeft = canvasUI.add(contentPanelLeft, configPanelLeft);
const panelRight = canvasUI.add(contentPanelRight, configPanelRight);
// open the door to new usefull methods ?
panelLeft.hide(); // show() // destroy // move // rotate // setModal // moveForward // moveBackward ..
// globals methods ?
canvasUI.hideAll(); // showAll // destroyAll // enable/visible ..
devrv
build process should optionaly also remove some code, i.e console.log, console.debug.
source code => cleanup =>production code => minify
It should provide both js/jsm files :
CanvasUI.js - bundled production code
CanvasUI.min.js - bundled minified production code
CanvasUI.debug.js - bundled source code
$ npm run build
> [email protected] build
> node node_modules/@eviltik/devsrv/bin/devsrv.js -b && node node_modules/rollup/dist/bin/rollup -c ./rollup.config.js
Sat, 05 Mar 2022 11:57:59 GMT devsrv - info: use config file ..GIT\CanvasUI\devsrv.config.js
Sat, 05 Mar 2022 11:57:59 GMT devsrv - info: build success
C:\Users\T\Desktop\DATA\DEV\GIT\CanvasUI\dist\examples\jsm_src\CanvasUI.js => dist/examples/js/CanvasUI.js...
created dist/examples/js/CanvasUI.js in 929ms
C:\Users\T\Desktop\DATA\DEV\GIT\CanvasUI\dist\examples\jsm_src\VRButton.js => dist/examples/js/VRButton.js...
created dist/examples/js/VRButton.js in 55ms
C:\Users\T\Desktop\DATA\DEV\GIT\CanvasUI\dist\examples\jsm_src\CanvasUI.js => dist/examples/jsm/CanvasUI.js...
created dist/examples/jsm/CanvasUI.js in 200ms
C:\Users\T\Desktop\DATA\DEV\GIT\CanvasUI\dist\examples\jsm_src\VRButton.js => dist/examples/jsm/VRButton.js...
created dist/examples/jsm/VRButton.js in 35ms
should generate dist/
including both concatened jsm and js files, like threejs.
Warning, threejs version will be replaced by the defaultValue
from ./devsrv.config.js
in buildOptions
config.
Can CanvasUI face the camera? Like sprites do?
Plenty of code in examples can be stored in one js class witch can be extended :
App:
Final code example:
import { Environnement } from '../Environnement.js';
import { CanvasUI } from '../../jsm/CanvasUI.js';
class App extends Environnement {
constructor() {
super();
}
createUI() {
...
}
}
The Environnement class will have little methods like setupRenderer, setupCamera, so we can override methods in the main class easily.
@NikLever will you accept this change ?
We are using r119.
In oculus browser, we have a non blocking error
The WebGL context was not marked as XR compatible. Call MakeXRCompatible to mark it as such. This will result in a fatal error in the future. See https://immersive-web.github.io/webxr/#dom-webglrenderingcontextbase-makexrcompatible
We should upgrade at least r125.
I've tested with r135, everything is smooth.
@NikLever are you ok for r135 ?
(master & dev branch)
We can see an Arial font used rather than the targeted one (Gochi Hand)
hi~ i'm starting learning three.js and my English is poor, /(ใoใ)/~~ i hope you can understand my question.
your work is great,and i'm trying to use it, and i click your online version but it seemed don't work,
the result is that:
so i download your code but the result is the same,and it shows that
how to solve them~~~thank you so much
Hi @NikLever,
Congratulations ! this is very clean. I'm a fan ๐ฅ
There is no serious working datui/datguivr on both browser and webxr context today. Something more customizable (styles). Something like UI widgets suite, to make menus, drop down, sliders, color pickers ...)
I plan to make an aframe component on top of canvasui. But not sure. I want to work on some 3D UI widgets. I started with an html2canvas wrapper (aframe-htmlembed-component) , giving some nice results (demo) which is working great with latest aframe version (1.3.0), so latest threejs too.
aframe-htmlembed-component does not support scrolling and i didn't try to use it through a web worker yet.
CanvasUI support scrolling area. Nice shot.
Perhaps we can have really nice (static) effect mixing SVG+CSS (i have to make some tests).
What do you think about adding some features/helpers into canvasUI ?
So we can seriously think about a working DatGUIXR. What is missing ?
Please add the licence you want, i'd like to fork the repo and start playing with this new toy !
example:
functon onSelect() {
}
function onRaycasterEnter() {
// play sound
}
function onRaycasterLeave() {
// play sound
}
mybutton: {
type: "button",
position:{ top: 64, left: 0 },
width: 64,
fontColor: "#bb0",
hover: "#ff0",
onSelect,
onRaycasterEnter,
onRaycasterLeave
},
Just take a look if a bridge with https://github.com/niklasvh/html2canvas is easy to do.
I already used it : it's not perfect, but it make the job.
Don't forget to install eslint globaly
npm install eslint -g
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.