exokitxr / exokit Goto Github PK
View Code? Open in Web Editor NEWNative VR/AR/XR engine for JavaScript ๐ฆ
License: MIT License
Native VR/AR/XR engine for JavaScript ๐ฆ
License: MIT License
Ran the install script with (X)Ubuntu 16.04 xenial x86_64 Linux 4.4.0-122-generic. Extracted the files just fine, when running exokit
however some OpenVR library is missing. Is this expected? Didnt see any prerequisite requirements in the readme.
internal/modules/cjs/loader.js:707
return process.dlopen(module, path.toNamespacedPath(filename));
^
Error: libopenvr_api.so: cannot open shared object file: No such file or directory
at Object.Module._extensions..node (internal/modules/cjs/loader.js:707:18)
at Module.load (internal/modules/cjs/loader.js:589:32)
at tryModuleLoad (internal/modules/cjs/loader.js:528:12)
at Function.Module._load (internal/modules/cjs/loader.js:520:3)
at Module.require (internal/modules/cjs/loader.js:626:17)
at require (internal/modules/cjs/helpers.js:20:18)
at Object.<anonymous> (/usr/local/lib/exokit/native-bindings.js:2:18)
at Module._compile (internal/modules/cjs/loader.js:678:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:689:10)
at Module.load (internal/modules/cjs/loader.js:589:32)
It's almost great, just the ScriptProcessorNode
implementation doesn't respect the buffer size, resulting in hitching across two different breeds of browser.
Fix for this is to use the current frame size (128
) as a basis and queue up until we have enough data in LabSound
to emit an onaudioprocess
.
Related repository is the home environment itself: https://github.com/webmixedreality/exokit-home
Basically vrdisplayactivate
events need to be emitted when you enter a site.
We actually already have a rough implementation of the emit, but we should clean it up to only emit once, at the right time, and preferably only when the HMD is actually nudged in the first place. I think OpenVR supports querying for this.
Additionally, it's worth thinking about how this should work with our reality tabs (tm :P) interface, where multiple sites are opened at once.
Commenting out video support fixes it, but that's not a fix.
Since only node 9+ is supported.
How would I run this normally, I downloaded the git and when I do npm run it doesn't work I did install the dependencies. please help i'm in the slack
https://developer.oculus.com/downloads/package/oculus-sdk-for-os-x/
I have an Oculus DK1 headset. It might be good to support that. I don't know whether DK2 is already supported.
A git submodule pull failed, the build succeeded anyway it seems for windows.
I noticed in some VR game -- I think VRChat -- that you can see the battery level for each controller.
I wanted to add this feature to my own site, but it seems that the WebVR spec doesn't provide the info.
Is there a way battery level could be provided to sites anyway? What would that look like?
We can do something like:
if (navigator.userAgent.startsWith("MixedReality")) { .... }
But that's cumbersome and prone to breakage; probably the worst way to do it.
Currently loading e.g. http://youtube.com/
will lock up the browser, but it should instead display an error in MR.
TypeError: Cannot read property 'responseStart' of undefined
at http://youtube.com/:1:1807
at http://youtube.com/:1:2575
at _runJavascript (C:\Users\avaer\Documents\GitHub\exokit\core.js:3215:15)
at HTMLScriptElement.on.innerHTML (C:\Users\avaer\Documents\GitHub\exokit\core.js:2486:9)
at HTMLScriptElement.emit (events.js:182:13)
at HTMLScriptElement.EventEmitter.emit (domain.js:442:20)
at HTMLScriptElement._emit (C:\Users\avaer\Documents\GitHub\exokit\core.js:1343:40)
at HTMLScriptElement.set innerHTML [as innerHTML] (C:\Users\avaer\Documents\GitHub\exokit\core.js:2528:10)
at HTMLScriptElement.run (C:\Users\avaer\Documents\GitHub\exokit\core.js:2549:24)
at element.traverseAsync (C:\Users\avaer\Documents\GitHub\exokit\core.js:3186:16)
We already have Magic Leap + Leap Motion APIs in the code ๐. However, what we're lacking is site examples showcasing the awesomeness within.
There's some of this stuff in our fork of THREE.js.
In particular;
We could use more examples, as well as a structured place to publish them. Either a site or a dedicated repository.
https://threejs.org/examples/#webvr_vive only loads in left eye of Vive, assumed true for some other examples
It we don't do this, the OSX app will feel un-OSXlike and hang around in the taskbar.
[email protected] postinstall C:\Users\LucidWeb\Downloads\node_modules\exokit
> node downloader.js
[โโโ.................] exokit-windows.zip v0.0.36 2778194 bps 16% 68.8sevents.js:183
throw er; // Unhandled 'error' event
^
Error: invalid distance too far back
at InflateRaw.zlibOnError (zlib.js:153:15)
exokit.cmd . https://aframe.io/aframe/examples/animation/aframe-logo/?ui=false
Crashes.
We don't do 2D rendering in the Exokit core, and we probably never will. The future is 3D.
The world needs to catch up, and most of the web is still 2D. So a common request from Exokit is to render 2D sites as a plane. While we don't want to get into implementing the 2D part of the web ourselves, it's not too hard to use another 2D browser engine like Chromium and frame it as a plane in 2D.
We've done this before in Zeo, so we know it's doable. Basically we just need to open a true Chromium instance and feed the framebuffer through to Exokit's GL as a texture. With something like Puppeteer and simple plan intersections emulating mouse/touch events, we would get pretty close to having a 2D browser environment in our 3D browser environment ๐ป.
We should figure out what the Leap Motion API looks like, as exposed to the use sites.
Ideally we don't stray too far from WebVR/WebXR/device API. Whatever we do should be thoroughly documented on http://docs.webmr.io ๐
We don't have an Oculus headset in the office right now, though it should be supported via OpenVR.
It would be great if a volunteer could check if the basics work in the Oculus VR environment, and file any gotchas/bugs to the main issue tracker on Github.
Exokit currently only supports the webgl
context. WebGL2 adds a lot of nice stuff to that API.
Worry not! This actually is much easier than it sounds; we already use straight up OpenGL 3.2 for out WebGL backing, which is already a superset of what we need for WebGL2. So implementing WebGL2 would be mostly a matter of exposing/defaulting functionality in Javascript per the spec, rather than implementing much native code.
We should try to follow the WebGL/WebGL2 exposure rules in the spec as much as possible.
Exokit does a lot of stuff, such as Magic Leap, Leap Motion, and more. It also runs a ton of VR sites, faster than other browsers. None of this is really mentioned anywhere, and there are no pretty pictures of it in the README.md
. Let's fix that!
Basically, it would be great if our frontend content like the README.md
had a lot more meat ๐. This is a fairly open-ended issue.
Exokit essentially reimplements all of WebGL.
We have bugs and regressions with various sites as our code changes, which it does frequently! Some of these are soft-crashes in JS (throws), some is hardcrashes in the native bindings (SEGV, hangs), and some is incorrect rendering.
We could automate checking a list of sites against a build of Exokit in our CI environments on Appveyor and Travis CI.
In particular it seems it would be straightforward to check that some sites load and run to completion -- the harder part of this is differential rendering verification, which we might want to punt on till later.
One wrinkle is that we might want to make this work for all three OSes we support (Windows, OSX, Linux), so we might need three harnesses. Also, since Exokit is a graphical app that requires either OpenGL or an emulation of it (e.g. mesa
on Linux), we'll need to make sure the CI environment runs such support.
This could be broken up into separate issues if this proves to be too much to chew at once ๐.
The three.js examples are a goldmine of simple but thorough test suites.
The ratio is high but not all of them examples load and render. However, those that do should continue to do so and can be whitelisted for verification.
For responsive triggers/buttons and tooltips. Plus make it actually match the hardware.
Need to expose a web API for it.
Exokit supports <video>
tags via its implementation of HTMLVideoElement
.
These are backed by the nativeVideo
binary binding in deps
.
Testing example:
https://threejs.org/examples/?q=vide#webgl_video_panorama_equirectangular
A related nice-to-have:
https://threejs.org/examples/?q=vide#webgl_materials_video_webcam
But this probably would be more complicated, to acquire the webcam feed via getUserMedia
.
Would be nice if this worked:
exokit https://www.youtube.com/watch?v=dQw4w9WgXcQ
It could open the video in desktop mode, with enter VR/AR capability.
Missing node? Probably needs a full build download.
>exokit
C:\Users\Noah Schiffman\AppData\Roaming\npm\node_modules\exokit>IF EXIST ".\node\node.exe" (.\node_modules\isolator\lib\windows\isolator.exe -- .\node\node.exe . ) ELSE (.\node_modules\isolator\lib\windows\isolator.exe -- node . )
*os.PathError fork/exec .\node\node.exe: Access is denied.
C:/Users/Ultimus/go/src/github.com/modulesio/isolator/runner/winsandbox_windows.go:133 (0x8b9e12)
C:/Users/Ultimus/go/src/github.com/modulesio/isolator/cmd/runner/runner.go:156 (0x8bbd2d)
C:/Users/Ultimus/go/src/github.com/modulesio/isolator/cmd/runner/runner.go:36 (0x8bb592)
C:/Users/Ultimus/go/src/github.com/modulesio/isolator/main.go:236 (0x90d919)
C:/Users/Ultimus/go/src/github.com/modulesio/isolator/main.go:125 (0x90cf8a)
C:/Go/src/runtime/proc.go:198 (0x4306fe)
C:/Go/src/runtime/asm_amd64.s:2361 (0x45cbc1)
Embedding Exokit in a nodejs project or running it directly from the node prompt does not work.
Code snippet:
Enter node
and then paste
const exokit = require("exokit");
exokit.load('https://threejs.org/examples/#webgl_animation_cloth', {})
This one's a doozy.
But since Exokit is written in Javascript, we already support WebVR, and we have access to native OpenVR, and I envision a great deal of copypasta from our WebVR/OpenVR support.
I (@modulesio) will probably tackle this.
We have supported the Magic Leap simulator for a while now. We will also support the real hardware when it ships!
However our approach/API exposition is not documented anywhere. We should clean up the API to conform to web expectations, with minimal deviation from WebXR, and then write up the documentation on https://docs.webmr.io ๐.
We currently use both Appveyor and Travis CI (OSX, Linux) for building, verifying, signing, and publishing our releases.
The releases are published to Github and that is where they are pulled from when a user hits https://get.webmr.io/.
We need to document this flow on http://docs.webmr.io/. Or rethink it if there's a better approach!
Streaming is a big thing. I spend more time on Twitch than I'd care to admit.
It would be good to harness that wave by adding stream integration into Exokit.
The integration should be developer-focused at first, since those are who we want streaming Exokit anyway. Maybe we can bundle OBS and make it a one-click setup?
Need to mention:
windows-build-tools
npm i exokit
C:\gh\exokit>node .
> fetch('https://mdn.github.io/fetch-examples/fetch-array-buffer/viper.ogg').then(x => x.arrayBuffer()).then(x => { console.log(x instanceof ArrayBuffer); })
Promise { <pending> }
> false
Would be nice for development if npm install
built only changed files.
Currently our builds will error during a PR build: https://travis-ci.org/webmixedreality/exokit/builds/377086347?utm_source=slack&utm_medium=notification
This is by design: the PRing Github account does not (and should not) have access to secure variables in the build, such as our code signing keys. But it would be nice for the build to not fail, since build success is highly useful info for PRs.
The straightforward fix for this would be to skip the code signing steps (for Windows on Appveyor and OSX on Travis CI if/when the PFX_KEY
environment variable is not defined -- and maybe log the fact that this is happening for later debugging in case this behavior leaks into master
.
CC @shawwn
The current best solution for a multipackage monorepo is lerna via Yarn Workspaces.
window-worker
is responsible for this functionality.
This can be repro'd with truncated scripts.
We currently support 5 platforms, with platform-specific repos;
https://github.com/modulesio/exokit-windows
https://github.com/modulesio/exokit-macos
https://github.com/modulesio/exokit-linux
https://github.com/modulesio/exokit-android
https://github.com/modulesio/exokit-ios
There are also a couple of helper repos:
https://github.com/modulesio/exokit // this one, javascript/DOM parts
https://github.com/modulesio/exokit-bindings // native bindings
But there's no good reason to have these hanging around long term -- most of the code is exactly the same, on all platforms, even the native parts. Plus, it would be nice to just npm i exokit
anywhere.
This mainly requires figuring out:
npm install
buildThere's some delicious low-hanging fruit in our docs site ๐. Although there is a good framework, we could both clean up and expand the site for commonly asked pointers.
We currently use docsify
as a Github page for the docs, but that's amenable to change. Whatever we use should be automatic and ๐ค.
1.0
?<title/>
tag?<meta/>
tags?Submit a PR to webmixedreality/webmr-docs ๐ฅ
Current site: https://exokit.webmr.io/
Needs work! We need to expand the site with:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.