xml3d / xml3d.js

The WebGL/JS implementation of XML3D

xml3d.js

XML3D

xml3d.js is a XML3D implementation based on WebGL and JavaScript. The aim of XML3D is to make the development of 3D-Web applications as easy as developing web pages. Every web developer who knows how to use the DOM (or jQuery) should also be able to use XML3D.

XML3D is also an evaluation platform of the W3C Community Group Declarative 3D for the Web and a FIWARE Generic Enabler.

Examples

Our small examples repository holds simple examples of individual features. Below are some more complete demos:

Usage

Download the library and include it in your html page.

<script src="http://www.xml3d.org/xml3d/script/xml3d.js"></script>

If a standard navigation mode is sufficient for your web application, you can include the camera controller that comes with xml3d.js:

<script src="http://www.xml3d.org/xml3d/script/xml3d.js"></script>
<script src="http://www.xml3d.org/xml3d/script/tools/camera.js"></script>

Testing

We have an extensive test suite and some known issues.

How to build

Dependencies

You will need node.js. Then install the grunt cli:

npm install -g grunt-cli

Clone

Clone a copy of the main xml3d.js git repo by running:

git clone git://github.com/xml3d/xml3d.git

Build

Enter the xml3d.js directory and run the build script:

cd xml3d.js && npm run build

The built version of xml3d.js will be put in the build/output/ subdirectory. Alternatively you have several grunt tasks that you can run directly (e.g. grunt min). Run grunt --help to get a list of available grunt tasks.

Documentation

We have an overview documentation in each subfolder of the project:

• build - The build system of xml3d.js.
• doc - Documentation related to FIWARE
• spec - The specification for XML3D in re-spec form
• src - The actual source code of the xml3d.js library
• tests - The test suite
• tools - Several tools that can be used optionally with xml3d.js

In addition a full specification and API description is provided, which also includes many examples and code snippets. This repository's Wiki also contains some tutorials and additional help.

Change log

5.2 - 10.05.2016

• Support for Web Components - demo 1, demo 2 spec, wiki
• CSS transforms now support CSS transitions - demo, spec
• Materials can now change certain WebGL states during rendering - demo, wiki
• Additional example scenes: STL plugin (link), custom material (link)

5.1 - 07.01.2016

• Resource system is now based on the Fetch API and Promises
• Better FormatHandler interface for custom mesh loaders - wiki
• Support for the CSS z-index property - demo, spec
• New onRequest interface for custom HTTP headers - wiki
• Memory performance improvements when removing elements from the DOM

5.0 - 16.10.2015 (Not backwards compatible!)

• New specification and API description - link
• Support for CSS 'display' property - demo, doc
• Improved math types - doc
• Simplified light definitions - 111
• New 'string' data type in Xflow - doc
• Various API changes - doc
• Improved standard camera controller (camera.js) - link

---

4.9 - 31.03.2015

• External Xflow Operators - demo, doc
• IE 11 Support
• Grunt build system now based on Browserify
• Source Mapping for the xml3d-dev build
• Bugfixes: 37, 73, 87, 88, 92, 98, 100, 104

4.8 - 18.12.2014

• Recursive Assets - demo, doc, issue
• Load Events - doc
• iOS 8 Support
• New build system based on grunt
• Improved texture filtering - doc
• Draw multiple line strips in a single mesh <mesh> - demo, demo, doc
• Bugfixes: 9, 46, 56, 74, 75, 76, 77, 79, 82

4.7 - 17.10.2014

• Assets / Configurable Instances - demo, slides
• shade.js integration (beta) - demo, slides
• Shadow Maps - demo
• More options for texture filtering
• Bugfixes: #69, #71

4.6 - 15.04.2014

• Custom RenderTrees and the RenderInterface
• Added Screen-space Ambient Occlusion to standard render pipeline
• Added system to set renderer-specific options
• Infrastructure for WebCL-based Xflow operators

4.5 - 14.11.2013

• Full support for HTML encoding, all demos in HTML now
• Set data values efficiently using TypedArray - demo, doc
• Override shader attributes on a per-object basis - demo
• Improved Error Messages
• Reuse of Xflow graphs using new <dataflow> element - demo, doc
• Many performance improvements, e.g. Frustum Culling and Paging
• Support for mult-touch events
• Dynamic near/far clip planes that adapt to scene size
• #24: WebWorker support for MeshLoader plug-ins - demo
• #25: Smarter handling of cached resources

4.4 - 23.04.2013

• Image Processing with Xflow (also as standalone library) - demo
• Transformations as Xflow sink
  • Augemented Reality (AR) - demo
  • Keyframe animations
• Generalized resources for meshes, shaders, etc.
  • Support for XML and JSON
  • External loader plug-ins: OpenCTM and MeshLab
• Xflow API: Observer Xflow graph from JavaScript
• Canvas resizing demo
• Issues: Fixed #2, Fixed #3

4.3 - 18.12.2012

• Xflow support, including
  • Skinning - demo & demo
  • Sequential Morphing - demo
  • Mechanism for custom operators - demo & demo
  • Prototypes - demo
• External references in XML format - demo
• 'onload' event for <xml3d> element
• Support for video textures - demo
• Support to use a webcam stream as video texture via WebRTC API - demo
• Support of spot lights - demo
• Support of CSS 3D Transforms in style attribute - demo
• Improved debug output

4.2 - 14.09.2012

• Hardware accelerated object picking expanded to 16,7 mio objects
• Emissive texture map support in diffuse and phong shader - demo
• Specular map support in phong shader - demo
• New mechanism for custom shaders - demo
• Support of directional lights (finally) - demo
• Support of external data resources in JSON format - demo
• New mechanism to register loaders for external formats - demo

4.1 - 19.07.2012

• Initial release on GitHub

xml3d.js's Issues

Error when style="display:none;" on configuration

console:

WebGL: checkFramebufferStatus: attachment has a 0 dimension
Incomplete framebuffer: FRAMEBUFFER_INCOMPLETE_ATTACHMENT

Sometimes you may want to start loading everything in the background and only begin showing the scene after everything has arrived.
Current workaround is to style="position:absolute; opacity: 0;" so the canvas doesn't impede layout flow and is not visible, but still passes configuration.

Firefox Support of Xflow's AR-Demos

My browser doesn't support XML3D. What to do?

The error message tells me to "Please visit http://www.xml3d.org to get information about browsers supporting XML3D."
However, a search on xml3d.org for "Browser" reveals little to no information.
The error message should be more informative and/or there should be a dedicated link that leads to a browser compatibility page.

camera.js prevents some events from firing

While using http://www.xml3d.org/xml3d/script/tools/camera.js as a camera controller some events seem to be prevented from firing when using Chrome. This behaviour did not occur when using Firefox.

In my case I was attaching a change event to a listbox which only fired when dragging to select an option, not when simply clicking on an option.

I was able to track this behaviour down to lines 190 & 191 where mouse events are attached to the document.
Replacing document. with this.canvas. in lines 190 & 191 fixed the problem.
This however prevents the camera from being rotated when the mouse leaves the canvas :)

Feature: Projected object locations

From CSSOM, HTMLElements have

partial interface HTMLElement {
  readonly attribute Element offsetParent;
  readonly attribute long offsetTop;
  readonly attribute long offsetLeft;
  readonly attribute long offsetWidth;
  readonly attribute long offsetHeight;
};

It would be very convenient if XML3D could provide these as well for scene elements. (Think text flow around single objects, positioning of supporting 2D content alongside, etc.) The values could be calculated by performing a projection calculation on the bounding box, for example.

Add eyelight shader and use it as default

I think an eyelight shader as default shader would be nice.
It makes the 3D shape more recognizable than simple flat shading or phong shading without light sources.
This is especially useful for first steps with XML3D.

getBoundingBox of mesh and group are empty immediately after addition into DOM

The method getBoundingBox on group and mesh elements don't work probably immediately after adding a mesh and return an empty bounding box. Only after a render pass, the information is correct.

Adding lights dynamically through JS does not work

Adding a light via createElementNS() does not work. Removing one through removeChild() also doesn't work.

getBoundingBox not in local space

According to the specification getBoundingBox always returns the bounding box of the element in local space.
However, calling getBoundingBox on a group element returns a bounding box with values that are obviously in world space.

xml3d.getElementByPoint() with absolute coordinates

getElementByPoint() needs to be called with x and y relative to the xml3d element. This is
okay, as long as I'm using something like jQuery's offset() to obtain the relative coordinates.
If I don't have that at hand I have to find a way to calculate the offset or even if I have it,
I find it tedious to always have to convert the coordinates first.

I would find it great if we could just pass it values, that are in a mouse event. document.getElementFromPoint() (https://developer.mozilla.org/en-US/docs/DOM/document.elementFromPoint) is a similar method and here we can pass in directly an event's pageX/pageY coordinates.

Improvement: Interface of XML3DRotation.setQuaternion() confusing

It takes a vector and a float, but that's not an axis-angle. It should take just 4 float values.

After multiple picks picking fails

While the first pick is correct, consequent picks fail. This affects manual picking (e.g. xml3d.getElementByPoint) as well as events (e.g. "mousedown").

Firefox can't handle relative refs in external remote files.

Chrome and Firefox behaves differently when loading .xml files from remote server using remote refs in meshes and shaders. For example appending xml3d element with this group:

<group id = "someId" shader="https://someUrl/someModel.xml#material">
        <mesh src="https://someUrl/someModel.xml#someMesh" type="triangles"/>
 </group>

with the someMesh part in the someModel.xml file:

<data id="someMesh">
    <float3 name="position">
    ...           
    </float3>
    <float3 name="normal">
    ...           
    </float3>
    <float2 name="texcoord">
    ...          
    </float2>
</data>
<data id="subMesh">
    <data src="#someMesh"/>
        <int name="index">
        ...       
        </int>
</data>

works fine in Chrome, but fails when using Firefox. Everything works fine when the ref in subMesh is absolute:

<data src="https://someUrl/someModel.xml#someMesh"/>

Same thing happens when the shader has a texture which has a relative reference to some image.

onload-Event on meshes

It would be great to have an onload-Event on meshes. In particular, I want to be notified when the external reference of a mesh is resolved and the renderer went over it once (i.e. the bounding box is valid and the mesh is visible for the user).

querySelector with ids not working for xml3d elements in Firefox

Using document.querySelector does not work for xml3d elements when using Firefox and using id as selector.

How to reproduce:

• go to http://www.xml3d.org/xml3d/demos/25_Chess/
• open js console
• type document.querySelector('#transform_rookBlack1')
  • in Chrome: returns the element as expected
  • in FF: returns null

URI Handling in VideoDataAdapter

The Line 15452 in xml3d.js of the VideoDataAdapter
var uri = new XML3D.URI(url).getAbsoluteURI(this.node.ownerDocument.documentURI);
corrupts the WebRTC URL in Firefox like
mediastream:c7180ca8-6363-4571-bc95-681a8cbe6d1b

Xflow: time data source

Another feature request, this time for xflow. An operator yielding time would be very neat, so I could do

<data compute="transform = xflow.createTransform(translation, rotation)">
    <data compute="animationKey = xflow.time() % 10.0" />
    <float4 key="0.0" name="rotation">0.0 0.0 0.0 1.0</float4>
    <data compute="translation = xflow.lerpSeq(translation, animationKey)">
        <float3 key="0.0" name="translation">0.0 0.0 0.0</float3>
        <float3 key="5.0" name="translation">0.0 100.0 0.0</float3>
        <float3 key="10.0" name="translation">0.0 0.0 0.0</float3>
    </data>
</data>

where the resulting transform would repeatedly move 100 units along the y-axis up and down in 10 seconds time.
This is similar in usage to Quartz Composer's Patch Time.
Note that I have blindly assumed the existence of a modulo (%) operator and possibly abused the syntax due to lack of documentation ;)
PS: I have a ton more of these, maybe it wold be easier to document how to write new operators?

Feature request: orthographic view

Orthographic views are quiet common in scene editing tools.

The caramel editor would benefit from this feature.

Smarter handling of cached resources

Currently the XHR request gets cached as it could be reused in structured file formats that allow to adress multiple objects within one resource. However, the cached responses claim a lot of memory.

The desired behaviour:

1. Don't cache responses for unstructured file formats (e.g. OpenCTM, xml3d-json)
2. Add heuristic that manages freeing the response for structured resources. A simple one could be to free the resources once there is no request in the queue that requires the resource. Another options include using a LRU or LFU strategy

adapterHandler.status of cached adapters for data nodes still NOT_FOUND after re-inserting node

Re-Inserting a node to the DOM that was previously deleted does not change the adapter status to LOADING or READY and throws a "node not found" error if it is referenced by another node later

How to reproduce the error:

1. Enter a node to the DOM via JavaScript
2. Enter a node with a node that references this data node to the DOM
3. Remove the node from the DOM via JavaScript
4. Remove the node from the DOM via JavaScript
5. Perform step 1 with same node ID.
6. Perform step 2 with same reference.

Step 6 will fail with error message "Could not find element of url #dataID for src". The node will be present in the DOM, but still marked as NOT_FOUND in the adapter entry which is cached in c_cachedAdapterHandlers in ResourceManager.getAdapterHandle . Check for the adapter status then throws the error.

Expected behaviour : After re-inserting the data node, it is available to be referenced again.

Improve Usability of XML3DVec3 and XML3DMatrix types

On the one hand, we don't provide common methods like Vector to Matrix multiplication, on the other hand, we make it particular hard to convert from XML3D Data types to array based datatypes like glMatrix.

• we should extend our matrix interfaces (I would like to see at least a Matrix Vector multiplication option)
• we should get a way to access the datatypes as arrays (like a getArray method) that allows to a quick conversion to other libraries like glMatrix.

WebWorker support for MeshLoader plug-ins

Exploit WebWorker to decode mesh data in handlers. This is in particular important for mesh formats that are expensive to deccode, for instance LZMA encoded OpenCTM resources.

Shoudn't be a big issue as Xflow has the capabilities to asychronously fill the data.

Xflow: Internal type for grayscale images

We should support storing grayscale images more efficiently.
Currently we only support image with RGBA colors. For grayscale images, we only use 25% of the memory. We probably should introduce fake ImageData for grayscale images that are mapped to proper RGBA images if required.

Since it is not possible to declare grayscale images with XML3D (the Web only understands RGBA), we will only have grayscale images as internal type.

Implement efficient interface to set Xflow data using types arrays

Why do we need this?
For several projects we need to implement particle effects with XML3D. This is currently hard to achieve with Xflow for arbitrary effects. A good work around for now is to animate particles with JavaScript. However, it is inefficient to set XflowData by converting float data to text to set the content of a value element (such as <float3>). An interface to set the data of a value element directly with a types array would be way more efficient.. We have had this idea a long time ago. It's time we implement it.

Proposed interface:

The following interface will be implemented by all value elements:

var valueElement = document.getElementByTagName("float3")[0];
// set data of value element
valueElement.setOverwriteValue(someTypedArray);
// clear the override value:
valueElement.overwritten = false;

When calling any of these funciton, the corresponding value element is modified to reflect the changes as following:

1. The value element without overwritten data:

<float3 name="position" >0.5 1.5 -0.3 ... </float3>

1. The value element after data has been overwritten

<float3 name="position" overwritten="true" >0.5 1.5 -0.3 ...</float3>

Note that we don't remove the originally declared data or replace it. We only add the overwritten attribute to mark the element accordingly. This way, we don't have any major overhead when setting the data, but still some reflection in the declarative form.

1. After clearing the element's overwrite value:

<float3 name="position" >0.5 1.5 -0.3 ...</float3>

The overwritten attribute is removed, the declarative content did not changed and is used again.

The overwritten attribute can be set as well, this has the following semantic:

var valueElement = document.getElementByTagName("float3")[0];
valueElement.overwritten = true; // original value is overwritten, but no value is specified.
// consequently, the element will have no value
valueElemet.overwritten = false; // original value is not overwritten anymore. If an overwrite value was specified before, the corresponding typed array is detached

Some more examples for detailed behavior;

var valueElement = document.getElementByTagName("float3")[0];
valueElement.setOverwriteValue(someTypedArray);  // value = someTypedArray
valueElement.overwritten = true; // value = someTypedArray (nothing changed)
valueElement.overwritten = false; // value = original value;
valueElement.overwritten = true; // value = null (someTypedArray has been detached)

Accepted values:

float, float2, float3, float4, float4x4: Float32Array
int: Int32Array
bool: ByteArray (?)
texture: Image, or Canvas or ImageData?

Discussion:

• Don't drop previously added typed array when specifying: overwritten=false (I think this is bad for memory management)
• Better a special function to clear OverwriteValue? (e.g. valueElement.clearOverwriteData() )

getters & setters for referenced elements

It would be nice to be able to call something like group.getTransform() or maybe even simpler group.transform to be able to manipulate the referenced element. This also applies to all other referenced elements such as shader or data for all elements that reference these.

Usually this can be done by simply removing the # in front of the reference and do a document.getElementById but this gets increasingly difficult with external resources, especially if those are not in xml format. Having an interface provided by xml3d.js would remedy this problem.

wiki typos

Just noticed a couple typos in the Wiki docs as was reading them -- actually fixed the other in my fork but apparently there's now way to do pull requests from github wikis so the recommended way is to just file an issue..

Is trivial: 'then' -> 'than' in "To work with XML3D, you need to understand the set of XML3D elements. Good news: there are only few of them. Currently less then 20." in the getting started doc, my fixed version is at https://github.com/antont/xml3d.js/wiki/Getting-started

I think there was another in the 'Multidata' idea doc but was really minor and that's only an idea doc so doesn't matter, I also forgot what it was and couldn't find quickly again.

But the getting started one is quite prominent so perhaps nice to fix.

FR: Proper handling of xml3d node removal

A proper handling of an xml3d node removal should be ensured. Currently, there seem to be some issues with the resource manager whenever an xml3d element is emptied. Even if you remove an xml3d element completely and add a new element with the same name, some artefacts from the old element remain somewhere.

Raycasting not yet implemented

The xml3d element has a method getElementByRay(), but it is not implemented. Raycasting is a pretty fundamental feature, it would be very nice to have.

Unloading of unused web resources

Resources are cached in an optimized data structure because they can be used from multiple places inside the scene. The resource manager should track all users of a resource and free the cached resource as soon as no user exists for the resource anymore.

Make the WebGL not supported screen configurable

Currently a red screen appears when WebGL is not supported. Make this configurable.

Support array of textures

It would be good for image processing operators to access/output array of textures.

We can declare array of textures by having several images inside a element:

<texture name="images" >
  <img src="image1.png" />
  <img src="image2.png" />
  <img src="image3.png" />
</texture>

Images in the array can be of different size.

Xflow operators operating on images should be always compatible to array of images.
In that case you need to iterate over each image like you would iterate over an array of e.g. float3. That would also create meaning for the 'array' access attribute for texture inputs.

This features requires a lot of modifications in Xflow/XML3D but it would probably be the correct way to do it, as it is consistent with the iteration concept of other data types.

Array of images are also important for image processing operators that are required to output a dynamic number of images in certain situations (e.g. image pyramids).

Throw an error if two xml3d.js files have been included in the same document

This might happen more often than you think and this error is pretty hard to spot

Transparency buggy for shader overrides

When overriding the "transparency" value of a shader from a mesh, it is not correctly evaluated. The rendering object doesn't seem to be correctly marked as transparent and the transparency itself seems to be invalid (it doesn't really match the provided value).

Disabled face culling

Hi,

I noticed that face culling has been disabled everywhere in the XML3D.js renderer, is there a reason for this? Usually one wants to enable at least backface culling by default.

Feature request: reference group nodes

Similar to being able to reference shaders, transforms and meshes it should be possible to reference group nodes. Already in two applications we had to implement an xml parser that reads the group hierarchy and instantiates it in the target scene.

The referenced hierarchy can be completely inaccessible, I just want to instantiate an asset so to say.

Handlers within XML3D stop 'DOMNodeInserted' and 'DOMNodeRemoved' from propagating further.

As I had been browsing the source code, I found this issue is already documented here, so I thought it would be good if I put it in the issue tracker also.

Currently it is not possible to listen to DOMNodeInserted and DOMNodeRemoved on a single XML3D element, for example as in my case, listen to the node changes (adding / removing of children) of a <group> element.

Extending Array prototype creates error

Whenever I try to extend the Array prototype the following error repeatedly occurs:

TypeError: Cannot call method 'getProgram' of undefined
    at XML3D.extend.renderObjectsToActiveBuffer (http://localhost:8080/xml3d/js/xml3d-4.5.1.js:23663:46)
    at XML3D.extend.renderScene (http://localhost:8080/xml3d/js/xml3d-4.5.1.js:23622:26)
    at XML3D.extend.renderToCanvas (http://localhost:8080/xml3d/js/xml3d-4.5.1.js:23145:39)
    at CanvasHandler.draw (http://localhost:8080/xml3d/js/xml3d-4.5.1.js:19319:39)
    at tick (http://localhost:8080/xml3d/js/xml3d-4.5.1.js:19220:22) 

Feature request: multiple views on the same scene

In 3D editors it is natural to have multiple views on the same scene. It would be nice to have a feature in XML3D, that enables this. Right now the only way I can think of is to copy an existing scene to another tag, make a different view active there and then synchronize the changes. But this is a lot of effort and not fast at all I think.

Userdefined near and far planes in view frustum

Having a large scene can cause a lot of z-fighting problems with small objects.
It would be nice to be able to define the near and far values of the view frustum via parameters of the view node.

Minified build of develop branch fails

The minified develop build fails after commit 6cd5f13

Drawing complex meshes is not supported

We have a case where a triangulated plane containing a quite large amount of vertices is created and it should be rendered with XML3D.js.

However, because WebGL cannot handle so many vertices in a single draw call and XML3D seems to try to draw the whole mesh at once, the drawing will fail. One solution would automatically splitting the mesh in smaller chunks of max 65535 vertices and then drawing the chunks one by one.

I checked out the code in mesh.js and it seems that you have sketched some code there related to segmented meshes, but it seems unfinished. It would be nice if you could finish this feature so we can render larger meshes with XML3D.js more effortlessly.

State progress tracking

In relation to #11, it would be nice to have a decent progress model in the specification.
Seeing how arbitrary data can possibly be streamed in asynchronously it is currently very hard to programmatically determine what state a scene currently is in.
A solution analogous to HTML5 Media Ready States or Progress Events would be nice.
As a minimum, it should be possible to determine when a scene has finished all pending asynchronous operations and is ready to render the next frame.

"visible" attribute in the <group> element is not taken into account

It is not parsed when new groups are dynamically added or when it is dynamically set. I didn't check if it's correctly handled when an entity is already present in the scene.

After executing either this:

groupNodeInScene.setAttribute("visible", "false");

or this:

var newGroupNode = XML3D.createElement("group");
newGroupNode.appendChild(someMeshWithGeometry);
newGroupNode.setAttribute("visible", "false");
xml3dRootElement.appendChild(newGroupNode);

the added group is still visible.

Events for loading external resources

I am looking for a way to attach an event handler to an externally referenced resource such as a mesh or rather to its parent group.

Currently whenever I try to access information of dynamically added external resources I get an error (in my case calling getBoundingBox on the group containing an external mesh).

Is there a way to bind an event handler to a group containing external resources that gets called as soon as all external resources are loaded and available?

Orthographic Camera Support

It would be awesome to have an option, e.g. on the xml3d tag, to select an orthographic camera mode.

Make attribute-names case-insensitive

The parsing of the attribute names, at least of the xml3d element, seem to be case-sensitive. The following piece works as expected, the referenced view is the active one.

<xml3d activeView="#myView" />

The following however does not work:

<xml3d activeview="#myView" />

Support for Texture Compression

xml3d.js supports the standard browser image formats as textures. Some platforms support compressed texture formats as an WebGL extensions. Most desktops support e.g. S3TC format.

xml3d.js should:

• Support proper setting of textures delivered in a compressed texture format
• Decode compressed textures if used in a Xflow dataflow that requires to access pixel data
• Encode supported texture formats in the HTTP request thus the server can send textures in the best available representation

Library that could be used:
https://github.com/toji/webgl-texture-utils

How to export animation?

I've seen the examples using animation and skinning and was wondering if there is any xml3d exporter capable of animation?
Both the c4d and the blender exporter haven't been updated in a long time and do not include animation export, and animating complex models (such as those used in the examples) just using a text editor is out of the question.

Shader overrides not working when using src attribute in mesh

Shader overrides are ignored when the meshdata is referenced using the src attribute of the mesh node instead of using a child data node.

Feature request: runtime visibility checking

The grand vision for this feature would be to be able to say

$("#some_xml3d_element").is(":visible");

and have it do the right thing. According to jQuery's release notes, their calculation is

an element is visible if its browser-reported offsetWidth or offsetHeight is greater than 0

In general, having offset{Width,Height} do the right thing would be very nice, see #13, but that probably won't suffice, given that geometry can still be obscured. Maybe some duck-punching of jQuery will be required.

So far, a "workaround hack" is to manually perform the calculations for projection into screen-space and then check via getElementByPoint() if the object is actually visible.