godotvr / tiltfivegodot4 Goto Github PK

One of the issues I seem to have run into while streaming, though I could be wrong, is that when I changed world scale the IPD seemed wrong.

I'll investigate more soon

It would be convenient for users to be able to see the dimensions for the game board in the editor scene. There sould be a widget node that can display the board.

Need to add support for number of active players the game will support. 1-4
Also maybe support for spectators with a different glasses scene.

The plugin needs the control string mapping someplace.

The Tilt Five glasses have a tangible tracking camera. The intent of this camera is to allow mixing of physical media, like cards or figures, with the digital game. The frame capture from this camera is exposed via the NDK. This functionality should be exposed to Godot.

Special consideration should be given to feeding this video stream to libraries like OpenCV for image processing.

This is absolutely a nitpicking issue and I'll find an hour somewhere to polish this up. Just recording this as something to remind me :)

The design of the XRServer is such that XRInterfaces are registered early, and the initialize/uninitialize function build up/tear down the system. For TiltFive its the other way around.

There is a bug here that is actually producing better looking results.

These lines revel a bug where the system was not advancing though the swap chain. the line _current_frame_idx = (_current_frame_idx + 1) % _swap_chain_frames.size(); really needs to be moved to just under the t5SendFrameToGlasses call.

I did try this fix but discovered that weird tracking distortions appeared in the Vulkan output. I'm not sure what the problem is yet. It could be that something about this config is fighting with the predictive pose estimation in the glasses or it could be that we need to set up some Vulkan barrier before submitting the frames.

So as it stands only one frame in the swap chain is being used and that bug actually produces the best results under Vulkan. There is still some small jitter in the Vulkan output so it seems there is still some sort of problem there.

Make whatever code and build changes to support Android devices.

The players glasses and wand currently cast shadows on the board, we should mark these objects as not casting shadows.

Make whatever changes to the code and build to support Linux.
I've got a Linux version building but Tilt Five doesn't have drivers for Linux that match the NDK 1.4 beta yet.

Will be experimenting with this more but one thing I really missed was a way to react when a glasses scene was added and/or removed. We should probably emit a signal from the T5 manager.

We also need to look into easier ways for the glasses scene to communicate with the main playing logic.

The csharp code need documentation strings

To correctly place the camera node in the world XRCamera3D binds itself to the head tracker (which is missing from the plugin). Other than that its near and far is used when rendering. In Godot 3 it had further functionality that has now been removed so it doesn't do that much more than the normal camera, all the real magic happens in the XRInterface.

So for correctly placing the camera nodes we could introduce trackers for each glasses and create a T5Camera3D node that simply inherits Camera3D, binds to the correct tracker and updates its position accordingly.

Whether this will have a lot of benefit I don't know, it doesn't seem common for the form factor to have elements childed to the glasses, but it will allow us to show glasses (optionally) in a spectator view.

plugin.gd class should be renamed TiltFivePlugin.gd. The same should be done for C#

Check for XR shaders when the plugin is enabled

Going to need to be able to fill out this T5 structure

void GodotT5Service::use_vulkan_api() {
    LOG_ERROR("Vulkan is not implemented yet");
    T5_GraphicsContextVulkan graphics_context;
    // graphics_context.instance = 
    // graphics_context.physicalDevice =
    // graphics_context.device =
    // graphics_context.queue =
    // graphics_context.queueFamilyIndex =
    set_graphics_context(graphics_context);
}

Also going to need to get vkImage handles for left and right views. I'm guessing that Godot 4 uses whatever is the GL texture array equivalent for Vulkan. So either there is a way to create these vkImage from layers or we need to communicate the VkImage format to T5 to see if they will support it.

The Tilt Five system can use different types of game boards like LE/XE-flat/XE-raised. There are NDK functions for getting the current board type as well as dimensions that should be exposed to Godot.

Thank you for creating the GDExtension for the Tilt Five glasses. It is working great!

How do you export the project with the extension enabled? The extension works fine in development mode, but when I export the project and run the binary file, the rendering does not happen with the glasses. The same happens with and without the "debug" export option. Thanks.

These errors are listed when the app using the extension closes. The renderer marks the created textures as render targets. Because of this they can't be deallocated as normal so they leak when the app exits.

ERROR: Condition "t->is_render_target" is true.
   at: GLES3::TextureStorage::texture_free (drivers\gles3\storage\texture_storage.cpp:706)
ERROR: 1 RID allocations of type 'struct GLES3::Texture' were leaked at exit.
ERROR: Texture Layered: 82: leaked 9961416 bytes.
   at: GLES3::Utilities::~Utilities (drivers\gles3\storage\utilities.cpp:75)

Audio for muli-glasses play in the same application is still a subject that is not really handled yet. Tilt Five doesn't even handle it yet on other officially supported engines. Each pair for glasses is exposed to the OS as an audio device. However, how to correlate these devices with the glasses in game is still an open question.

Can the Godot engine support different simultaneous audio listeners per scene?
Can the output from one listener be multiplexed to different audio outputs?

Need usage documentation and tutorials