wheaney / xrlinuxdriver Goto Github PK

System: GPD Win Max 2 2022 edition (AMD 6800U)
OS: Manjaro Linux
Xreal Air glasses

Xreal_Driver installs and loads without issue and is enabled

Plugged in glasses and no response from mouse cursor on head movement

.xreal_driver_log:
Waiting for glasses
Waiting for glasses
Waiting for glasses

System log just shows:
Started XREAL Air user-space driver.

Right now I've had to use the "dead zone" setting on my virtual joystick controller to filter out the noise that comes from the XREAL glasses, without this "dead zone" applied -- and even still with it applied sometimes -- the driver will drift in one direction. This often seems to follow a movement of some kind (move right and then it drifts right, etc...).

Someone suggested in discord that I look into applying something like a Kalman filter to reduce drift. Ideally, with the glasses' sensors as precise as we know they are (e.g. people don't seem to complain about drift with the official Nebula software), we should be able to have a virtual joystick that requires no dead zone setting, so that head movements are tracked exactly 1:1 by the joystick. That should be the goal of this task.

hi.
i was wonder if it's possible to transfer and display frames or images via usb-c to usb-c from pc to glasses like android phones.
this driver recognizes glasses and have some output acts, but how can i send frames into glass and display them ?

A couple issues have come up that prevent the setup script from working perfectly on Steam Deck:

1. wget doesn't come preinstalled on SteamOS
2. The read-only file system can prevent the copying of udev rules

Without the mirroring functionality, it's fine to rely on Steam's own native sensitivity sliders in the controller settings. But once we have the mirrored device functionality working (see issue #4) we'll want to allow users to control the joystick input sensitivity from the glasses separately from the device they're mirroring. Setting configs isn't super convenient with the current script, but some type of UX (e.g. Decky plugin for Steam Deck) could allow the driver to have its own slider in the future.

when running bin/package I get the error that the UA_API_SECRET environment variable is missing. This seems to be related to google analytics.

Can I build without using Google Analytics?

It appears my XREAL glasses are very sensitive and don't work well with the tapping method for re-centering/re-calibrating. It would be great if an additional method of a button combo could trigger this functionality - something like both triggers and both bumpers at the same time. Regardless - thanks for this awesome application!

Hey there! It's probably not the best way to reach out, but we have a discord with all our findings and projects we've been working on. The one thing we never got to solve 100% is yaw drift. I'm curious if you made any progress on this.

Xreal Air glasses connect to a device in 60hz mode by default. If user holds the Volume Up rocker until two beeps are heard, this puts the glasses in 120hz mode. When doing this while connected to the Steam Deck, the plug-in no longer recognises that the Xreal Air glasses are plugged in, requiring user to unplug the glasses and plug them back in. Doing so resets the power to the glasses and puts it back in 60hz mode, thus making 120hz mode while using the plug-in currently impossible. It's requested that the plug-in recognizes the switch to 120hz mode in the Xreal Air glasses.

This looked like an interesting project so I had a go at building it on NixOS. Seeing as SteamOS added a /nix directory in 3.5.5 NixPkg might be an interesting way of deploying this in the future! Because of how Nix works, your existing install and packaging scripts wouldn't work.

For this test, I was trying to get XRLinuxDriver running on my laptop using sway 1.8.1, Linux 6.1.69 and with xreal Air 1.

The Nix derivation (build script) is basically just checking out the repo & dependencies and running cmake - it looks like:

{ lib
  , stdenv
  , fetchgit
  , cmake
  , pkg-config

  , json_c
  , libevdev
  , libusb
}:

stdenv.mkDerivation rec {
  pname = "xr-linux-driver";
  version = "0.6.1-beta";

  src = (fetchgit {
    url = "https://github.com/wheaney/XRLinuxDriver";
    rev = "0f74025ccee6d9017f07e39db9bcdacfaac7e134";
    sha256 = "sha256-fpIxNiMW8IelrNMuHtOPePIxeI1sxUNigZq5f/CqjyA=";
    fetchSubmodules = true;
  }).overrideAttrs (_: {
    GIT_CONFIG_COUNT = 1;
    GIT_CONFIG_KEY_0 = "url.https://gitlab.com/.insteadOf";
    GIT_CONFIG_VALUE_0 = "[email protected]:";
  });

  buildInputs = [
    json_c
    libevdev
    libusb
  ];

  nativeBuildInputs = [
    pkg-config
    cmake
  ];

  outputs = [ "out" ];

  patches = [
    ./install.patch
  ];

  meta = with lib; {
    description = "Custom user-space Linux driver for XR devices";
    homepage = "https://github.com/wheaney/XRLinuxDriver";
    license = licenses.mit;
    platforms = platforms.linux;
    maintainers = [ "wheaney" ];
  };
}

The patch applied is to CMakeLists.txt to add install targets to the output:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 9d55cec..073303b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -55,4 +55,8 @@ target_include_directories(xrealAirLinuxDriver
 
 target_link_libraries(xrealAirLinuxDriver
 		PRIVATE ${LIBEVDEV_LIBRARIES} nrealAirLibrary Threads::Threads m ${VITURE_LIB_NAME} ${LIBUSB_LIBRARY}
-)
\ No newline at end of file
+)
+
+install(TARGETS xrealAirLinuxDriver DESTINATION bin)
+
+install(TARGETS hidapi_hidraw DESTINATION lib)

This runs fine as far as I can tell, the compiled binary executes and creates .xreal_driver_{log,config,lock}, although it only ever logs:

Starting up XR driver
Waiting for glasses
Starting up XR driver
Waiting for glasses
Starting up XR driver
Waiting for glasses
...

modprobe was run elsewhere:

$ lsmod | grep uinput          
uinput                 20480  0

This is the slightly modified service to work around the hardcoded ~ and not actually having a writable home:

[Unit]
After=network.target
Description=xr-linux-driver
Type=simple

[Service]
Environment=HOME=/run/user/1000/xr-linux-driver
ExecStart=/nix/store/0kngb662m75hrxw88jsqb1mvdkxv0ixb-xr-linux-driver-0.6.1-beta/bin/xrealAirLinuxDriver
PrivateNetwork=true
Restart=on-failure

I can see that it's managing to create a device:

$ journalctl
Jan 02 19:18:34 hostname kernel: usb 1-1: new full-speed USB device number 5 using xhci_hcd
Jan 02 19:18:34 hostname kernel: usb 1-1: New USB device found, idVendor=3318, idProduct=0424, bcdDevice= 0.01
Jan 02 19:18:34 hostname kernel: usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Jan 02 19:18:34 hostname kernel: usb 1-1: Product: Air
Jan 02 19:18:34 hostname kernel: usb 1-1: Manufacturer: Vendor
Jan 02 19:18:34 hostname kernel: usb 1-1: SerialNumber: A00020:14:32
Jan 02 19:18:34 hostname kernel: hid-generic 0003:3318:0424.002F: hiddev98,hidraw4: USB HID v1.10 Device [Vendor Air] on usb-0000:00:14.0-1/input3
Jan 02 19:18:34 hostname kernel: hid-generic 0003:3318:0424.0030: hiddev99,hidraw5: USB HID v1.10 Device [Vendor Air] on usb-0000:00:14.0-1/input4
Jan 02 19:18:34 hostname kernel: hid-generic 0003:3318:0424.0031: hiddev100,hidraw6: USB HID v1.10 Device [Vendor Air] on usb-0000:00:14.0-1/input5
Jan 02 19:18:34 hostname kernel: input: Vendor Air as /devices/pci0000:00/0000:00:14.0/usb1/1-1/1-1:1.6/0003:3318:0424.0032/input/input45
Jan 02 19:18:34 hostname kernel: hid-generic 0003:3318:0424.0032: input,hidraw7: USB HID v1.10 Device [Vendor Air] on usb-0000:00:14.0-1/input6

$ libinput list-devices
...
Device:           Vendor Air
Kernel:           /dev/input/event20
Group:            6
Seat:             seat0, default
Capabilities:     keyboard 
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      n/a
Nat.scrolling:    n/a
Middle emulation: n/a
Calibration:      n/a
Scroll methods:   none
Click methods:    none
Disable-w-typing: n/a
Disable-w-trackpointing: n/a
Accel profiles:   n/a
Rotation:         0.0

$ swaymsg -t get_inputs
{
  {
    "identifier": "13080:1060:Vendor_Air",
    "name": "Vendor Air",
    "vendor": 13080,
    "product": 1060,
    "type": "keyboard",
    "repeat_delay": 600,
    "repeat_rate": 25,
    "xkb_layout_names": [
      "English (UK)"
    ],
    "xkb_active_layout_index": 0,
    "xkb_active_layout_name": "English (UK)",
    "libinput": {
      "send_events": "enabled"
    }
  },
  ...
}

But it identifies as a keyboard. I've done before and after lists of libinput and it can only see one new device being created.

I've tried changing the output_mode=mouse to output_mode=joystick and the driver can see the change, but that doesn't change the detected device.

$ journalctl
...
Jan 02 18:48:49 hostname systemd[1975]: Stopping xr-linux-driver...
Jan 02 18:48:49 hostname systemd[1975]: Stopped xr-linux-driver.
Jan 02 18:48:49 hostname systemd[1975]: Started xr-linux-driver.
Jan 02 18:48:49 hostname xrealAirLinuxDriver[120743]: Output mode has been changed to 'joystick'

Any ideas?

EDIT: I've just read this back and it's not super clear what I'm asking. The Vendor Air does appear to be from the glasses itself, rather than the user-mode driver. So I guess this is more of a question around the behaviour of XRLinuxDriver, and what the correct behaviour is supposed to be at this point. Presumably they should be called XR virtual {joystick,mouse} based on: outputs.c

As of now, after setup, the virtual controller will be created every time you plug in your glasses. Since this is just a second controller, it could technically just be ignored most of the time if someone doesn't want their game to use it, but it is spawning a background process that's pretty lightweight but still not insignificant. It would be nice to have the driver support user-settings; these could be expanded in the future but at least initially a "disabled" setting would allow users to control whether or not the driver runs when the device is plugged in.

Looking forward to it thank you

One-time installation script that checks for dependencies (uinput, libevdev, json-c, etc...) and installs or reports anything missing, checks for and fixes udev rule if missing

This driver could theoretically use the gyro/accel sensors from another device (Steam Deck, in this case) to cancel out movements from car/bus/airplane. As it is right now, if you're on a vehicle that's accelerating or turning, your glasses will turn your frame of reference in the game.

Without a second device for reference, it could still try to detect a constant velocity and remove that from its readings, but this wouldn't work quite as well (e.g. how do you detect a bump in the road vs a small head movement? you probably don't).

Find/replace nReal -> xReal in package, test everything, update README, upload latest release

Some situations arise where the roll movement is inverted. Tilting your head left makes the screen tilt right and visa versa. Unfortunately there aren't any settings within game/sim environments that invert this function in a similar manner to y-axis/x-axis inversion. Toggles for axis inversion within the plugin might(?) work

I imagine this might take a lot of work, so I understand if this ends up as a wontfix. I have a pair I can test with.

More like a summary issue resulting from the conversion in the MR

I'd like to collect some information about the usb devices to build the udev rules.

# usb-devices

T:  Bus=03 Lev=01 Prnt=07 Port=04 Cnt=01 Dev#= 12 Spd=12   MxCh= 0
D:  Ver= 2.01 Cls=ef(misc ) Sub=02 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=35ca ProdID=1011 Rev=02.00
S:  Manufacturer=VITURE One
S:  Product=XR Glasses
S:  SerialNumber=2074306C4156
C:  #Ifs= 4 Cfg#= 1 Atr=a0 MxPwr=300mA
I:  If#= 0 Alt= 0 #EPs= 2 Cls=03(HID  ) Sub=00 Prot=00 Driver=usbfs
E:  Ad=01(O) Atr=03(Int.) MxPS=  64 Ivl=1ms
E:  Ad=81(I) Atr=03(Int.) MxPS=  64 Ivl=1ms
I:  If#= 1 Alt= 0 #EPs= 2 Cls=03(HID  ) Sub=00 Prot=00 Driver=usbfs
E:  Ad=02(O) Atr=03(Int.) MxPS=  64 Ivl=1ms
E:  Ad=82(I) Atr=03(Int.) MxPS=  64 Ivl=1ms
I:  If#= 2 Alt= 0 #EPs= 1 Cls=02(commc) Sub=02 Prot=01 Driver=cdc_acm
E:  Ad=86(I) Atr=03(Int.) MxPS=   8 Ivl=16ms
I:  If#= 3 Alt= 0 #EPs= 2 Cls=0a(data ) Sub=02 Prot=00 Driver=cdc_acm
E:  Ad=03(O) Atr=02(Bulk) MxPS=  64 Ivl=0ms
E:  Ad=85(I) Atr=02(Bulk) MxPS=  64 Ivl=0ms

$ ls -la /dev/serial/by-id

lrwxrwxrwx 1 root root 13  7. Jun 12:53 usb-VITURE_One_XR_Glasses_2074306C4156-if02 -> ../../ttyACM0

I'm still figuring out the rules

$ udevadm info -a -n /dev/serial/by-id/usb-VITURE_One_XR_Glasses_2074306C4156-if02

  looking at device '/devices/pci0000:00/0000:00:14.0/usb3/3-5/3-5:1.2/tty/ttyACM0':
    KERNEL=="ttyACM0"
    SUBSYSTEM=="tty"
    DRIVER==""
    ATTR{power/control}=="auto"
    ATTR{power/runtime_active_time}=="0"
    ATTR{power/runtime_status}=="unsupported"
    ATTR{power/runtime_suspended_time}=="0"

Right now the driver won't start up if the glasses aren't already plugged in and ready for use. Users need to plug in the device, wait a few seconds, then start up the driver and leave it running in the background. Ideally, once "installed," the driver would be able to start up automatically when the device is plugged in.

The standalone driver (virtual joystick) code I've built out applies filtering of invalid movements so that the sudden shifting of gyro measurements that comes from the glasses doesn't affect the virtual joystick. This issue made me realize I should be building this kind of filtering into the library so anyone using it will get the benefits of it (which would also apply to my OpenVR implementation that consumes this driver).

For anyone wishing to completely remove the driver and udev rules.

Right now I'm not tying the setup script to a particular version of the driver, it just downloads the latest gzip file from the Releases in Github. The problem with that is that the setup script has install logic that IS version-sensitive. For example, the change to move to systemd has changes in both the setup script and the resulting gzip, so if someone tries to reinstall from an old setup script right now it just won't work.

The solution I've come up with is to either:

1. Have the setup script check a release version against its own, and redownload itself if the versions don't match. The problem with this is that I have to maintain versions in the code and remember to update them and keep them in sync with the release tags. I don't like this solution.
2. Move all setup logic into another setup script that the user doesn't execute directly. The setup script that the user executes would do nothing setup-specific, instead it would just download the latest version of this secondary setup script each time and run that.

Hi,

it would be nice to an alternative joy stick mode. The current one tracks the velocity and converts it to the right joy stick. My use case is Automobilista 2 where the right joy sticks absolute position is the head rotation inside the car. It would be awesome to look around inside the car, like a light version of VR in this game. Maybe (and hopefully) it is not the only use case for this feature.

I am unable to update submodules because I don't have access to GitLab so I get permission denied.
How to update submodules without having to create a GitLab account?

When using VR Lite with >60 sensitivity, camera shake starts to hinder user experience.

Deadzone setting should allow the retention of gyroscope functionality while keeping the camera/mouse stationary when the head is stationary.

One of the first things I tried when developing the current driver was setting the INPUT_PROP_ACCELEROMETER flag so that I could report the euler values directly in my libevdev events rather than converting to a joystick. This didn't work with some tinkering so I fell back to the joystick. The joystick approach has flaws, such as needing to set an arbitrary maximum movement degree value that limits how quickly someone could move their head before maxing out the joystick (and thus losing some of their movement).

I was looking at how the playstation controller drivers report their gyro movements, using the same prop value above, and we might be able to get that working if we mimic this kind of driver more closely. It might also be easier to get this working in conjunction with the work to combine our events with a gamepad as that will produce a full gamepad that would mimic ps/xbox controllers even more closely.

Hi

I am testing the xreal air glass.
I am observing a weird phenomenon, that when i play with just yaw and keep it still, suddenly it shift to random value. This is happening every-time. I attach a video also here. This is happening at every angle, not only at zero yaw.

I don't know if I have to write this issue to Fusion repository.

Do you observe the same?

• Jai

Hey, I've got an Air 2 Pro and it's not being detected.
I've attached the lsusb output, please say if you need more info.

xreal_air2_pro.txt

The "disabled" state used to completely turn off the driver. But after I added the "connected" status, the driver disabled state will still detect connected supported devices by capturing them and preventing other processes from using them. Let's bring back a proper "stopped" state that doesn't look for devices until started again.

Didn't see a discussion tab (or a mailing list), so sorry for creating a new issue.

One of the things I want to implement is a larger virtual desktop that follows head tracking. E.g. a 4k virtual desktop that you can pan while moving your head.

Has anyone thought about how one might do this? I can't even figure out how to make a virtual desktop in wayland with edge panning (or is this specific to a compositor like Kwin?)

Using udev to trigger the driver as a long-running program has been problematic as it often tries to launch the driver dozens of times, and also occasionally kills the driver for no reason. I'd like to move to a better solution.

Possible options:

1. /etc/init.d/ script
2. systemd service

/etc/init.d/ seems to lack a standard, so its support and setup varies across Linux distributions. systemd seems more universal

Linked from breezy-desktop issue as implementation was in the XRLinuxDriver:

wheaney/breezy-desktop#11

I've worked on updating driver.c multitap.c and output.c programs to make 1,2,3 and 4 taps work pretty well.

The constants have been updated in my code, and calibrate and center have been pushed down in priority, making room for single and double tap.

I implemented the single left click, and double left click using using the appropriate mouse driver code.

Will submit a pr eventually.

At the moment, the driver creates a gamepad that only produces joystick output, users still need a second gamepad or keyboard/mouse to control their games. Steam will support having two controllers, but from my testing games won't support two joystick controllers working at the same time, so the xReal controller settings have to be updated to produce "joystick as mouse" output. For any games that allow mouse-look while also using a gamepad, this effectively works around not being able to use two controllers, but this means that compatibility is limited.

There are libraries like uinput-mapper that allow for mapping two devices to one virtual-device; something like this would help solve this problem and would already work with this driver in the current state. But this means this driver just isn't completely viable right out of the box. It would be nice if the driver could detect other gamepads and combine with them into a single virtual device. If there are multiple other devices, then users would need some way to specify which to use, which becomes a little more complicated without a UI or technical users, but if one device is detected, this becomes pretty simple.