My condolences 🫂

When I was trying to get my layer loaded and print a hello, I encountered a bug in the OpenXR Loader. Of course it was open-source, and I managed to build a patched version and later get my bugfix merged, which is now also included in Steam Link.

I'm very lucky Valve didn't put any fences into Steam Link and the app uses any extension that's advertised as supported, no matter the headset model!

Also, Rust is an amazing tool for this project, and OpenXR is such a great open-source & extensible API allowing me to do this.

- Draws IMGUI stuff onto a texture and then injects it as a quad layer during the `xrEndFrame` call, which is the debug UI.

Next on the list is support for UVC (GC0308!), and compilation as a standalone Android app (for HTTP streaming or standalone OSC maybe, why not haha?)

- Implements the `XR_EXT_eye_gaze_interaction`, `XR_FB_eye_tracking_social`, and `XR_FB_face_tracking2` extensions (same as QPro) to allow Steam Link request ET/FT data.
- Replies to the ET/FT requests with the data from the model, effectively providing foveated encoding as well.

- Using the FD on native side, starts serial communication with the camera.
- Runs ONNX runtime with a bundled model for eye tracking and expressions using my crude model haha.
- Also runs a camera streaming HTTP server so you can use your favorite eye tracking software from PC.

The binary itself does the following:
- Extracts OpenGL, Android, and JVM context during the initialization of the host app.
- Thanks to an existing library, using JNI starts watching for USB connections, requests for permission, and extracts the USB file descriptor.

This is an OpenXR API Layer that's compiled into a shared library (.so) and some JSON metadata, inserted into Steam Link APK file as per OpenXR docs to be loaded as a layer. A layer is pretty much a middleware and as a layer dev you can intercept and modify any calls!

Performance wise, I can confirm that they actually can run at 120fps! In fact, it's more like 124-125fps when measuring. That's only at 320x240 using MJPEG, but it also advertises NV12. 160x120 and 640x480 are advertised at 30fps or lower.

The power consumption is 641mW when running.

The module also has two IR LEDs, which are nice for face tracking, effectively making it a perfect out-of-the-box FT module. Someone has already made a great looking case for it: fxtwitter.com/ugokutennp/s...

But for eye tracking, the LEDs and soldered camera need modifications.

I also got some GC0308 camera boards to try them for face and, more importantly, eye tracking. The seller advertises 120 fps at 320x240 resolution, which is perfect for eye tracking, and the module itself is exposed as a standard UVC device (webcam). Which means no more serial stuff like with ESPs.

Oh it's iOS, nevermind me xD

Have you looked at ADB logs, see if it's some process maybe that's spamming errors or something?