Headtracking Overprediction on Vision Pro given jittery/shaky view when zoomed in

[ZigZag2010]

Description

Using ALVR v20.11.1 on Apple Vision Pro I get a lot of jitter when zooming in in flight sims. The issue is not that noticeable when zoomed "normally" but when zoomed in it results in the viewpoint shaking quite a bit and makes it very hard to ID contacts and results in nausea.

General Troubleshooting

-I carefully followed the instructions in the README and successfully completed the setup wizard
-I read the ALVR GitHub Wiki

I believe this issue is due to overprediction. On Quest3 using Oculus Link I am able to successfully eliminate the jitter 100% using the "Overprediction reduction" feature of the OpenXR toolkit. Basically it seems to filter out the high frequency low amplitude jitter. Unfortunately this setting in OXR toolkit is not usable with SteamVR OpenXR runtime.

I believe what's needed to solve it is an overprediction reduction feature like in OXR toolkit, but specifically for ALVR. Or some other way to reduce this head tracking overprediction jitter.

Here's a video, notice how stable my finger is compared to the jittery aircraft. This jitter does not happen in the VisionOS native apps so I think it's related to ALVR.

https://www.youtube.com/watch?v=LlzttdWHkfo

I suspect this is related to the predict_motion function within lib.rs

pub fn send_tracking(
    &self,
    poll_timestamp: Duration,
    mut device_motions: Vec<(u64, DeviceMotion)>,
    hand_skeletons: [Option<[Pose; 26]>; 2],
    face_data: FaceData,
) {
    dbg_client_core!("send_tracking");

    let target_timestamp =
        if let Some(stats) = &*self.connection_context.statistics_manager.lock() {
            poll_timestamp + stats.average_total_pipeline_latency()
        } else {
            poll_timestamp
        };

    for (id, motion) in &mut device_motions {
        if *id == *HEAD_ID {
            *motion = predict_motion(target_timestamp, poll_timestamp, *motion);

            let mut head_pose_queue = self.connection_context.head_pose_queue.write();

            head_pose_queue.push_back((target_timestamp, motion.pose));

            while head_pose_queue.len() > 1024 {
                head_pose_queue.pop_front();
            }

            // This is done for backward compatibiity for the v20 protocol. Will be removed with the
            // tracking rewrite protocol extension.
            motion.linear_velocity = Vec3::ZERO;
            motion.angular_velocity = Vec3::ZERO;
        } else if let Some(stats) = &*self.connection_context.statistics_manager.lock() {
            let tracker_timestamp = poll_timestamp + stats.tracker_prediction_offset();

            *motion = predict_motion(tracker_timestamp, poll_timestamp, *motion);
        }
    }

Here's an excerpt for the OXR toolkit overprediction reduction that I used to solve a similar jitter problem previously (but wont work with steamvr/alvr):

https://github.com/mbucchia/OpenXR-Toolkit

Shaking reduction, formerly Prediction dampening (only when supported by the system): The prediction override, which can be use to dampen the prediction for head, controllers, and hand movements.

            // Apply prediction dampening if possible and if needed.
            if (m_hasPerformanceCounterKHR) {
                const int predictionDampen = m_configManager->getValue(config::SettingPredictionDampen);
                if (predictionDampen != 100) {
                    // Find the current time.
                    LARGE_INTEGER qpcTimeNow;
                    QueryPerformanceCounter(&qpcTimeNow);

                    XrTime xrTimeNow;
                    CHECK_XRCMD(
                        xrConvertWin32PerformanceCounterToTimeKHR(GetXrInstance(), &qpcTimeNow, &xrTimeNow));

                    XrTime predictionAmount = frameState->predictedDisplayTime - xrTimeNow;
                    if (predictionAmount > 0) {
                        frameState->predictedDisplayTime = xrTimeNow + (predictionDampen * predictionAmount) / 100;
                    }

                    m_stats.predictionTimeUs += predictionAmount;
                }

Environment

Hardware

CPU: 7800X3D

GPU: 4090

GPU Driver Version: 565.90

Audio:

Installation

ALVR Version:

ALVR Settings File:

SteamVR Version:

Install Type:

• Packaged (exe, deb, rpm, etc)
• Portable (zip)
• Source

OS Name and Version (winver on Windows or grep PRETTY_NAME /etc/os-release on most Linux distributions):

[ZigZag2010]

I think basically what we want is to be able to change this code so that it doesn't extrapolate the head tracking so much

pub fn send_tracking(
    &self,
    poll_timestamp: Duration,
    mut device_motions: Vec<(u64, DeviceMotion)>,
    hand_skeletons: [Option<[Pose; 26]>; 2],
    face_data: FaceData,
) {
    dbg_client_core!("send_tracking");

    let target_timestamp =
        if let Some(stats) = &*self.connection_context.statistics_manager.lock() {
            poll_timestamp + stats.average_total_pipeline_latency()
        } else {
            poll_timestamp
        };