Text of developer.apple.com › forums › thread › 833337 as Styx crawled it on Nov 19, 2026. The page may have changed since. Only its words are kept, without pictures or layout.

Open the live page

Foveated Streaming with CloudXR us… | Apple Developer Forums

Foveated Streaming with CloudXR using KRVR's native PCVR streaming app Spatial Computing Foveated Streaming & Spatial Tracking Game Controller You’re now watching this thread. If you’ve opted in to email or web notifications, you’ll be notified when there’s activity. Click again to stop watching or visit your profile to manage watched threads and notifications. You’ve stopped watching this thread and will no longer receive emails or web notifications when there’s activity.

Click again to start watching. Kross09 OP Created Jun ’26 Replies 1 Boosts 0 Views 623 Participants 2 This post is from the Activity Foveated Streaming & Spatial Tracking Q&A . I'm shipping a PCVR streaming app using PSVR2 Sense controllers on the Vision Pro. Four pain points I'd love clarity on: PSVR2 L3/R3 thumbstick click at rest — Apple's GameController framework doesn't fire a stick-click event unless there's axis deflection.

Users physically clicking the stick straight in get nothing. Is this an intentional filter, and is a change planned? Haptic forwarding PC → AVP. On CloudXRKit there's NSKStreamClientHapticDelegate for dispatching server-generated haptic events into CHHapticEngine. Does FoveatedStreaming have an equivalent delegate, or are streamer apps expected to bring their own side-channel for haptics? Controller pose — we're currently going through ARKit's AccessoryTrackingProvider and combining with GCController input.

Is that the recommended supported path for FoveatedStreaming apps, or is there something more native to the framework planned? Gaze clarity :- The framework already uses gaze internally to drive foveation tile selection on the server. Is any per-frame metadata returned to the client that tells us — even loosely — where the user is looking? A gaze quadrant, a foveal-tile coordinate, distance-from-fovea-centre. Anything we can feed back to the PC to tune its own pipeline.

Privacy-preserving aggregates — would Apple consider exposing fixation/saccade ratios, gaze stability over the last N frames, dwell-on-region signals? Lets apps push higher detail without ever learning raw direction. 3. What's the actual barrier — strictly privacy, or also performance? If it's privacy, would an on-device processing surface (App Intent style, Background Asset extension, etc.) be a viable path to let apps act on gaze without raw access?

4. are there public APIs in this area today that I've missed? I've been working from the swiftinterface + WWDC sessions and don't want to be reinventing something that already exists in a quieter corner of the SDK. Answered by Vision Pro Engineer in 892793022 Hello! PSVR2 L3/R3 thumbstick click at rest — Apple's GameController framework doesn't fire a stick-click event unless there's axis deflection. Users physically clicking the stick straight in get nothing.

Is this an intentional filter, and is a change planned? This sounds like a bug. Please file a feedback item via feedback assistant . I'm curious if you are seeing this specifically with the FoveatedStreaming framework, or also when using GameController framework more generally? Haptic forwarding PC → AVP. On CloudXRKit there's NSKStreamClientHapticDelegate for dispatching server-generated haptic events into CHHapticEngine.

Does FoveatedStreaming have an equivalent delegate, or are streamer apps expected to bring their own side-channel for haptics? This is a known issue with the framework on 26.4 and 27.0 beta 1, and something the team is looking into. Controller pose — we're currently going through ARKit's AccessoryTrackingProvider and combining with GCController input. Is that the recommended supported path for FoveatedStreaming apps, or is there something more native to the framework planned?

This is the correct path if you are rolling your own pose + controller input. The framework already uses gaze internally to drive foveation tile selection on the server. Is any per-frame metadata returned to the client that tells us — even loosely — where the user is looking? A gaze quadrant, a foveal-tile coordinate, distance-from-fovea-centre. Anything we can feed back to the PC to tune its own pipeline. Privacy-preserving aggregates — would Apple consider exposing fixation/saccade ratios, gaze stability over the last N frames, dwell-on-region signals?

Lets apps push higher detail Eye input data is restricted to the foveated streaming protocol. This is intentional to help protect user privacy. are there public APIs in this area today that I've missed? I've been working from the swiftinterface + WWDC sessions and don't want to be reinventing something that already exists in a quieter corner of the SDK. Yes - good news! We have shipped a new API this year, which allows streaming providers (separate from NVIDIA CloudXR) to extend visionOS with their own Foveated Streaming Protocol.

These streaming providers are granted access to approximate eye input data, in a separate extension process, to facilitate the implementation of foveated streaming. Please see my post here for more information. Replies 1 Boosts 0 Views 623 Participants 2 Vision Pro Engineer OP Apple Jun ’26 Accepted Answer Recommended Hello! PSVR2 L3/R3 thumbstick click at rest — Apple's GameController framework doesn't fire a stick-click event unless there's axis deflection.

Users physically clicking the stick straight in get nothing. Is this an intentional filter, and is a change planned? This sounds like a bug. Please file a feedback item via feedback assistant . I'm curious if you are seeing this specifically with the FoveatedStreaming framework, or also when using GameController framework more generally? Haptic forwarding PC → AVP. On CloudXRKit there's NSKStreamClientHapticDelegate for dispatching server-generated haptic events into CHHapticEngine.

Does FoveatedStreaming have an equivalent delegate, or are streamer apps expected to bring their own side-channel for haptics? This is a known issue with the framework on 26.4 and 27.0 beta 1, and something the team is looking into. Controller pose — we're currently going through ARKit's AccessoryTrackingProvider and combining with GCController input. Is that the recommended supported path for FoveatedStreaming apps, or is there something more native to the framework planned?

This is the correct path if you are rolling your own pose + controller input. The framework already uses gaze internally to drive foveation tile selection on the server. Is any per-frame metadata returned to the client that tells us — even loosely — where the user is looking? A gaze quadrant, a foveal-tile coordinate, distance-from-fovea-centre. Anything we can feed back to the PC to tune its own pipeline. Privacy-preserving aggregates — would Apple consider exposing fixation/saccade ratios, gaze stability over the last N frames, dwell-on-region signals?

Lets apps push higher detail Eye input data is restricted to the foveated streaming protocol. This is intentional to help protect user privacy. are there public APIs in this area today that I've missed? I've been working from the swiftinterface + WWDC sessions and don't want to be reinventing something that already exists in a quieter corner of the SDK. Yes - good news! We have shipped a new API this year, which allows streaming providers (separate from NVIDIA CloudXR) to extend visionOS with their own Foveated Streaming Protocol.

These streaming providers are granted access to approximate eye input data, in a separate extension process, to facilitate the implementation of foveated streaming. Please see my post here for more information. Add comment