From 56005d8c3d22474720a8c68c5ff33daba4c901b7 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Fri, 28 Mar 2025 14:09:55 -0700 Subject: [PATCH 01/20] Explainer first draft --- GamepadButtonAndAxisEvents/explainer.md | 291 ++++++++++++++++++++++++ 1 file changed, 291 insertions(+) create mode 100644 GamepadButtonAndAxisEvents/explainer.md diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md new file mode 100644 index 00000000..6593bebd --- /dev/null +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -0,0 +1,291 @@ +# Gamepad Button and Axis Events + +## Authors: + +- Sneha Agarwal (https://github.com/snehagarwal_microsoft) +- Steve Becker (https://github.com/SteveBeckerMSFT) + +## Participate +- [Issue tracker] +- [Discussion forum] + +## Status of this Document + +This document is a starting point for engaging the community and standards bodies in developing collaborative solutions fit for standardization. As the solutions to the problems described in this document progress along the standards-track, we will retain this document as an archive and use this section to keep the community up-to-date with the most current standards venue and content location of future work and discussions. + +- This document status: **Active** +- Expected venue: +- Current version: **This document** + +## Introduction + +The current Gamepad API relies on constant polling to detect input changes, which can introduce input latency and increase CPU usage. This proposal adopts a hybrid approach that combines event-driven input handling with frame-based state consolidation, reducing unnecessary polling while maintaining responsiveness. + +- Event-Driven Input Handling: Immediate events (buttondown, buttonup) fire when discrete inputs like button presses or releases occur. + +- Frame-Based Consolidation: A gamepadchange event aggregates all input changes per frame, preventing event spam and improving efficiency. + +- Threshold-Based Polling for Analog Inputs: Analog inputs, such as joystick movements, are polled at a reduced frequency and only trigger updates when they exceed a defined threshold, ensuring meaningful changes are captured. + +By integrating these techniques, this approach minimizes CPU overhead while ensuring low-latency input processing, providing developers with a more efficient and flexible way to handle gamepad interactions. This proposal builds upon earlier work by Chromium developers while refining the hybrid model for improved performance. + +## User-Facing Problem + +The Gamepad API does not natively support event-driven input handling, and applications must rely on continuous polling for updates, which: + +- Introduces input latency, as polling intervals may not align with the precise timing of input changes. + +- Increases CPU usage, particularly when polling at high frequencies. + +- Lacks efficiency in handling gamepad input changes, as it does not account for meaningful input variations, leading to unnecessary events being triggered. + +### Goals + +1) **Enhance gamepad input handling** by transitioning from continuous polling to event-driven mechanisms, improving responsiveness. + +2) **Optimize performance** by reducing input latency and minimizing CPU overhead through a hybrid approach of event handling and threshold-based polling. + +3) **Provide flexibility for developers** by allowing efficient management of gamepad input events, enabling granular control over when and how input changes are processed. + +### Non-goals + +Deprecating the existing polling mechanism: The polling mechanism will not be fully deprecated, as it is still necessary for detecting small input changes that events alone may not address. + +## User research + +Research and experimental implementations in Firefox suggest that event-driven input handling can significantly improve responsiveness and efficiency. Based on this research, the following challenges with prior implementations have been identified and addressed: + +- Inadequate handling of analog button states: Previous implementations only triggered events when the button's pressed state changed, without accounting for changes in the button's numeric value or "touched" state. This hybrid approach ensures that both meaningful value changes and touch states are captured. + +- Potential loss of gamepad state before event listeners execute: In high-speed input scenarios, if a button is quickly pressed and released, the Gamepad’s button state may indicate that the button is already unpressed by the time the listener is invoked. By combining polling with event-driven updates, this issue is mitigated by ensuring the most recent state is captured and events are triggered only when there are significant changes. + +- Unintuitive event targets: Previous event systems fired on the global window object, which was counterintuitive. The windows object is not directly tied to a specific gamepad instance and since multiple gamepads can be connected, handleing events at the gamepad level will be more logical. Most input APIs like KeyboardEvent and MouseEvent dispatch events on the relevant input device not the global window. Gamepad events on window break this consistency. +With the new proposal, events will be triggered directly on the Gamepad object that generated the event, providing a clearer and more developer-friendly experience. + +- Event spam from per-frame axis updates: When multiple axes update in a single frame, firing an event for each change could lead to excessive event firing, creating unnecessary CPU load and potential latency. By introducing thresholds for axis and button changes, this approach reduces unnecessary events, ensuring that only significant input changes are processed, thus optimizing performance. + +## Proposed Approach +To address the challenges of handling gamepad input efficiently while ensuring real-time responsiveness, a hybrid approach is proposed to event-driven input handling along with frame-based state consolidation. This approach combines event-based triggers with thresholding and controlled polling for axes and analog inputs, offering a balance between responsiveness and minimizing CPU usage. + +**Key Considerations** +Event-Driven Mechanisms: Gamepad inputs are inherently event-based (e.g., button presses and axis movements). We'll leverage events to notify game developers only when significant changes occur, such as a button being pressed, released, or an axis moving beyond a threshold. This reduces unnecessary event firing and improves performance. + +**Polling with Thresholding:** While events are suitable for discrete actions like button presses and releases, analog inputs (such as joystick movements) require continuous monitoring. We can use polling for these inputs in combination with a threshold. This ensures that only meaningful changes trigger events, and small, irrelevant fluctuations are ignored. This thresholding prevents event spam while still allowing for real-time responsiveness. + +**Event Types and Behavior**: +buttondown / buttonup: These events fire only when a button is pressed or released. Events are triggered for discrete actions, reducing CPU usage and avoiding unnecessary checks. + +axischange: Fires when an axis value moves beyond a defined threshold (e.g., when a joystick moves a significant amount or the trigger is pressed beyond a certain value). This helps to minimize event spam by ensuring only notable changes trigger events, reducing overhead while maintaining responsiveness. + +**How This Works with Thresholding:** +Button Events: +- For buttondown and buttonup events, the browser will still trigger these events based on button state changes (e.g., pressed or released). +- These events can happen as soon as the state changes, without needing to wait for polling, because buttons are discrete (either pressed or not). + +Axis Changes: +- The axischange event will fire only when the axis value exceeds a predefined threshold (like 0.2 or -0.2, customizable), which helps to prevent unnecessary axis events if there’s just minor, non-meaningful input fluctuation. +- Polling occurs at a lower frequency to reduce CPU usage, but when an axis value changes meaningfully, it triggers the event. + +```js +// Example Event Code + +// When a button is pressed. +buttondown { + "type": "buttondown", + "gamepadIndex": 0, + "buttonIndex": 2, + "buttonSnapshot": { "pressed": true, "touched": true, "value": 1 }, + "gamepadTimestamp": 1234.567 +} + +// When a button is released. +buttonup { + "type": "buttonup", + "gamepadIndex": 0, + "buttonIndex": 2, + "buttonSnapshot": { "pressed": false, "touched": false, "value": 0 }, + "gamepadTimestamp": 1234.567 +} + +// When an axis changes and crosses a threshold. +axischange { + "type": "axischange", + "gamepadIndex": 0, + "axisIndex": 1, + "axisSnapshot": -0.65, + "gamepadTimestamp": 1234.567 +} + +``` + +### Proposed IDL for GamepadButtonEvent and GamepadAxisEvent + +``` +interface GamepadButtonEvent : Event { + readonly attribute long gamepadIndex; + readonly attribute long buttonIndex; + readonly attribute GamepadButton buttonSnapshot; + readonly attribute DOMHighResTimeStamp gamepadTimestamp; +}; + +interface GamepadAxisEvent : Event { + readonly attribute long gamepadIndex; + readonly attribute long axisIndex; + readonly attribute double axisSnapshot; + readonly attribute DOMHighResTimeStamp gamepadTimestamp; +}; +``` + +2) **Frame-Based Consolidated Event**: + +This approach consolidates all state changes that occur within a single frame into one event, reducing the number of events fired and preventing event spam. This is especially important for analog inputs and buttons that may change frequently within a single frame. + +- gamepadchange event: will fire once per frame, and will include all the changes (buttons and axes) that happened within that frame. This ensures that input is captured with minimal overhead while maintaining responsiveness. + +**Handling Analog Button States** + +To address issues such as inadequate handling of analog button states (e.g., button pressure or touch sensitivity), this proposal introduces improvements in the gamepadchange event, providing richer data and more precise control for developers. + +- Enhanced gamepadchange Event: The gamepadchange event will include not just basic button states (pressed/released), but also information about whether a button was touched and its analog value (pressure). This ensures that analog buttons are represented with more granular data, providing developers with full control over button behavior, including pressure sensitivity and touch detection. + +- Consolidate button state changes within a single event structure: Instead of firing a separate event for each button state change (pressed/released, touched, value), all these changes will be consolidated into one gamepadchange event per frame. This prevents event spam and improves the efficiency of event handling. + +- This will also reduce CPU overhead, as the browser will fire fewer events overall, and game developers will get all necessary button information in a single event. + +```js +// Example gamepadchange Event: +gamepadchange { + type: "gamepadchange", + gamepadIndex: 0, + // Timestamp for the event. + gamepadTimestamp: 1234.567, + changes: { + buttons: [ + { + // The index of the button that has changed. + buttonIndex: 2, + buttonSnapshot: { + // Whether the button is pressed. + pressed: true, + // Whether the button is touched. + touched: true, + // The analog value of the button (e.g., pressure). + value: 0.85 + } + } + ], + axes: [ + { + // Index of the axis that changed. + axisIndex: 1, + // The new value of the axis after change. + axisSnapshot: 0.5 + } + ] + } +} + +``` + +``` +// Web IDL for the gamepadchange event + + +in progress +``` + +**Event Definitions** +- **Fine-Grained Events** - These events only fire when a button is pressed, released, or an axis moves significantly. +- *Button up/down events* fires only once per actual press/release (not every frame). Avoids event spam by not sending redundant data. +- *Axis Movement Events* Threshold-based (fires only if the axis changes by more than (some %) from the last state). Prevents excessive event spam when a joystick is slightly jittering. + +- **Frame-based Consolidated Event** -Instead of firing individual events for every small change, a frame-based gamepadchange event collects multiple updates into one. +- Fires once per animation frame (e.g., 60Hz) and includes all changes. +- Optimized for efficiency (batch updates into one event). + +### Dependencies on non-stable features + +None identified at this stage. The proposal builds upon the existing Gamepad API. + +### Solving goal 1 with this approach + +Goal: Improve Gamepad Input Handling by Introducing Event-Driven Mechanisms. + +Event-based Mechanisms: For discrete inputs such as button presses, we will rely on event-driven updates (e.g., buttondown, buttonup). These events will be triggered based on changes to button states (pressed, released, etc.). + +Event Propagation: Ensure that each change (e.g., button press, axis movement) is dispatched with specific event details. For instance, an event might include which button was pressed, whether it was touched, or how far an axis moved. + +Threshold-based Polling for Analog Inputs: For continuous inputs such as joystick movements, we will use polling with thresholds. If the movement exceeds a certain threshold (e.g., 0.2), the event will fire to notify the application. This ensures that only meaningful changes in input are captured while reducing the need for continuous polling. + +Implementation Details +Polling at Intervals: Instead of polling the gamepad's state every frame, poll at a reduced frequency (e.g., every 50ms). This minimizes CPU usage while maintaining real-time responsiveness. + +Event Handling with Thresholding: +We will only fire events for significant changes in button or axis states. For analog inputs, this will involve checking if the change exceeds a threshold value before triggering an event. For buttons, this includes checking if the pressed or value exceeds a certain threshold. For axes, this means checking whether the movement exceeds a predefined value. + +```js +// Sample Code for Event Handling with Thresholding +``` + +### Solving goal 2 with this approach. + +Goal: Enhance Performance by Reducing Input Latency and Minimizing CPU Usage. +Thresholding for Analog Inputs: To optimize performance and minimize CPU usage, thresholding will be used to ensure that only significant input changes are processed. This helps avoid unnecessary polling of small, insignificant movements or fluctuations. + +Threshold Example: A threshold of 0.2 will be applied to both buttons and axes, which means that an event or update is only triggered if the value changes by more than 0.2. + +Polling Intervals: Polling will occur at controlled intervals (e.g., every 50ms) instead of continuously on every frame. This reduces CPU overhead by avoiding constant polling when there is no meaningful change. + +### Solving goal 3 with this approach. +Goal: Provide Flexibility for Developers to Efficiently Manage Gamepad Input Events. + +Customizable Event Handling: Developers can set their own threshold values for button or axis changes, enabling different types of input handling (e.g., more sensitive input for precise games like racing sims, or less sensitivity for more casual games). +Developers can decide how to handle different gamepad states (pressed, released, touched) through event detail properties such as buttonSnapshot or axisSnapshot. + +Developer-Defined Event Handling: By using events like gamepadchange, developers can register event listeners to define exactly how gamepad input should be processed, such as applying force feedback when a button is pressed or initiating an action when a specific button combination is detected. + +Access to Raw Data: The game developer can access both raw and threshold-processed data (e.g., axis values and button states) to design input controls that suit the needs of the game, such as adjusting for gamepad types, handling different input devices, or implementing specific interactions. + +## Alternatives considered + +### Alternative 1: Continue Using Polling + +Polling remains an option but does not address the latency and CPU issues. + +### Alternative 2: Per-Frame Gamepad Snapshot Events + +This approach by it self was rejected due to potential event spam and performance concerns. + +### Alternative 3: getIntermediateEvents +Captures fine-grained input changes (such as axis movements or button states) that don't fit into the normal event cycle. Rather than having the browser fire events for every intermediate state, the game developer manually checks for significant changes and fires events when needed. The threshold mechanism replaces the need for getIntermediateEvents in this proposal. + +## Accessibility, Privacy, and Security Considerations +To prevent abuse and fingerprinting: +User Gesture Required: Gamepad events won’t start firing until a user interacts with the gamepad (e.g., pressing a button). [Gamepad user gesture](https://www.w3.org/TR/gamepad/#dfn-gamepad-user-gesture) + +Limit Persistent Tracking (fingerprinting): gamepadchange event will not expose device-specific identifiers. By default, no gamepad state is exposed to the tab even when gamepads are connected. [Fingerprinting in Web](https://www.w3.org/TR/fingerprinting-guidance/) + +## Stakeholder Feedback / Opposition +Firefox: Implemented experimental support for gamepadbuttondown and gamepadbuttonup, but identified issues with event spam and lost state. + +Chromium: Evaluating different approaches to event-driven gamepad input handling.[Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0) + +Other Browsers: No official statements yet. + +## References & acknowledgements + +Thanks to the following contributors and prior work that influenced this proposal: + +Firefox’s experimental implementation. + +Chromium Prior discussions on improving gamepad input handling. + +Many thanks for valuable feedback and advice from: +- Gabriel Brito +- Matt Reynolds +- Steve Becker + +Thanks to the following proposals, projects, libraries, frameworks, and languages +for their work on similar problems that influenced this proposal. + +- [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0) From 1cf12c6a79bb1d74f4c1e52b39f6ff24cab61fe0 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Tue, 1 Apr 2025 15:28:53 -0700 Subject: [PATCH 02/20] Updated Explainer - removed threshold --- GamepadButtonAndAxisEvents/explainer.md | 227 ++++++++---------------- 1 file changed, 71 insertions(+), 156 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 6593bebd..ae2bd862 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -14,77 +14,57 @@ This document is a starting point for engaging the community and standards bodies in developing collaborative solutions fit for standardization. As the solutions to the problems described in this document progress along the standards-track, we will retain this document as an archive and use this section to keep the community up-to-date with the most current standards venue and content location of future work and discussions. - This document status: **Active** -- Expected venue: +- Expected venue:**[W3C Web Applications Working Group](https://www.w3.org/groups/wg/webapps/)** - Current version: **This document** +### Definitions +*Fine-Grained Events* - These events only fire when a button is pressed, released. +*Button up/down events* - fires only once per actual press/release (not every frame). +*Frame-based Consolidated Event* - Instead of firing individual events for every small change, a frame-based gamepadchange event collects multiple updates into one. +- Fires once per animation frame (e.g., 60Hz) and includes all changes. + ## Introduction -The current Gamepad API relies on constant polling to detect input changes, which can introduce input latency and increase CPU usage. This proposal adopts a hybrid approach that combines event-driven input handling with frame-based state consolidation, reducing unnecessary polling while maintaining responsiveness. +The current Gamepad API relies on continuous polling to detect input changes, which can lead to input latency and increased CPU usage. This proposal introduces a more efficient approach that combines event-driven input handling for button presses and releases with frame-based state consolidation, where a single event encapsulates all gamepad state changes that occur within an input frame. - Event-Driven Input Handling: Immediate events (buttondown, buttonup) fire when discrete inputs like button presses or releases occur. -- Frame-Based Consolidation: A gamepadchange event aggregates all input changes per frame, preventing event spam and improving efficiency. +- Frame-Based Consolidation: gamepadchange event encapsulates all the gamepad state changes that occurred within an input frame, preventing event spam. -- Threshold-Based Polling for Analog Inputs: Analog inputs, such as joystick movements, are polled at a reduced frequency and only trigger updates when they exceed a defined threshold, ensuring meaningful changes are captured. - -By integrating these techniques, this approach minimizes CPU overhead while ensuring low-latency input processing, providing developers with a more efficient and flexible way to handle gamepad interactions. This proposal builds upon earlier work by Chromium developers while refining the hybrid model for improved performance. +This proposal builds upon earlier work by Chromium developers while refining the hybrid model for improved performance. ## User-Facing Problem -The Gamepad API does not natively support event-driven input handling, and applications must rely on continuous polling for updates, which: - -- Introduces input latency, as polling intervals may not align with the precise timing of input changes. - -- Increases CPU usage, particularly when polling at high frequencies. - -- Lacks efficiency in handling gamepad input changes, as it does not account for meaningful input variations, leading to unnecessary events being triggered. +The Gamepad API does not natively support event-driven input handling, and applications must rely only on continuous polling for updates. This polling-based approach introduces input latency because scripts cannot sync their polling with the arrival of new input, leading to a delay between input detection and handling. If an application polls at a regular interval, the average added latency is half the polling interval. For example, polling at 60Hz results in a latency of about 8.33 milliseconds. While decreasing the polling interval can reduce this latency, it comes at the cost of increased CPU usage. ### Goals -1) **Enhance gamepad input handling** by transitioning from continuous polling to event-driven mechanisms, improving responsiveness. - -2) **Optimize performance** by reducing input latency and minimizing CPU overhead through a hybrid approach of event handling and threshold-based polling. - -3) **Provide flexibility for developers** by allowing efficient management of gamepad input events, enabling granular control over when and how input changes are processed. +Reduce input latency by moving away from constant polling and introducing event-driven input handling. ### Non-goals -Deprecating the existing polling mechanism: The polling mechanism will not be fully deprecated, as it is still necessary for detecting small input changes that events alone may not address. +The existing polling mechanism will not be deprecated at all. We are just proposing an alternative way of handling input events and applications are free to select whichever they prefer. ## User research Research and experimental implementations in Firefox suggest that event-driven input handling can significantly improve responsiveness and efficiency. Based on this research, the following challenges with prior implementations have been identified and addressed: -- Inadequate handling of analog button states: Previous implementations only triggered events when the button's pressed state changed, without accounting for changes in the button's numeric value or "touched" state. This hybrid approach ensures that both meaningful value changes and touch states are captured. +- Inadequate handling of analog button states: Previous implementations only triggered events when the button's pressed state changed, without accounting for changes in the button's numeric value or "touched" state. The new proposal ensures that all relevant state changes, including value adjustments and touch states, are captured, providing more comprehensive input data. -- Potential loss of gamepad state before event listeners execute: In high-speed input scenarios, if a button is quickly pressed and released, the Gamepad’s button state may indicate that the button is already unpressed by the time the listener is invoked. By combining polling with event-driven updates, this issue is mitigated by ensuring the most recent state is captured and events are triggered only when there are significant changes. +- Potential loss of gamepad state before event listeners execute: The gamepad state at the time the event was created may be lost, as the event includes a reference to the live Gamepad object, which may have updated its state before the event listener is invoked. For instance, if a button is quickly pressed and released, the Gamepad's button state may already indicate that the button is unpressed when the event listener is called. By combining polling with event-driven updates, the new approach mitigates this issue by ensuring the most recent state is captured, and events are triggered only when significant changes occur, reducing the risk of missing relevant input data. -- Unintuitive event targets: Previous event systems fired on the global window object, which was counterintuitive. The windows object is not directly tied to a specific gamepad instance and since multiple gamepads can be connected, handleing events at the gamepad level will be more logical. Most input APIs like KeyboardEvent and MouseEvent dispatch events on the relevant input device not the global window. Gamepad events on window break this consistency. -With the new proposal, events will be triggered directly on the Gamepad object that generated the event, providing a clearer and more developer-friendly experience. +- Unintuitive event targets: Previous event systems fired on the global window object, which was counterintuitive. The windows object is not directly tied to a specific gamepad instance and since multiple gamepads can be connected, handleing events at the gamepad level will be more logical. Most input APIs like KeyboardEvent and MouseEvent dispatch events on the relevant input device not the global window. Gamepad events on window break this consistency. With the new proposal, events will be triggered directly on the Gamepad object that generated the event, providing a clearer and more developer-friendly experience. -- Event spam from per-frame axis updates: When multiple axes update in a single frame, firing an event for each change could lead to excessive event firing, creating unnecessary CPU load and potential latency. By introducing thresholds for axis and button changes, this approach reduces unnecessary events, ensuring that only significant input changes are processed, thus optimizing performance. +- Event spam from per-frame axis updates:When a separate event for each changed axis on each frame of input is fired, if events are received too rapidly then they will queue, introducing input latency. ## Proposed Approach -To address the challenges of handling gamepad input efficiently while ensuring real-time responsiveness, a hybrid approach is proposed to event-driven input handling along with frame-based state consolidation. This approach combines event-based triggers with thresholding and controlled polling for axes and analog inputs, offering a balance between responsiveness and minimizing CPU usage. - -**Key Considerations** -Event-Driven Mechanisms: Gamepad inputs are inherently event-based (e.g., button presses and axis movements). We'll leverage events to notify game developers only when significant changes occur, such as a button being pressed, released, or an axis moving beyond a threshold. This reduces unnecessary event firing and improves performance. - -**Polling with Thresholding:** While events are suitable for discrete actions like button presses and releases, analog inputs (such as joystick movements) require continuous monitoring. We can use polling for these inputs in combination with a threshold. This ensures that only meaningful changes trigger events, and small, irrelevant fluctuations are ignored. This thresholding prevents event spam while still allowing for real-time responsiveness. +To address the challenges of handling gamepad input efficiently while ensuring real-time responsiveness, an approach is proposed to event-driven input handling along with frame-based state consolidation.This proposal would add four new events that fire on the Gamepad object: -**Event Types and Behavior**: -buttondown / buttonup: These events fire only when a button is pressed or released. Events are triggered for discrete actions, reducing CPU usage and avoiding unnecessary checks. +1) buttonup: These events fire only when a button is pressed (pressed attribute changes to false). -axischange: Fires when an axis value moves beyond a defined threshold (e.g., when a joystick moves a significant amount or the trigger is pressed beyond a certain value). This helps to minimize event spam by ensuring only notable changes trigger events, reducing overhead while maintaining responsiveness. +2) buttondown: These events fire only when a button is released (pressed attribute changes to true). -**How This Works with Thresholding:** -Button Events: -- For buttondown and buttonup events, the browser will still trigger these events based on button state changes (e.g., pressed or released). -- These events can happen as soon as the state changes, without needing to wait for polling, because buttons are discrete (either pressed or not). - -Axis Changes: -- The axischange event will fire only when the axis value exceeds a predefined threshold (like 0.2 or -0.2, customizable), which helps to prevent unnecessary axis events if there’s just minor, non-meaningful input fluctuation. -- Polling occurs at a lower frequency to reduce CPU usage, but when an axis value changes meaningfully, it triggers the event. +If multiple input frames have been received since the last time the event listener was invoked then the event listener is invoked only once with the most recent data received from the device. The getIntermediateEvents method returns the list of event objects representing the intermediate events that were not dispatched. An application only needs to subscribe to the types of events it is interested in handling. ```js // Example Event Code @@ -107,18 +87,9 @@ buttonup { "gamepadTimestamp": 1234.567 } -// When an axis changes and crosses a threshold. -axischange { - "type": "axischange", - "gamepadIndex": 0, - "axisIndex": 1, - "axisSnapshot": -0.65, - "gamepadTimestamp": 1234.567 -} - ``` -### Proposed IDL for GamepadButtonEvent and GamepadAxisEvent +### Proposed IDL for GamepadButtonEvent ``` interface GamepadButtonEvent : Event { @@ -126,138 +97,80 @@ interface GamepadButtonEvent : Event { readonly attribute long buttonIndex; readonly attribute GamepadButton buttonSnapshot; readonly attribute DOMHighResTimeStamp gamepadTimestamp; -}; -interface GamepadAxisEvent : Event { - readonly attribute long gamepadIndex; - readonly attribute long axisIndex; - readonly attribute double axisSnapshot; - readonly attribute DOMHighResTimeStamp gamepadTimestamp; + sequence getIntermediateEvents(); }; -``` -2) **Frame-Based Consolidated Event**: - -This approach consolidates all state changes that occur within a single frame into one event, reducing the number of events fired and preventing event spam. This is especially important for analog inputs and buttons that may change frequently within a single frame. - -- gamepadchange event: will fire once per frame, and will include all the changes (buttons and axes) that happened within that frame. This ensures that input is captured with minimal overhead while maintaining responsiveness. +``` +3) gamepadchange event: Will also fire on the Gamepad object, and will include the snapshot of all the changes (buttons and axes) that happened within that input frame. The axesChange and buttonsChanged arrays contain indices of the axes and buttons for which the value attribute changed in the input frame, and the buttonsPressed and buttonsReleased arrays contain indices of buttons for which the pressed attribute changed in the input frame. -**Handling Analog Button States** +The getCoalescedEvents() method is used to return a sequence of events that have been coalesced (combined) together. -To address issues such as inadequate handling of analog button states (e.g., button pressure or touch sensitivity), this proposal introduces improvements in the gamepadchange event, providing richer data and more precise control for developers. +How it woriks: +To avoid firing too many events in quick succession for performance issues, the browser may choose to delay firing the gamepadchange event. When this happens, the browser adds the event to an internal queue. -- Enhanced gamepadchange Event: The gamepadchange event will include not just basic button states (pressed/released), but also information about whether a button was touched and its analog value (pressure). This ensures that analog buttons are represented with more granular data, providing developers with full control over button behavior, including pressure sensitivity and touch detection. +Before firing a buttondown or buttonup event (indicating a button has been pressed or released) or before running animation callbacks (e.g., requestAnimationFrame), the event queue is flushed. This means that all the events that have been delayed will be combined into one single event, representing the union of all changes up to that point. -- Consolidate button state changes within a single event structure: Instead of firing a separate event for each button state change (pressed/released, touched, value), all these changes will be consolidated into one gamepadchange event per frame. This prevents event spam and improves the efficiency of event handling. +The final, combined gamepadchange event represents the combined state of the gamepad from all the events that were delayed and coalesced. When this event is dispatched, it contains all the changes that occurred during the delayed period. -- This will also reduce CPU overhead, as the browser will fire fewer events overall, and game developers will get all necessary button information in a single event. +4) rawgamepadchange event: This event is proposed for applications that need to consume each gamepad change as soon as it occurs. The rawgamepadchange event carries the same information as gamepadchange but is never delayed or coalesced. ```js // Example gamepadchange Event: gamepadchange { type: "gamepadchange", - gamepadIndex: 0, - // Timestamp for the event. - gamepadTimestamp: 1234.567, - changes: { - buttons: [ - { - // The index of the button that has changed. - buttonIndex: 2, - buttonSnapshot: { - // Whether the button is pressed. - pressed: true, - // Whether the button is touched. - touched: true, - // The analog value of the button (e.g., pressure). - value: 0.85 - } - } - ], - axes: [ - { - // Index of the axis that changed. - axisIndex: 1, - // The new value of the axis after change. - axisSnapshot: 0.5 - } - ] - } + gamepadSnapshot: Gamepad {id: "Example gamepad", index: 0, …}, + axesChanged: [0, 1, 2, 3], + buttonsChanged: [0], + buttonsPressed: [0], + buttonsReleased: [], } +rawgamepadchange { + type: "rawgamepadchange", + gamepadSnapshot: Gamepad {id: "Example gamepad", index: 0, …}, + axesChanged: [0, 1, 2, 3], + buttonsChanged: [0], + buttonsPressed: [0], + buttonsReleased: [], +} ``` ``` // Web IDL for the gamepadchange event +interface GamepadChangeEvent : Event { + readonly attribute Gamepad gamepadSnapshot; + readonly attribute FrozenArray axesChanged; + readonly attribute FrozenArray buttonsChanged; + readonly attribute FrozenArray buttonsPressed; + readonly attribute FrozenArray buttonsReleased; -in progress -``` - -**Event Definitions** -- **Fine-Grained Events** - These events only fire when a button is pressed, released, or an axis moves significantly. -- *Button up/down events* fires only once per actual press/release (not every frame). Avoids event spam by not sending redundant data. -- *Axis Movement Events* Threshold-based (fires only if the axis changes by more than (some %) from the last state). Prevents excessive event spam when a joystick is slightly jittering. + sequence getCoalescedEvents(); +}; -- **Frame-based Consolidated Event** -Instead of firing individual events for every small change, a frame-based gamepadchange event collects multiple updates into one. -- Fires once per animation frame (e.g., 60Hz) and includes all changes. -- Optimized for efficiency (batch updates into one event). +interface GamepadChangeEvent : Event { + readonly attribute Gamepad gamepadSnapshot; + readonly attribute FrozenArray axesChanged; + readonly attribute FrozenArray buttonsChanged; + readonly attribute FrozenArray buttonsPressed; + readonly attribute FrozenArray buttonsReleased; +}; +``` ### Dependencies on non-stable features None identified at this stage. The proposal builds upon the existing Gamepad API. -### Solving goal 1 with this approach - -Goal: Improve Gamepad Input Handling by Introducing Event-Driven Mechanisms. - -Event-based Mechanisms: For discrete inputs such as button presses, we will rely on event-driven updates (e.g., buttondown, buttonup). These events will be triggered based on changes to button states (pressed, released, etc.). - -Event Propagation: Ensure that each change (e.g., button press, axis movement) is dispatched with specific event details. For instance, an event might include which button was pressed, whether it was touched, or how far an axis moved. - -Threshold-based Polling for Analog Inputs: For continuous inputs such as joystick movements, we will use polling with thresholds. If the movement exceeds a certain threshold (e.g., 0.2), the event will fire to notify the application. This ensures that only meaningful changes in input are captured while reducing the need for continuous polling. - -Implementation Details -Polling at Intervals: Instead of polling the gamepad's state every frame, poll at a reduced frequency (e.g., every 50ms). This minimizes CPU usage while maintaining real-time responsiveness. - -Event Handling with Thresholding: -We will only fire events for significant changes in button or axis states. For analog inputs, this will involve checking if the change exceeds a threshold value before triggering an event. For buttons, this includes checking if the pressed or value exceeds a certain threshold. For axes, this means checking whether the movement exceeds a predefined value. - -```js -// Sample Code for Event Handling with Thresholding -``` - -### Solving goal 2 with this approach. - -Goal: Enhance Performance by Reducing Input Latency and Minimizing CPU Usage. -Thresholding for Analog Inputs: To optimize performance and minimize CPU usage, thresholding will be used to ensure that only significant input changes are processed. This helps avoid unnecessary polling of small, insignificant movements or fluctuations. - -Threshold Example: A threshold of 0.2 will be applied to both buttons and axes, which means that an event or update is only triggered if the value changes by more than 0.2. - -Polling Intervals: Polling will occur at controlled intervals (e.g., every 50ms) instead of continuously on every frame. This reduces CPU overhead by avoiding constant polling when there is no meaningful change. - -### Solving goal 3 with this approach. -Goal: Provide Flexibility for Developers to Efficiently Manage Gamepad Input Events. - -Customizable Event Handling: Developers can set their own threshold values for button or axis changes, enabling different types of input handling (e.g., more sensitive input for precise games like racing sims, or less sensitivity for more casual games). -Developers can decide how to handle different gamepad states (pressed, released, touched) through event detail properties such as buttonSnapshot or axisSnapshot. - -Developer-Defined Event Handling: By using events like gamepadchange, developers can register event listeners to define exactly how gamepad input should be processed, such as applying force feedback when a button is pressed or initiating an action when a specific button combination is detected. - -Access to Raw Data: The game developer can access both raw and threshold-processed data (e.g., axis values and button states) to design input controls that suit the needs of the game, such as adjusting for gamepad types, handling different input devices, or implementing specific interactions. - ## Alternatives considered -### Alternative 1: Continue Using Polling +### Alternative 1: axischange and buttonchange events -Polling remains an option but does not address the latency and CPU issues. +axischange event: that fires when a member of the Gamepad's axes array changes. +buttonchange event: that fires when a GamepadButton's value attribute changes. -### Alternative 2: Per-Frame Gamepad Snapshot Events - -This approach by it self was rejected due to potential event spam and performance concerns. - -### Alternative 3: getIntermediateEvents -Captures fine-grained input changes (such as axis movements or button states) that don't fit into the normal event cycle. Rather than having the browser fire events for every intermediate state, the game developer manually checks for significant changes and fires events when needed. The threshold mechanism replaces the need for getIntermediateEvents in this proposal. +- Event Overload: A gamepad with multiple axes and analog buttons may trigger many events within a single input frame, especially if several axis or button values are modified at the same time. +- Sequential Input Processing: Applications may prefer to process related inputs simultaneously rather than one after the other. For example, both the X and Y axes of a thumbstick are usually handled together. ## Accessibility, Privacy, and Security Considerations To prevent abuse and fingerprinting: @@ -266,17 +179,19 @@ User Gesture Required: Gamepad events won’t start firing until a user interac Limit Persistent Tracking (fingerprinting): gamepadchange event will not expose device-specific identifiers. By default, no gamepad state is exposed to the tab even when gamepads are connected. [Fingerprinting in Web](https://www.w3.org/TR/fingerprinting-guidance/) ## Stakeholder Feedback / Opposition -Firefox: Implemented experimental support for gamepadbuttondown and gamepadbuttonup, but identified issues with event spam and lost state. +Firefox: [#554 Gamepad API button and axis event](https://github.com/mozilla/standards-positions/issues/554) + +WebKit JS: Positive -Chromium: Evaluating different approaches to event-driven gamepad input handling.[Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0) +Safari: No Signal -Other Browsers: No official statements yet. +Web Developers: Positive ## References & acknowledgements Thanks to the following contributors and prior work that influenced this proposal: -Firefox’s experimental implementation. +Firefox’s experimental implementation Chromium Prior discussions on improving gamepad input handling. From 433075222ae0cf9804a8f12f01701684d6090c37 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Tue, 1 Apr 2025 15:46:10 -0700 Subject: [PATCH 03/20] Removed definitions section, updated IDL --- GamepadButtonAndAxisEvents/explainer.md | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index ae2bd862..a3e1d54a 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -17,12 +17,6 @@ This document is a starting point for engaging the community and standards bodie - Expected venue:**[W3C Web Applications Working Group](https://www.w3.org/groups/wg/webapps/)** - Current version: **This document** -### Definitions -*Fine-Grained Events* - These events only fire when a button is pressed, released. -*Button up/down events* - fires only once per actual press/release (not every frame). -*Frame-based Consolidated Event* - Instead of firing individual events for every small change, a frame-based gamepadchange event collects multiple updates into one. -- Fires once per animation frame (e.g., 60Hz) and includes all changes. - ## Introduction The current Gamepad API relies on continuous polling to detect input changes, which can lead to input latency and increased CPU usage. This proposal introduces a more efficient approach that combines event-driven input handling for button presses and releases with frame-based state consolidation, where a single event encapsulates all gamepad state changes that occur within an input frame. @@ -140,6 +134,7 @@ rawgamepadchange { // Web IDL for the gamepadchange event interface GamepadChangeEvent : Event { + const DOMString type = "gamepadchange"; readonly attribute Gamepad gamepadSnapshot; readonly attribute FrozenArray axesChanged; readonly attribute FrozenArray buttonsChanged; @@ -149,13 +144,15 @@ interface GamepadChangeEvent : Event { sequence getCoalescedEvents(); }; -interface GamepadChangeEvent : Event { +nterface RawGamepadChangeEvent : Event { + const DOMString type = "rawgamepadchange"; readonly attribute Gamepad gamepadSnapshot; readonly attribute FrozenArray axesChanged; readonly attribute FrozenArray buttonsChanged; readonly attribute FrozenArray buttonsPressed; readonly attribute FrozenArray buttonsReleased; }; + ``` ### Dependencies on non-stable features From 5654d6edaad14213e98f41645350752d23f9408a Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Tue, 1 Apr 2025 15:50:36 -0700 Subject: [PATCH 04/20] formatting --- GamepadButtonAndAxisEvents/explainer.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index a3e1d54a..9d7b1001 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -82,10 +82,9 @@ buttonup { } ``` - -### Proposed IDL for GamepadButtonEvent - ``` +// Proposed IDL for GamepadButtonEvent + interface GamepadButtonEvent : Event { readonly attribute long gamepadIndex; readonly attribute long buttonIndex; @@ -129,9 +128,8 @@ rawgamepadchange { buttonsReleased: [], } ``` - ``` -// Web IDL for the gamepadchange event +// Web IDL for the gamepadchange and rawgamepadchange event. interface GamepadChangeEvent : Event { const DOMString type = "gamepadchange"; @@ -144,7 +142,7 @@ interface GamepadChangeEvent : Event { sequence getCoalescedEvents(); }; -nterface RawGamepadChangeEvent : Event { +interface RawGamepadChangeEvent : Event { const DOMString type = "rawgamepadchange"; readonly attribute Gamepad gamepadSnapshot; readonly attribute FrozenArray axesChanged; From 2d140d9eb6f0bd7c0801ad3db41ce6583ad44530 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal <103469166+snehagarwal1@users.noreply.github.com> Date: Wed, 2 Apr 2025 10:42:41 -0700 Subject: [PATCH 05/20] Update GamepadButtonAndAxisEvents/explainer.md Co-authored-by: Gabriel Brito <80070607+gabrielsanbrito@users.noreply.github.com> --- GamepadButtonAndAxisEvents/explainer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 9d7b1001..985361eb 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -58,7 +58,7 @@ To address the challenges of handling gamepad input efficiently while ensuring r 2) buttondown: These events fire only when a button is released (pressed attribute changes to true). -If multiple input frames have been received since the last time the event listener was invoked then the event listener is invoked only once with the most recent data received from the device. The getIntermediateEvents method returns the list of event objects representing the intermediate events that were not dispatched. An application only needs to subscribe to the types of events it is interested in handling. +If multiple input frames have been received since the last time the event listener was invoked, then the event listener is invoked only once with the most recent data received from the device. The getIntermediateEvents method returns the list of event objects representing the intermediate events that were not dispatched. An application only needs to subscribe to the types of events it is interested in handling. ```js // Example Event Code From 32a3c72483b1b3be3b0ebcebd2b62762e36d7ea1 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal <103469166+snehagarwal1@users.noreply.github.com> Date: Wed, 2 Apr 2025 10:44:18 -0700 Subject: [PATCH 06/20] Update GamepadButtonAndAxisEvents/explainer.md Co-authored-by: Gabriel Brito <80070607+gabrielsanbrito@users.noreply.github.com> --- GamepadButtonAndAxisEvents/explainer.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 985361eb..3e054bdf 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -54,9 +54,9 @@ Research and experimental implementations in Firefox suggest that event-driven i ## Proposed Approach To address the challenges of handling gamepad input efficiently while ensuring real-time responsiveness, an approach is proposed to event-driven input handling along with frame-based state consolidation.This proposal would add four new events that fire on the Gamepad object: -1) buttonup: These events fire only when a button is pressed (pressed attribute changes to false). +1) buttonup: These events fire only when a button is released (pressed attribute changes to false). -2) buttondown: These events fire only when a button is released (pressed attribute changes to true). +2) buttondown: These events fire only when a button is pressed (pressed attribute changes to true). If multiple input frames have been received since the last time the event listener was invoked, then the event listener is invoked only once with the most recent data received from the device. The getIntermediateEvents method returns the list of event objects representing the intermediate events that were not dispatched. An application only needs to subscribe to the types of events it is interested in handling. From 453e2a19f00f95f474dfb2cb4464a8db8deed872 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Fri, 4 Apr 2025 16:17:13 -0700 Subject: [PATCH 07/20] rawgamepadinputchange --- GamepadButtonAndAxisEvents/explainer.md | 213 ++++++++++++------------ 1 file changed, 107 insertions(+), 106 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 9d7b1001..1b06c6b6 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -4,6 +4,7 @@ - Sneha Agarwal (https://github.com/snehagarwal_microsoft) - Steve Becker (https://github.com/SteveBeckerMSFT) +- Gabriel Brito (https://github.com/gabrielsanbrito) ## Participate - [Issue tracker] @@ -19,17 +20,18 @@ This document is a starting point for engaging the community and standards bodie ## Introduction -The current Gamepad API relies on continuous polling to detect input changes, which can lead to input latency and increased CPU usage. This proposal introduces a more efficient approach that combines event-driven input handling for button presses and releases with frame-based state consolidation, where a single event encapsulates all gamepad state changes that occur within an input frame. +The current Gamepad API relies on continuous polling to detect input changes, which can lead to input latency and increased CPU usage. This proposal suggests using a single event API that is fired as soon as an input change (button and axis) is detected. If multiple chnages happen in a rapid succession, each change triggers a separate event instead of merging these changes in to a single event. -- Event-Driven Input Handling: Immediate events (buttondown, buttonup) fire when discrete inputs like button presses or releases occur. +This proposal builds upon earlier work by Chromium developers while refining the hybrid model for improved performance, [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0) -- Frame-Based Consolidation: gamepadchange event encapsulates all the gamepad state changes that occurred within an input frame, preventing event spam. - -This proposal builds upon earlier work by Chromium developers while refining the hybrid model for improved performance. ## User-Facing Problem -The Gamepad API does not natively support event-driven input handling, and applications must rely only on continuous polling for updates. This polling-based approach introduces input latency because scripts cannot sync their polling with the arrival of new input, leading to a delay between input detection and handling. If an application polls at a regular interval, the average added latency is half the polling interval. For example, polling at 60Hz results in a latency of about 8.33 milliseconds. While decreasing the polling interval can reduce this latency, it comes at the cost of increased CPU usage. +The Gamepad API lacks event-driven input handling, forcing applications to rely on continuous polling. This introduces input latency, as scripts cannot synchronize with new input, and reducing the polling interval to mitigate latency increases CPU usage, impacting efficiency and battery life. + +This issue is particularly problematic for Xbox cloud gaming (xCloud), which streams Xbox games to a browser and depends on real-time gamepad input. To minimize latency,they currently poll every 4ms, but this high-frequency polling increases CPU usage and battery drain, especially on laptops and mobile devices. Additionally, since the Gamepad API operates only on the main UI thread, it can cause thread contention, particularly on low-end devices. In some cases, these constraints force them to reduce polling frequency, increasing input latency as a trade-off. + +A more efficient solution would be an event-driven Gamepad API, similar to mouse and keyboard events, enabling real-time responsiveness without the overhead of constant polling. ### Goals @@ -39,116 +41,116 @@ Reduce input latency by moving away from constant polling and introducing event- The existing polling mechanism will not be deprecated at all. We are just proposing an alternative way of handling input events and applications are free to select whichever they prefer. -## User research - -Research and experimental implementations in Firefox suggest that event-driven input handling can significantly improve responsiveness and efficiency. Based on this research, the following challenges with prior implementations have been identified and addressed: - -- Inadequate handling of analog button states: Previous implementations only triggered events when the button's pressed state changed, without accounting for changes in the button's numeric value or "touched" state. The new proposal ensures that all relevant state changes, including value adjustments and touch states, are captured, providing more comprehensive input data. - -- Potential loss of gamepad state before event listeners execute: The gamepad state at the time the event was created may be lost, as the event includes a reference to the live Gamepad object, which may have updated its state before the event listener is invoked. For instance, if a button is quickly pressed and released, the Gamepad's button state may already indicate that the button is unpressed when the event listener is called. By combining polling with event-driven updates, the new approach mitigates this issue by ensuring the most recent state is captured, and events are triggered only when significant changes occur, reducing the risk of missing relevant input data. - -- Unintuitive event targets: Previous event systems fired on the global window object, which was counterintuitive. The windows object is not directly tied to a specific gamepad instance and since multiple gamepads can be connected, handleing events at the gamepad level will be more logical. Most input APIs like KeyboardEvent and MouseEvent dispatch events on the relevant input device not the global window. Gamepad events on window break this consistency. With the new proposal, events will be triggered directly on the Gamepad object that generated the event, providing a clearer and more developer-friendly experience. - -- Event spam from per-frame axis updates:When a separate event for each changed axis on each frame of input is fired, if events are received too rapidly then they will queue, introducing input latency. - ## Proposed Approach -To address the challenges of handling gamepad input efficiently while ensuring real-time responsiveness, an approach is proposed to event-driven input handling along with frame-based state consolidation.This proposal would add four new events that fire on the Gamepad object: - -1) buttonup: These events fire only when a button is pressed (pressed attribute changes to false). - -2) buttondown: These events fire only when a button is released (pressed attribute changes to true). - -If multiple input frames have been received since the last time the event listener was invoked then the event listener is invoked only once with the most recent data received from the device. The getIntermediateEvents method returns the list of event objects representing the intermediate events that were not dispatched. An application only needs to subscribe to the types of events it is interested in handling. - -```js -// Example Event Code - -// When a button is pressed. -buttondown { - "type": "buttondown", - "gamepadIndex": 0, - "buttonIndex": 2, - "buttonSnapshot": { "pressed": true, "touched": true, "value": 1 }, - "gamepadTimestamp": 1234.567 -} - -// When a button is released. -buttonup { - "type": "buttonup", - "gamepadIndex": 0, - "buttonIndex": 2, - "buttonSnapshot": { "pressed": false, "touched": false, "value": 0 }, - "gamepadTimestamp": 1234.567 -} - -``` -``` -// Proposed IDL for GamepadButtonEvent - -interface GamepadButtonEvent : Event { - readonly attribute long gamepadIndex; - readonly attribute long buttonIndex; - readonly attribute GamepadButton buttonSnapshot; - readonly attribute DOMHighResTimeStamp gamepadTimestamp; +To address the challenges of input latency an approach is proposed to event-driven input handling. This proposal would add one new event that fires on the Gamepad object: - sequence getIntermediateEvents(); -}; - -``` -3) gamepadchange event: Will also fire on the Gamepad object, and will include the snapshot of all the changes (buttons and axes) that happened within that input frame. The axesChange and buttonsChanged arrays contain indices of the axes and buttons for which the value attribute changed in the input frame, and the buttonsPressed and buttonsReleased arrays contain indices of buttons for which the pressed attribute changed in the input frame. +rawgamepadinputchange event fires on the Gamepad object and delivers a snapshot of all gamepad input changes (axes and buttons) that occurred since the last event. -The getCoalescedEvents() method is used to return a sequence of events that have been coalesced (combined) together. +- axesChanged and buttonsChanged contain the indices of all axes and buttons whose value changed since last input. -How it woriks: -To avoid firing too many events in quick succession for performance issues, the browser may choose to delay firing the gamepadchange event. When this happens, the browser adds the event to an internal queue. +- buttonsPressed and buttonsReleased include indices of buttons whose pressed state changed (i.e., transitioned to or from a pressed state) since last input. -Before firing a buttondown or buttonup event (indicating a button has been pressed or released) or before running animation callbacks (e.g., requestAnimationFrame), the event queue is flushed. This means that all the events that have been delayed will be combined into one single event, representing the union of all changes up to that point. +- The gamepadSnapshot provides the full current state of the gamepad, including button states, axis values, and metadata like ID, index, and timestamp. -The final, combined gamepadchange event represents the combined state of the gamepad from all the events that were delayed and coalesced. When this event is dispatched, it contains all the changes that occurred during the delayed period. +- A new rawgamepadinputchange event is dispatched for every individual input state change, enabling latency-sensitive applications (such as game streaming) to respond immediately to input events, without relying on polling. These events are not delayed or coalesced, and reflect each change as it happens, offering precise, real-time responsiveness. -4) rawgamepadchange event: This event is proposed for applications that need to consume each gamepad change as soon as it occurs. The rawgamepadchange event carries the same information as gamepadchange but is never delayed or coalesced. +- This design is especially beneficial for scenarios like cloud gaming, rhythm games (guitar hero), or real-time multiplayer controls where timely input processing is critical. ```js -// Example gamepadchange Event: -gamepadchange { - type: "gamepadchange", - gamepadSnapshot: Gamepad {id: "Example gamepad", index: 0, …}, - axesChanged: [0, 1, 2, 3], - buttonsChanged: [0], - buttonsPressed: [0], - buttonsReleased: [], -} - -rawgamepadchange { - type: "rawgamepadchange", - gamepadSnapshot: Gamepad {id: "Example gamepad", index: 0, …}, - axesChanged: [0, 1, 2, 3], +// Example rawgamepadinputchange Event: +rawgamepadinputchange { + type: "rawgamepadchange", + gamepadSnapshot: Gmepad { + id: "Xbox Wireless Controller (STANDARD GAMEPAD Vendor: 045e Product: 02fd)", + index: 0, + connected: true, + mapping: "standard", + buttons: [ + // index 0 - button A + { pressed: true, touched: true, value: 1 }, + // index 1 - button B + { pressed: false, touched: false, value: 0 }, + ... + ], + // [left stick X, left stick Y, right stick X, right stick Y] + axes: [0.25, -0.5, 0.0, 0.0], + timestamp: 9123456.789 + }, + // Left stick X and Y moved + axesChanged: [0, 1], + // button index 0 (A) state changed buttonsChanged: [0], + // button index 0 (A) pressed buttonsPressed: [0], + // No button was released buttonsReleased: [], + // High-res timestamp when change was detected + gamepadTimestamp: 9123456.789 } ``` +## Proposed IDL ``` -// Web IDL for the gamepadchange and rawgamepadchange event. - -interface GamepadChangeEvent : Event { - const DOMString type = "gamepadchange"; - readonly attribute Gamepad gamepadSnapshot; - readonly attribute FrozenArray axesChanged; - readonly attribute FrozenArray buttonsChanged; - readonly attribute FrozenArray buttonsPressed; - readonly attribute FrozenArray buttonsReleased; - - sequence getCoalescedEvents(); +[Exposed=Window, SecureContext] +interface Gamepad { + // New attributes + attribute EventHandler onrawgamepadinputchange; + + // Existing attributes + readonly attribute DOMString id; + readonly attribute long index; + readonly attribute boolean connected; + readonly attribute DOMHighResTimeStamp timestamp; + readonly attribute GamepadMappingType mapping; + readonly attribute FrozenArray axes; + readonly attribute FrozenArray buttons; }; +``` +### RawGamepadInputChangeEvent interface IDL, used for rawgamepadinputchange. +``` +[Exposed=Window, SecureContent] +interface RawGamepadInputChangeEvent : Event { + constructor(DOMString type, optional RawGamepadInputChangeEventInit eventInitDict = {}); -interface RawGamepadChangeEvent : Event { - const DOMString type = "rawgamepadchange"; readonly attribute Gamepad gamepadSnapshot; - readonly attribute FrozenArray axesChanged; - readonly attribute FrozenArray buttonsChanged; - readonly attribute FrozenArray buttonsPressed; - readonly attribute FrozenArray buttonsReleased; + readonly attribute FrozenArray axesChanged; + readonly attribute FrozenArray buttonsChanged; + readonly attribute FrozenArray buttonsPressed; + readonly attribute FrozenArray buttonsReleased; +}; +``` +## Developer code sample + +```JS +// Listen for when a gamepad is connected +window.ongamepadconnected = (connectEvent) => { + const gamepad = connectEvent.gamepad; + + console.log(`Gamepad connected: ${gamepad.id} (index: ${gamepad.index})`); + + // Listen for input changes on this gamepad. + gamepad.onrawgamepadinputchange = (changeEvent) => { + const snapshot = changeEvent.gamepadSnapshot; + + for (const axisIndex of changeEvent.axesChanged) { + const axisValue = snapshot.axes[axisIndex]; + console.log(`Axis ${axisIndex} on gamepad ${snapshot.index} changed to ${axisValue}`); + } + + for (const buttonIndex of changeEvent.buttonsChanged) { + const buttonValue = snapshot.buttons[buttonIndex]?.value ?? 'unknown'; + console.log(`Button ${buttonIndex} on gamepad ${snapshot.index} changed to ${buttonValue}`); + } + }; + + // Listen for button press. + gamepad.onbuttondown = (buttonEvent) => { + console.log(`Button ${buttonEvent.buttonIndex} on gamepad ${buttonEvent.gamepadIndex} pressed`); + }; + + // Listen for button release. + gamepad.onbuttonup = (buttonEvent) => { + console.log(`Button ${buttonEvent.buttonIndex} on gamepad ${buttonEvent.gamepadIndex} released`); + }; }; ``` @@ -158,14 +160,13 @@ interface RawGamepadChangeEvent : Event { None identified at this stage. The proposal builds upon the existing Gamepad API. ## Alternatives considered +gamepadchange event: Similar to rawgamepadinputchange event but instead the getCoalescedEvents() method is used to return a sequence of events that have been coalesced (combined) together. -### Alternative 1: axischange and buttonchange events +How it works: To avoid firing too many events in quick succession for performance issues, the browser may choose to delay firing the gamepadchange event. When this happens, the browser adds the event to an internal queue. -axischange event: that fires when a member of the Gamepad's axes array changes. -buttonchange event: that fires when a GamepadButton's value attribute changes. +Before firing a buttondown or buttonup event (indicating a button has been pressed or released) or before running animation callbacks (e.g., requestAnimationFrame), the event queue is flushed. This means that all the events that have been delayed will be combined into one single event, representing the union of all changes up to that point. -- Event Overload: A gamepad with multiple axes and analog buttons may trigger many events within a single input frame, especially if several axis or button values are modified at the same time. -- Sequential Input Processing: Applications may prefer to process related inputs simultaneously rather than one after the other. For example, both the X and Y axes of a thumbstick are usually handled together. +The final, combined gamepadchange event represents the combined state of the gamepad from all the events that were delayed and coalesced. When this event is dispatched, it contains all the changes that occurred during the delayed period. ## Accessibility, Privacy, and Security Considerations To prevent abuse and fingerprinting: @@ -176,17 +177,17 @@ Limit Persistent Tracking (fingerprinting): gamepadchange event will not expose ## Stakeholder Feedback / Opposition Firefox: [#554 Gamepad API button and axis event](https://github.com/mozilla/standards-positions/issues/554) -WebKit JS: Positive - Safari: No Signal +xCloud: Positive + Web Developers: Positive ## References & acknowledgements Thanks to the following contributors and prior work that influenced this proposal: -Firefox’s experimental implementation +Firefox’s experimental implementation. Chromium Prior discussions on improving gamepad input handling. From 90c968cb727d0712af99cb232dbf4a483b718f3f Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Fri, 4 Apr 2025 20:10:51 -0700 Subject: [PATCH 08/20] Update rawgamepadinput changed event and usage details. --- GamepadButtonAndAxisEvents/explainer.md | 54 +++++++++++++++---------- 1 file changed, 33 insertions(+), 21 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 1b06c6b6..478c5936 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -20,16 +20,16 @@ This document is a starting point for engaging the community and standards bodie ## Introduction -The current Gamepad API relies on continuous polling to detect input changes, which can lead to input latency and increased CPU usage. This proposal suggests using a single event API that is fired as soon as an input change (button and axis) is detected. If multiple chnages happen in a rapid succession, each change triggers a separate event instead of merging these changes in to a single event. +The current Gamepad API relies on continuous polling to detect input changes, which can lead to input latency and increased CPU usage. This proposal suggests using a single event API that is fired as soon as an input change (button and axis) is detected. If multiple changes happen in a rapid succession, each change triggers a separate event instead of merging these changes in to a single event. -This proposal builds upon earlier work by Chromium developers while refining the hybrid model for improved performance, [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0) +This proposal builds upon earlier work by Chromium developers for improved performance, [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0) ## User-Facing Problem -The Gamepad API lacks event-driven input handling, forcing applications to rely on continuous polling. This introduces input latency, as scripts cannot synchronize with new input, and reducing the polling interval to mitigate latency increases CPU usage, impacting efficiency and battery life. +The Gamepad API lacks event-driven input handling, forcing applications to rely on continuous polling to query for changes in gamepad input. The continuous polling introduces input latency, as scripts cannot synchronize with new input fast enough. If the script reduces the polling interval to mitigate input latency, it increases CPU usage, impacting efficiency and battery life of the device. -This issue is particularly problematic for Xbox cloud gaming (xCloud), which streams Xbox games to a browser and depends on real-time gamepad input. To minimize latency,they currently poll every 4ms, but this high-frequency polling increases CPU usage and battery drain, especially on laptops and mobile devices. Additionally, since the Gamepad API operates only on the main UI thread, it can cause thread contention, particularly on low-end devices. In some cases, these constraints force them to reduce polling frequency, increasing input latency as a trade-off. +This issue is particularly problematic for Xbox cloud gaming (xCloud), which streams Xbox games to a browser and depends on real-time gamepad input. To minimize latency, they currently poll every 4ms, but this high-frequency polling increases CPU usage and battery drain, especially on laptops and mobile devices. Additionally, since the Gamepad API is only supported on the main UI thread, it causes thread contention, particularly on low-end devices. In some cases, these constraints force them to reduce polling frequency, increasing input latency as a trade-off. A more efficient solution would be an event-driven Gamepad API, similar to mouse and keyboard events, enabling real-time responsiveness without the overhead of constant polling. @@ -39,7 +39,7 @@ Reduce input latency by moving away from constant polling and introducing event- ### Non-goals -The existing polling mechanism will not be deprecated at all. We are just proposing an alternative way of handling input events and applications are free to select whichever they prefer. +The existing polling mechanism will not be deprecated. We are just proposing an alternative way of handling input events and applications are free to select whichever they prefer. ## Proposed Approach To address the challenges of input latency an approach is proposed to event-driven input handling. This proposal would add one new event that fires on the Gamepad object: @@ -76,7 +76,7 @@ rawgamepadinputchange { axes: [0.25, -0.5, 0.0, 0.0], timestamp: 9123456.789 }, - // Left stick X and Y moved + // Left stick X and Y moved since last event. axesChanged: [0, 1], // button index 0 (A) state changed buttonsChanged: [0], @@ -113,9 +113,9 @@ interface RawGamepadInputChangeEvent : Event { readonly attribute Gamepad gamepadSnapshot; readonly attribute FrozenArray axesChanged; - readonly attribute FrozenArray buttonsChanged; + readonly attribute FrozenArray buttonsValueChanged; readonly attribute FrozenArray buttonsPressed; - readonly attribute FrozenArray buttonsReleased; + readonly attribute FrozenArray buttonsTouched; }; ``` ## Developer code sample @@ -123,7 +123,10 @@ interface RawGamepadInputChangeEvent : Event { ```JS // Listen for when a gamepad is connected window.ongamepadconnected = (connectEvent) => { - const gamepad = connectEvent.gamepad; + + const connectedGamepads = navigator.getGamepads(); + + const gamepad = connectedGamepads[connectEvent.gamepad.index]; console.log(`Gamepad connected: ${gamepad.id} (index: ${gamepad.index})`); @@ -136,20 +139,29 @@ window.ongamepadconnected = (connectEvent) => { console.log(`Axis ${axisIndex} on gamepad ${snapshot.index} changed to ${axisValue}`); } - for (const buttonIndex of changeEvent.buttonsChanged) { - const buttonValue = snapshot.buttons[buttonIndex]?.value ?? 'unknown'; - console.log(`Button ${buttonIndex} on gamepad ${snapshot.index} changed to ${buttonValue}`); + // Analog buttons (ex: triggers) + for (let buttonIndex of changeEvent.buttonsValueChanged) { + const buttonValue = changeEvent.gamepadSnapshot.buttons[buttonIndex].value; + console.log('button ' + buttonIndex + + ' on gamepad ' + changeEvent.gamepadSnapshot.index + + ' changed to value ' + buttonValue); } - }; - // Listen for button press. - gamepad.onbuttondown = (buttonEvent) => { - console.log(`Button ${buttonEvent.buttonIndex} on gamepad ${buttonEvent.gamepadIndex} pressed`); - }; + // Binary buttons pressed + for (let buttonIndex of changeEvent.buttonsPressed) { + const buttonPressed = changeEvent.gamepadSnapshot.buttons[buttonIndex].pressed; + console.log('button ' + buttonIndex + + ' on gamepad ' + changeEvent.gamepadSnapshot.index + + ' changed pressed to ' + buttonPressed); + } - // Listen for button release. - gamepad.onbuttonup = (buttonEvent) => { - console.log(`Button ${buttonEvent.buttonIndex} on gamepad ${buttonEvent.gamepadIndex} released`); + // Buttons touched + for (let buttonIndex of changeEvent.buttonsTouched) { + const buttonTouched = changeEvent.gamepadSnapshot.buttons[buttonIndex].touched; + console.log('button ' + buttonIndex + + ' on gamepad ' + changeEvent.gamepadSnapshot.index + + ' changed touched to ' + buttonTouched); + } }; }; @@ -164,7 +176,7 @@ gamepadchange event: Similar to rawgamepadinputchange event but instead the getC How it works: To avoid firing too many events in quick succession for performance issues, the browser may choose to delay firing the gamepadchange event. When this happens, the browser adds the event to an internal queue. -Before firing a buttondown or buttonup event (indicating a button has been pressed or released) or before running animation callbacks (e.g., requestAnimationFrame), the event queue is flushed. This means that all the events that have been delayed will be combined into one single event, representing the union of all changes up to that point. +Before running animation callbacks (e.g., requestAnimationFrame), the event queue is flushed. This means that all the events that have been delayed will be combined into one single event, representing the union of all changes up to that point. The final, combined gamepadchange event represents the combined state of the gamepad from all the events that were delayed and coalesced. When this event is dispatched, it contains all the changes that occurred during the delayed period. From 36b7a7e531de3619153c4a549601fa9ed4cdd4df Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Fri, 4 Apr 2025 20:32:52 -0700 Subject: [PATCH 09/20] typo --- GamepadButtonAndAxisEvents/explainer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index bc5ca556..2c373136 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -60,7 +60,7 @@ rawgamepadinputchange event fires on the Gamepad object and delivers a snapshot // Example rawgamepadinputchange Event: rawgamepadinputchange { type: "rawgamepadchange", - gamepadSnapshot: Gmepad { + gamepadSnapshot: Gamepad { id: "Xbox Wireless Controller (STANDARD GAMEPAD Vendor: 045e Product: 02fd)", index: 0, connected: true, From 0ac8399eae5276929ebdcb98ec95ec1e28f8ecbe Mon Sep 17 00:00:00 2001 From: Sneha Agarwal <103469166+snehagarwal1@users.noreply.github.com> Date: Tue, 8 Apr 2025 20:28:58 -0700 Subject: [PATCH 10/20] Update GamepadButtonAndAxisEvents/explainer.md Co-authored-by: Gabriel Brito <80070607+gabrielsanbrito@users.noreply.github.com> --- GamepadButtonAndAxisEvents/explainer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 2c373136..9cb860b3 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -27,7 +27,7 @@ This proposal builds upon earlier work by Chromium developers for improved perfo ## User-Facing Problem -The Gamepad API lacks event-driven input handling, forcing applications to rely on continuous polling to query for changes in gamepad input. The continuous polling introduces input latency, as scripts cannot synchronize with new input fast enough. If the script reduces the polling interval to mitigate input latency, it increases CPU usage, impacting efficiency and battery life of the device. +The Gamepad API lacks event-driven input handling, forcing applications to rely on continuous polling to query for changes in gamepad input. The continuous polling introduces input latency, as scripts cannot synchronize with new input fast enough. If the script increases the polling frequency to mitigate input latency, it increases CPU usage, impacting efficiency and battery life of the device. This issue is particularly problematic for Xbox cloud gaming (xCloud), which streams Xbox games to a browser and depends on real-time gamepad input. To minimize latency, they currently poll every 4ms, but this high-frequency polling increases CPU usage and battery drain, especially on laptops and mobile devices. Additionally, since the Gamepad API is only supported on the main UI thread, it causes thread contention, particularly on low-end devices. In some cases, these constraints force them to reduce polling frequency, increasing input latency as a trade-off. From 39222c49500513d337e05b237942d51b1941004e Mon Sep 17 00:00:00 2001 From: Sneha Agarwal <103469166+snehagarwal1@users.noreply.github.com> Date: Tue, 8 Apr 2025 20:29:12 -0700 Subject: [PATCH 11/20] Update GamepadButtonAndAxisEvents/explainer.md Co-authored-by: Gabriel Brito <80070607+gabrielsanbrito@users.noreply.github.com> --- GamepadButtonAndAxisEvents/explainer.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 9cb860b3..df08faba 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -183,8 +183,7 @@ Before running animation callbacks (e.g., requestAnimationFrame), the event queu The final, combined gamepadchange event represents the combined state of the gamepad from all the events that were delayed and coalesced. When this event is dispatched, it contains all the changes that occurred during the delayed period. ## Accessibility, Privacy, and Security Considerations -To prevent abuse and fingerprinting: -User Gesture Required: Gamepad events won’t start firing until a user interacts with the gamepad (e.g., pressing a button). [Gamepad user gesture](https://www.w3.org/TR/gamepad/#dfn-gamepad-user-gesture) +To prevent abuse and fingerprinting, a ["gamepad user gesture"](https://www.w3.org/TR/gamepad/#dfn-gamepad-user-gesture) will be required before `RawGamepadInputChange` events start firing (e.g., pressing a button). Limit Persistent Tracking (fingerprinting): gamepadchange event will not expose device-specific identifiers. By default, no gamepad state is exposed to the tab even when gamepads are connected. [Fingerprinting in Web](https://www.w3.org/TR/fingerprinting-guidance/) From f8e756e9ef3ae38a15539efd1bd4f5a9e2ad1612 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal <103469166+snehagarwal1@users.noreply.github.com> Date: Tue, 8 Apr 2025 20:34:24 -0700 Subject: [PATCH 12/20] Update GamepadButtonAndAxisEvents/explainer.md Co-authored-by: Gabriel Brito <80070607+gabrielsanbrito@users.noreply.github.com> --- GamepadButtonAndAxisEvents/explainer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index df08faba..386648fc 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -42,7 +42,7 @@ Reduce input latency by moving away from constant polling and introducing event- The existing polling mechanism will not be deprecated. We are just proposing an alternative way of handling input events and applications are free to select whichever they prefer. ## Proposed Approach -To address the challenges of input latency an approach is proposed to event-driven input handling. This proposal would add one new event that fires on the Gamepad object: +To address the challenges of input latency an approach is proposed to event-driven input handling. This proposal would add one new event that fires on the [Gamepad](https://w3c.github.io/gamepad/#dom-gamepad) object: rawgamepadinputchange event fires on the Gamepad object and delivers a snapshot of all gamepad input changes (axes and buttons) that occurred since the last event. From c8290e4b39bcf3c2e7f01ae72fa1468052f3af48 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Wed, 9 Apr 2025 10:36:12 -0700 Subject: [PATCH 13/20] addressed comments --- GamepadButtonAndAxisEvents/explainer.md | 56 +++++++++++-------------- 1 file changed, 25 insertions(+), 31 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 2c373136..514f2b73 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -8,7 +8,7 @@ ## Participate - [Issue tracker] -- [Discussion forum] +- [Gamepad API input events #662](https://github.com/w3ctag/design-reviews/issues/662) ## Status of this Document @@ -29,7 +29,7 @@ This proposal builds upon earlier work by Chromium developers for improved perfo The Gamepad API lacks event-driven input handling, forcing applications to rely on continuous polling to query for changes in gamepad input. The continuous polling introduces input latency, as scripts cannot synchronize with new input fast enough. If the script reduces the polling interval to mitigate input latency, it increases CPU usage, impacting efficiency and battery life of the device. -This issue is particularly problematic for Xbox cloud gaming (xCloud), which streams Xbox games to a browser and depends on real-time gamepad input. To minimize latency, they currently poll every 4ms, but this high-frequency polling increases CPU usage and battery drain, especially on laptops and mobile devices. Additionally, since the Gamepad API is only supported on the main UI thread, it causes thread contention, particularly on low-end devices. In some cases, these constraints force them to reduce polling frequency, increasing input latency as a trade-off. +This issue is particularly problematic for cloud gaming platforms that stream games to a browser and rely on real-time gamepad input. For instance, to minimize latency, they often poll as frequently as every 4ms, but this high-frequency polling increases CPU usage and battery drain, especially on laptops and mobile devices. Additionally, because the Gamepad API is only supported on the main UI thread, it can cause thread contention—particularly on low-end devices. In some cases, these constraints force developers to reduce polling frequency, increasing input latency as a trade-off. A more efficient solution would be an event-driven Gamepad API, similar to mouse and keyboard events, enabling real-time responsiveness without the overhead of constant polling. @@ -42,22 +42,18 @@ Reduce input latency by moving away from constant polling and introducing event- The existing polling mechanism will not be deprecated. We are just proposing an alternative way of handling input events and applications are free to select whichever they prefer. ## Proposed Approach -To address the challenges of input latency an approach is proposed to event-driven input handling. This proposal would add one new event that fires on the Gamepad object: +To address the challenges of input latency, this proposal introduces a new event-driven mechanism: The `rawgamepadinputchange` event. This event fires directly on the Gamepad object and delivers real-time updates for each input change, eliminating the need for high-frequency polling. The `rawgamepadinputchange` event includes detailed information about the state of the gamepad at the moment of change: -rawgamepadinputchange event fires on the Gamepad object and delivers a snapshot of all gamepad input changes (axes and buttons) that occurred since the last event. +- `axesChanged` and `buttonsChanged`: Arrays of indices indicating which axes or button values changed since the last event. -- axesChanged and buttonsChanged contain the indices of all axes and buttons whose value changed since last input. +- `buttonsPressed` and `buttonsReleased`: Indices of buttons whose pressed state transitioned (from pressed to released or vice versa). -- buttonsPressed and buttonsReleased include indices of buttons whose pressed state changed (i.e., transitioned to or from a pressed state) since last input. +- `gamepadSnapshot`: A complete snapshot of the gamepad's current state, including all axes, buttons, ID, index, and timestamp. -- The gamepadSnapshot provides the full current state of the gamepad, including button states, axis values, and metadata like ID, index, and timestamp. - -- A new rawgamepadinputchange event is dispatched for every individual input state change, enabling latency-sensitive applications (such as game streaming) to respond immediately to input events, without relying on polling. These events are not delayed or coalesced, and reflect each change as it happens, offering precise, real-time responsiveness. - -- This design is especially beneficial for scenarios like cloud gaming, rhythm games (guitar hero), or real-time multiplayer controls where timely input processing is critical. +A new `rawgamepadinputchange` event is dispatched for every individual input state change, without delay or coalescing, enabling latency-sensitive applications—such as rhythm games, cloud gaming, or real-time multiplayer scenarios—to respond immediately and accurately to input. +## Example `rawgamepadinputchange` Event ```js -// Example rawgamepadinputchange Event: rawgamepadinputchange { type: "rawgamepadchange", gamepadSnapshot: Gamepad { @@ -66,33 +62,33 @@ rawgamepadinputchange { connected: true, mapping: "standard", buttons: [ - // index 0 - button A + // index 0 - button A. { pressed: true, touched: false, value: 1.0 }, - // index 1 - button B + // index 1 - button B. { pressed: false, touched: true, value: 0.0 }, - // index 2 - analog button + // index 2 - analog button. { pressed: false, touched: false, value: 0.5 }, ... ], - // [left stick X, left stick Y, right stick X, right stick Y] + // [left stick X, left stick Y, right stick X, right stick Y]. axes: [0.25, -0.5, 0.0, 0.0], timestamp: 9123456.789 }, // Left stick X and Y moved since last event. axesChanged: [0, 1], - // button index 0 was pressed, button index 2 value changed + // button index 0 was pressed, button index 2 value changed. buttonsValueChanged: [0, 2], - // button index 0 pressed + // button index 0 pressed. buttonsPressed: [0], - // button index 1 was touched + // button index 1 was touched. buttonsTouched: [1], - // High-res timestamp when change was detected + // High-res timestamp when change was detected. gamepadTimestamp: 9123456.789 } ``` ## Proposed IDL ``` -[Exposed=Window, SecureContext] +[Exposed=Window] interface Gamepad { // New attributes attribute EventHandler onrawgamepadinputchange; @@ -107,9 +103,9 @@ interface Gamepad { readonly attribute FrozenArray buttons; }; ``` -### RawGamepadInputChangeEvent interface IDL, used for rawgamepadinputchange. +### `RawGamepadInputChangeEvent` interface IDL, used for `rawgamepadinputchange`. ``` -[Exposed=Window, SecureContent] +[Exposed=Window] interface RawGamepadInputChangeEvent : Event { constructor(DOMString type, optional RawGamepadInputChangeEventInit eventInitDict = {}); @@ -123,7 +119,7 @@ interface RawGamepadInputChangeEvent : Event { ## Developer code sample ```JS -// Listen for when a gamepad is connected +// Listen for when a gamepad is connected. window.ongamepadconnected = (connectEvent) => { const connectedGamepads = navigator.getGamepads(); @@ -141,7 +137,7 @@ window.ongamepadconnected = (connectEvent) => { console.log(`Axis ${axisIndex} on gamepad ${snapshot.index} changed to ${axisValue}`); } - // Analog buttons (ex: triggers) + // Analog buttons (ex: triggers). for (let buttonIndex of changeEvent.buttonsValueChanged) { const buttonValue = changeEvent.gamepadSnapshot.buttons[buttonIndex].value; console.log('button ' + buttonIndex + @@ -149,7 +145,7 @@ window.ongamepadconnected = (connectEvent) => { ' changed to value ' + buttonValue); } - // Binary buttons pressed + // Binary buttons pressed. for (let buttonIndex of changeEvent.buttonsPressed) { const buttonPressed = changeEvent.gamepadSnapshot.buttons[buttonIndex].pressed; console.log('button ' + buttonIndex + @@ -157,7 +153,7 @@ window.ongamepadconnected = (connectEvent) => { ' changed pressed to ' + buttonPressed); } - // Buttons touched + // Buttons touched. for (let buttonIndex of changeEvent.buttonsTouched) { const buttonTouched = changeEvent.gamepadSnapshot.buttons[buttonIndex].touched; console.log('button ' + buttonIndex + @@ -174,7 +170,7 @@ window.ongamepadconnected = (connectEvent) => { None identified at this stage. The proposal builds upon the existing Gamepad API. ## Alternatives considered -gamepadchange event: Similar to rawgamepadinputchange event but instead the getCoalescedEvents() method is used to return a sequence of events that have been coalesced (combined) together. +gamepadchange event: Similar to `rawgamepadinputchange` event but instead the getCoalescedEvents() method is used to return a sequence of events that have been coalesced (combined) together. How it works: To avoid firing too many events in quick succession for performance issues, the browser may choose to delay firing the gamepadchange event. When this happens, the browser adds the event to an internal queue. @@ -186,15 +182,13 @@ The final, combined gamepadchange event represents the combined state of the gam To prevent abuse and fingerprinting: User Gesture Required: Gamepad events won’t start firing until a user interacts with the gamepad (e.g., pressing a button). [Gamepad user gesture](https://www.w3.org/TR/gamepad/#dfn-gamepad-user-gesture) -Limit Persistent Tracking (fingerprinting): gamepadchange event will not expose device-specific identifiers. By default, no gamepad state is exposed to the tab even when gamepads are connected. [Fingerprinting in Web](https://www.w3.org/TR/fingerprinting-guidance/) +Limit Persistent Tracking (fingerprinting): `rawgamepadinputchange` event will not expose device-specific identifiers. By default, no gamepad state is exposed to the tab even when gamepads are connected. [Fingerprinting in Web](https://www.w3.org/TR/fingerprinting-guidance/) ## Stakeholder Feedback / Opposition Firefox: [#554 Gamepad API button and axis event](https://github.com/mozilla/standards-positions/issues/554) Safari: No Signal -xCloud: Positive - Web Developers: Positive ## References & acknowledgements From 7c08165a4b7a2281604c3cf75654339ee69953bf Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Fri, 11 Apr 2025 14:49:44 -0700 Subject: [PATCH 14/20] updated with current polling example and title, comments addressed --- GamepadButtonAndAxisEvents/explainer.md | 77 ++++++++++++++++++------- 1 file changed, 57 insertions(+), 20 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 33c8f6b9..0779f2ce 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -1,13 +1,12 @@ -# Gamepad Button and Axis Events +# Event-Driven Gamepad Input API ## Authors: -- Sneha Agarwal (https://github.com/snehagarwal_microsoft) -- Steve Becker (https://github.com/SteveBeckerMSFT) -- Gabriel Brito (https://github.com/gabrielsanbrito) +- [Sneha Agarwal](https://github.com/snehagarwal_microsoft) +- [Steve Becker](https://github.com/SteveBeckerMSFT) +- [Gabriel Brito](https://github.com/gabrielsanbrito) ## Participate -- [Issue tracker] - [Gamepad API input events #662](https://github.com/w3ctag/design-reviews/issues/662) ## Status of this Document @@ -20,10 +19,9 @@ This document is a starting point for engaging the community and standards bodie ## Introduction -The current Gamepad API relies on continuous polling to detect input changes, which can lead to input latency and increased CPU usage. This proposal suggests using a single event API that is fired as soon as an input change (button and axis) is detected. If multiple changes happen in a rapid succession, each change triggers a separate event instead of merging these changes in to a single event. - -This proposal builds upon earlier work by Chromium developers for improved performance, [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0) +This explainer proposes an event-driven Gamepad Input API for the web, designed to complement the existing polling-based model. By enabling input events to be dispatched in response to changes in gamepad state, this API aims to support low-latency scenarios such as cloud gaming, where timely and reactive input delivery is critical. +This proposal builds on earlier work by Chromium engineers, [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0). ## User-Facing Problem @@ -31,7 +29,36 @@ The Gamepad API lacks event-driven input handling, forcing applications to rely This issue is particularly problematic for cloud gaming platforms that stream games to a browser and rely on real-time gamepad input. For instance, to minimize latency, they often poll as frequently as every 4ms, but this high-frequency polling increases CPU usage and battery drain, especially on laptops and mobile devices. Additionally, because the Gamepad API is only supported on the main UI thread, it can cause thread contention—particularly on low-end devices. In some cases, these constraints force developers to reduce polling frequency, increasing input latency as a trade-off. -A more efficient solution would be an event-driven Gamepad API, similar to mouse and keyboard events, enabling real-time responsiveness without the overhead of constant polling. +A more efficient solution would be an event-driven Gamepad API, similar to mouse and keyboard events, enabling real-time responsiveness without the overhead of constant polling. + +### Developer code sample of existing poll based API +```JS +function pollGamepadInput() { + const gamepads = navigator.getGamepads(); + + for (const gamepad of gamepads) { + if (!gamepad) continue; + // Example: Logging the first axis and button. + const axisX = gamepad.axes[0]; + const buttonA = gamepad.buttons[0].pressed; + + console.log(`Axis X: ${axisX}, Button A pressed: ${buttonA}`); + } + + // Continue polling in the next animation frame. + requestAnimationFrame(pollGamepadInput); +} + +// Start polling. +window.addEventListener('gamepadconnected', () => { + console.log('Gamepad connected!'); + requestAnimationFrame(pollGamepadInput); +}); +``` +#### Key Points: +- navigator.getGamepads() returns a snapshot of all connected gamepads. +- The polling loop is driven by `requestAnimationFrame`, typically around 60Hz (matching display refresh rate), which is much lower than the internal OS poll rate (eg., 250Hz). +- This method is not sufficient for latency-critical web applications. ### Goals @@ -165,26 +192,36 @@ window.ongamepadconnected = (connectEvent) => { ``` -### Dependencies on non-stable features +## Alternatives considered +`gamepadinputchange` event: Similar to `rawgamepadinputchange` event but instead the `getCoalescedEvents()` method is used to return a sequence of events that have been coalesced (combined) together. This event was proposed in the [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0). -None identified at this stage. The proposal builds upon the existing Gamepad API. +### Proposed IDL +``` +interface GamepadChangeEvent : Event { + readonly attribute Gamepad gamepadSnapshot; + readonly attribute FrozenArray axesChanged; + readonly attribute FrozenArray buttonsChanged; + readonly attribute FrozenArray buttonsPressed; + readonly attribute FrozenArray buttonsReleased; -## Alternatives considered -gamepadchange event: Similar to `rawgamepadinputchange` event but instead the getCoalescedEvents() method is used to return a sequence of events that have been coalesced (combined) together. + sequence getCoalescedEvents(); +}; -How it works: To avoid firing too many events in quick succession for performance issues, the browser may choose to delay firing the gamepadchange event. When this happens, the browser adds the event to an internal queue. +``` +### How it works: + To avoid firing too many events in quick succession for performance issues, the browser may choose to delay firing the gamepadchange event. When this happens, the browser adds the event to an internal queue. -Before running animation callbacks (e.g., requestAnimationFrame), the event queue is flushed. This means that all the events that have been delayed will be combined into one single event, representing the union of all changes up to that point. +Before running animation callbacks (e.g., `requestAnimationFrame`), the event queue is flushed. This means that all the events that have been delayed will be combined into one single event, representing the union of all changes up to that point. The final, combined gamepadchange event represents the combined state of the gamepad from all the events that were delayed and coalesced. When this event is dispatched, it contains all the changes that occurred during the delayed period. ## Accessibility, Privacy, and Security Considerations To prevent abuse and fingerprinting, a ["gamepad user gesture"](https://www.w3.org/TR/gamepad/#dfn-gamepad-user-gesture) will be required before `RawGamepadInputChange` events start firing (e.g., pressing a button). -Limit Persistent Tracking (fingerprinting): `rawgamepadinputchange` event will not expose device-specific identifiers. By default, no gamepad state is exposed to the tab even when gamepads are connected. [Fingerprinting in Web](https://www.w3.org/TR/fingerprinting-guidance/) +Limit Persistent Tracking (fingerprinting): `rawgamepadinputchange` event will not expose any new state that is not already exposed by polling [Fingerprinting in Web](https://www.w3.org/TR/fingerprinting-guidance/). ## Stakeholder Feedback / Opposition -Firefox: [#554 Gamepad API button and axis event](https://github.com/mozilla/standards-positions/issues/554) +Firefox: No Signal Safari: No Signal @@ -199,9 +236,9 @@ Firefox’s experimental implementation. Chromium Prior discussions on improving gamepad input handling. Many thanks for valuable feedback and advice from: -- Gabriel Brito -- Matt Reynolds -- Steve Becker +- [Steve Becker](https://github.com/SteveBeckerMSFT) +- [Gabriel Brito](https://github.com/gabrielsanbrito) +- [Matt Reynolds](https://github.com/nondebug) Thanks to the following proposals, projects, libraries, frameworks, and languages for their work on similar problems that influenced this proposal. From 0d7c55b12bb7f224ac50802eb9b6d0db3da5c784 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Fri, 11 Apr 2025 15:16:55 -0700 Subject: [PATCH 15/20] Gamepad inherits from EventTarget --- GamepadButtonAndAxisEvents/explainer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 0779f2ce..ec06b131 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -116,7 +116,7 @@ rawgamepadinputchange { ## Proposed IDL ``` [Exposed=Window] -interface Gamepad { +interface Gamepad : EventTarget { // New attributes attribute EventHandler onrawgamepadinputchange; From 73713487f919d0aa9ed4a64064fceae0c75192a8 Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Wed, 23 Apr 2025 13:38:19 -0700 Subject: [PATCH 16/20] added definition section --- GamepadButtonAndAxisEvents/explainer.md | 33 +++++++++++-------------- 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index ec06b131..812bb61c 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -14,14 +14,19 @@ This document is a starting point for engaging the community and standards bodies in developing collaborative solutions fit for standardization. As the solutions to the problems described in this document progress along the standards-track, we will retain this document as an archive and use this section to keep the community up-to-date with the most current standards venue and content location of future work and discussions. - This document status: **Active** -- Expected venue:**[W3C Web Applications Working Group](https://www.w3.org/groups/wg/webapps/)** +- Expected venue: **[W3C Web Applications Working Group](https://www.w3.org/groups/wg/webapps/)** - Current version: **This document** ## Introduction This explainer proposes an event-driven Gamepad Input API for the web, designed to complement the existing polling-based model. By enabling input events to be dispatched in response to changes in gamepad state, this API aims to support low-latency scenarios such as cloud gaming, where timely and reactive input delivery is critical. -This proposal builds on earlier work by Chromium engineers, [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0). +This proposal builds on earlier work by Chromium engineers, which explored event-driven gamepad input handling. (Note: The original proposal is documented in a [Google Doc](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0).) + +## Definitions + +### Input Frame: +Each input frame refers to a single timestamped update of a gamepad’s state, typically derived from a HID (Human Interface Device) report, including all button and axis values at that moment in time. ## User-Facing Problem @@ -57,8 +62,7 @@ window.addEventListener('gamepadconnected', () => { ``` #### Key Points: - navigator.getGamepads() returns a snapshot of all connected gamepads. -- The polling loop is driven by `requestAnimationFrame`, typically around 60Hz (matching display refresh rate), which is much lower than the internal OS poll rate (eg., 250Hz). -- This method is not sufficient for latency-critical web applications. +- The polling loop is driven by `requestAnimationFrame`, typically around 60Hz (matching display refresh rate), which is much lower than the internal OS poll rate (eg., 250Hz). This mismatch can result in missed input updates, making the 60Hz rate insufficient for latency-critical applications like cloud gaming. ### Goals @@ -69,7 +73,7 @@ Reduce input latency by moving away from constant polling and introducing event- The existing polling mechanism will not be deprecated. We are just proposing an alternative way of handling input events and applications are free to select whichever they prefer. ## Proposed Approach -To address the challenges of input latency, this proposal introduces a new event-driven mechanism: The `rawgamepadinputchange` event. This event fires directly on the [Gamepad](https://w3c.github.io/gamepad/#dom-gamepad) object and delivers real-time updates for each input change, eliminating the need for high-frequency polling. The `rawgamepadinputchange` event includes detailed information about the state of the gamepad at the moment of change: +To address the challenges of input latency, this proposal introduces a new event-driven mechanism: The `rawgamepadinputchange` event. This event fires directly on the [Gamepad](https://w3c.github.io/gamepad/#dom-gamepad) object and delivers real-time updates for each input frame, eliminating the need for high-frequency polling. The `rawgamepadinputchange` event includes detailed information about the state of the gamepad at the moment of change: - `axesChanged` and `buttonsChanged`: Arrays of indices indicating which axes or button values changed since the last event. @@ -77,7 +81,7 @@ To address the challenges of input latency, this proposal introduces a new event - `gamepadSnapshot`: A complete snapshot of the gamepad's current state, including all axes, buttons, ID, index, and timestamp. -A new `rawgamepadinputchange` event is dispatched for every individual input state change, without delay or coalescing, enabling latency-sensitive applications—such as rhythm games, cloud gaming, or real-time multiplayer scenarios—to respond immediately and accurately to input. +A new `rawgamepadinputchange` event is dispatched for every gamepad input state change, without delay or coalescing, enabling latency-sensitive applications—such as rhythm games, cloud gaming, or real-time multiplayer scenarios—to respond immediately and accurately to input. ## Example `rawgamepadinputchange` Event ```js @@ -167,33 +171,26 @@ window.ongamepadconnected = (connectEvent) => { // Analog buttons (ex: triggers). for (let buttonIndex of changeEvent.buttonsValueChanged) { const buttonValue = changeEvent.gamepadSnapshot.buttons[buttonIndex].value; - console.log('button ' + buttonIndex + - ' on gamepad ' + changeEvent.gamepadSnapshot.index + - ' changed to value ' + buttonValue); + console.log(`button ${buttonIndex} on gamepad ${changeEvent.gamepadSnapshot.index} changed to value ${buttonValue}`); } // Binary buttons pressed. for (let buttonIndex of changeEvent.buttonsPressed) { const buttonPressed = changeEvent.gamepadSnapshot.buttons[buttonIndex].pressed; - console.log('button ' + buttonIndex + - ' on gamepad ' + changeEvent.gamepadSnapshot.index + - ' changed pressed to ' + buttonPressed); + console.log(`button ${buttonIndex} on gamepad ${changeEvent.gamepadSnapshot.index} changed to value ${buttonPressed}`); } // Buttons touched. for (let buttonIndex of changeEvent.buttonsTouched) { const buttonTouched = changeEvent.gamepadSnapshot.buttons[buttonIndex].touched; - console.log('button ' + buttonIndex + - ' on gamepad ' + changeEvent.gamepadSnapshot.index + - ' changed touched to ' + buttonTouched); + console.log(`button ${buttonIndex} on gamepad ${changeEvent.gamepadSnapshot.index} changed to value ${buttonTouched}`); } }; }; - ``` ## Alternatives considered -`gamepadinputchange` event: Similar to `rawgamepadinputchange` event but instead the `getCoalescedEvents()` method is used to return a sequence of events that have been coalesced (combined) together. This event was proposed in the [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0). +`gamepadinputchange` event: Similar to `rawgamepadinputchange` event but instead the `getCoalescedEvents()` method is used to return a sequence of events that have been coalesced (combined) together. While `gamepadinputchange` reduces the number of events by coalescing them, this approach introduces latency and may result in missed intermediate states, making it unsuitable for scenarios requiring immediate responsiveness. This event was proposed in the [Original Proposal](https://docs.google.com/document/d/1rnQ1gU0iwPXbO7OvKS6KO9gyfpSdSQvKhK9_OkzUuKE/edit?pli=1&tab=t.0). ### Proposed IDL ``` @@ -209,7 +206,7 @@ interface GamepadChangeEvent : Event { ``` ### How it works: - To avoid firing too many events in quick succession for performance issues, the browser may choose to delay firing the gamepadchange event. When this happens, the browser adds the event to an internal queue. +To avoid firing too many events in quick succession for performance issues, the browser may choose to delay firing the gamepadchange event. When this happens, the browser adds the event to an internal queue. Before running animation callbacks (e.g., `requestAnimationFrame`), the event queue is flushed. This means that all the events that have been delayed will be combined into one single event, representing the union of all changes up to that point. From bde524013a84688c926b6cdada0c58195a0e056c Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Wed, 23 Apr 2025 16:29:25 -0700 Subject: [PATCH 17/20] Comments addressed --- GamepadButtonAndAxisEvents/explainer.md | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 812bb61c..4e359ca8 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -113,26 +113,15 @@ rawgamepadinputchange { buttonsPressed: [0], // button index 1 was touched. buttonsTouched: [1], - // High-res timestamp when change was detected. - gamepadTimestamp: 9123456.789 } ``` ## Proposed IDL ``` [Exposed=Window] -interface Gamepad : EventTarget { - // New attributes +partial interface Gamepad : EventTarget { attribute EventHandler onrawgamepadinputchange; - - // Existing attributes - readonly attribute DOMString id; - readonly attribute long index; - readonly attribute boolean connected; - readonly attribute DOMHighResTimeStamp timestamp; - readonly attribute GamepadMappingType mapping; - readonly attribute FrozenArray axes; - readonly attribute FrozenArray buttons; }; + ``` ### `RawGamepadInputChangeEvent` interface IDL, used for `rawgamepadinputchange`. ``` From ea5aae23c521a9ac7eead52f8e4fd5e1c05b568c Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Thu, 24 Apr 2025 12:16:10 -0700 Subject: [PATCH 18/20] user problem section updated --- GamepadButtonAndAxisEvents/explainer.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index 4e359ca8..a115ba1d 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -30,11 +30,9 @@ Each input frame refers to a single timestamped update of a gamepad’s state, t ## User-Facing Problem -The Gamepad API lacks event-driven input handling, forcing applications to rely on continuous polling to query for changes in gamepad input. The continuous polling introduces input latency, as scripts cannot synchronize with new input fast enough. If the script increases the polling frequency to mitigate input latency, it increases CPU usage, impacting efficiency and battery life of the device. +The Gamepad API lacks event-driven input handling, requiring applications to poll for input state changes. This polling model makes it difficult to achieve low-latency responsiveness, as input changes can be missed between polling intervals. Developers working on latency-sensitive applications, such as cloud gaming platforms, have reported needing to poll at very high frequencies to detect input as quickly as possible. However, even with aggressive polling, scripts may still struggle to react in real time, especially under heavy UI thread load or on resource-constrained devices. -This issue is particularly problematic for cloud gaming platforms that stream games to a browser and rely on real-time gamepad input. For instance, to minimize latency, they often poll as frequently as every 4ms, but this high-frequency polling increases CPU usage and battery drain, especially on laptops and mobile devices. Additionally, because the Gamepad API is only supported on the main UI thread, it can cause thread contention—particularly on low-end devices. In some cases, these constraints force developers to reduce polling frequency, increasing input latency as a trade-off. - -A more efficient solution would be an event-driven Gamepad API, similar to mouse and keyboard events, enabling real-time responsiveness without the overhead of constant polling. +These limitations make it challenging to deliver consistent, low-latency input experiences in the browser. An event-driven Gamepad API (similar to existing keyboard and mouse event models) would allow applications to respond immediately to input changes as they occur, reducing the reliance on polling and enabling real-time responsiveness for latency-critical use cases. ### Developer code sample of existing poll based API ```JS From 44d10a489f491c1534bfa695a563b8ddb2321d5d Mon Sep 17 00:00:00 2001 From: Sneha Agarwal <103469166+snehagarwal1@users.noreply.github.com> Date: Thu, 24 Apr 2025 17:31:06 -0700 Subject: [PATCH 19/20] Update GamepadButtonAndAxisEvents/explainer.md Co-authored-by: Gabriel Brito <80070607+gabrielsanbrito@users.noreply.github.com> --- GamepadButtonAndAxisEvents/explainer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index a115ba1d..ebb4609f 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -60,7 +60,7 @@ window.addEventListener('gamepadconnected', () => { ``` #### Key Points: - navigator.getGamepads() returns a snapshot of all connected gamepads. -- The polling loop is driven by `requestAnimationFrame`, typically around 60Hz (matching display refresh rate), which is much lower than the internal OS poll rate (eg., 250Hz). This mismatch can result in missed input updates, making the 60Hz rate insufficient for latency-critical applications like cloud gaming. +- The polling loop is driven by `requestAnimationFrame`, typically around 60Hz (matching display refresh rate), which is much lower than the internal OS poll rate (eg., 250Hz). This mismatch can result in missed input updates, making the 60Hz rate insufficient for latency-critical applications like cloud gaming. ### Goals From fdbe6e30ccedc3784cbc6a1b0d9132e07c5cf73a Mon Sep 17 00:00:00 2001 From: Sneha Agarwal Date: Fri, 25 Apr 2025 10:57:44 -0700 Subject: [PATCH 20/20] def RawGamepadInputChangeEvent --- GamepadButtonAndAxisEvents/explainer.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/GamepadButtonAndAxisEvents/explainer.md b/GamepadButtonAndAxisEvents/explainer.md index a115ba1d..644d7ce3 100644 --- a/GamepadButtonAndAxisEvents/explainer.md +++ b/GamepadButtonAndAxisEvents/explainer.md @@ -28,6 +28,9 @@ This proposal builds on earlier work by Chromium engineers, which explored event ### Input Frame: Each input frame refers to a single timestamped update of a gamepad’s state, typically derived from a HID (Human Interface Device) report, including all button and axis values at that moment in time. +### RawGamepadInputChangeEvent: +An event that represents a snapshot of a gamepad’s state at the moment a new input frame is received from the gamepad device. Each event corresponds to a full input report (e.g., a HID report) and contains the complete state of all buttons, axes. This event enables applications to react to input in a timely, event-driven manner, as an alternative to polling via navigator.getGamepads(). + ## User-Facing Problem The Gamepad API lacks event-driven input handling, requiring applications to poll for input state changes. This polling model makes it difficult to achieve low-latency responsiveness, as input changes can be missed between polling intervals. Developers working on latency-sensitive applications, such as cloud gaming platforms, have reported needing to poll at very high frequencies to detect input as quickly as possible. However, even with aggressive polling, scripts may still struggle to react in real time, especially under heavy UI thread load or on resource-constrained devices. @@ -105,10 +108,12 @@ rawgamepadinputchange { }, // Left stick X and Y moved since last event. axesChanged: [0, 1], - // button index 0 was pressed, button index 2 value changed. + // button index 0 was pressed and released, button index 2 value changed. buttonsValueChanged: [0, 2], // button index 0 pressed. buttonsPressed: [0], + // button index 0 released. + buttonsReleased: [0], // button index 1 was touched. buttonsTouched: [1], } @@ -131,6 +136,7 @@ interface RawGamepadInputChangeEvent : Event { readonly attribute FrozenArray axesChanged; readonly attribute FrozenArray buttonsValueChanged; readonly attribute FrozenArray buttonsPressed; + readonly attribute FrozenArray buttonsReleased; readonly attribute FrozenArray buttonsTouched; }; ``` @@ -167,6 +173,12 @@ window.ongamepadconnected = (connectEvent) => { console.log(`button ${buttonIndex} on gamepad ${changeEvent.gamepadSnapshot.index} changed to value ${buttonPressed}`); } + // Binary buttons released. + for (let buttonIndex of changeEvent.buttonsReleased) { + const buttonReleased = changeEvent.gamepadSnapshot.buttons[buttonIndex].released; + console.log(`button ${buttonIndex} on gamepad ${changeEvent.gamepadSnapshot.index} changed to value ${buttonReleased}`); + } + // Buttons touched. for (let buttonIndex of changeEvent.buttonsTouched) { const buttonTouched = changeEvent.gamepadSnapshot.buttons[buttonIndex].touched;