Sony's New AI Patent Predicts Button Presses Using Finger-Camera Tech

Sony has recently filed a new patent, WO2025010132, titled "TIMED INPUT/ACTION RELEASE," which aims to reduce latency in future gaming hardware. This innovative approach involves using an AI model supported by additional sensors to predict and streamline user inputs, enhancing the responsiveness of games. This development comes on the heels of Sony's introduction of PlayStation Spectral Super Resolution (PSSR) with the PlayStation 5 Pro, which, while capable of upscaling to 4K, can introduce latency issues due to frame generation technologies.

Latency, the delay between a player's action and the game's response, can significantly impact gameplay, particularly in genres like twitch shooters where responsiveness is crucial. Sony's patent seeks to address this by predicting what button a player will press next, using a machine-learning AI model and external sensors. For instance, a camera could be used to monitor the player's controller, providing input to the AI to anticipate the next command. The patent also suggests the possibility of using the controller's buttons themselves as sensors, leveraging Sony's experience with analog buttons in past controllers.

This new Sony patent could be a game-changer for PlayStation. Image credit: Sony Interactive Entertainment.

The rationale behind Sony's filing is clear: "there can be latency between the user's input action and the system's subsequent processing and execution of the command. This in turn results in delayed execution of the command and unintended consequences in the game itself." By predicting user inputs, Sony aims to minimize these delays, ensuring a smoother and more responsive gaming experience.

While the exact implementation of this technology in future consoles like the PlayStation 6 remains uncertain, the patent demonstrates Sony's commitment to tackling latency issues, especially in light of popular rendering technologies like FSR 3 and DLSS 3, which can add frame latency. If successfully integrated, this technology could significantly enhance gameplay in scenarios requiring high framerates and low latency, such as competitive gaming.