Ultra-Low-Latency Touch Sensing
Click to play. Videos muted by default.
This work centered on a new capacitive touch sensing architecture designed to dramatically reduce input latency while supporting high-resolution, multi-touch interaction. The core system, Fast Multi-Touch (FMT), replaced traditional sequential scanning with simultaneous orthogonal signal transmission, enabling touch sensing at kilohertz rates with end-to-end latencies well below human perceptual thresholds.
The video on the left shows the very first system I assembled (nick-named Frankenstein for obvious reasons) and the video on the right shows the resulting commercial technology built out by our startup's team of engineers.
The sensing approach was designed to be flexible, supporting dense touch input on both planar and non-planar surfaces, and forming the foundation for later form factors, such as touch-sensitive keyboard and steering wheels, and capabilities such as user differentiation.
Complementing the hardware, my work at Tactual Labs also included several efforts to reduce software latency - including projects on event handling, OS modifications, and AI-informed prediction.
Papers and Patents
 |
High rate, low-latency multi-touch sensing with simultaneous orthogonal multiplexing Darren Leigh, Clifton Forlines, Ricardo Jota, Steven Sanders, and Daniel Wigdor. 2014. High rate, low-latency multi-touch sensing with simultaneous orthogonal multiplexing. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14). ACM, New York, NY, USA, 355-364. |
 |
GhostID: Enabling Non-Persistent User Differentiation in Frequency-Division Capacitive Multi-Touch Sensors Sidharth Sahdev, Clifton Forlines, Ricardo Jota, Bruno De Araujo, Braon Moseley, Jonathan Deber, Steven Sanders, Darren Leigh, and Daniel Wigdor. 2017. GhostID: Enabling Non-Persistent User Differentiation in Frequency-Division Capacitive Multi-Touch Sensors. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 15-27. |
 |
Hammer Time!: A Low-Cost, High Precision, High Accuracy Tool to Measure the Latency of Touchscreen Devices Jonathan Deber, Bruno Araujo, Ricardo Jota, Clifton Forlines, Darren Leigh, Steven Sanders, and Daniel Wigdor. 2016. Hammer Time!: A Low-Cost, High Precision, High Accuracy Tool to Measure the Latency of Touchscreen Devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2857-2868. |
 |
How Much Faster is Fast Enough?: User Perception of Latency & Latency Improvements in Direct and Indirect Touch Jonathan Deber, Ricardo Jota, Clifton Forlines, and Daniel Wigdor. 2015. How Much Faster is Fast Enough?: User Perception of Latency & Latency Improvements in Direct and Indirect Touch. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1827-1836. |
 |
Zero-latency tapping: using hover information to predict touch locations and eliminate touchdown latency Haijun Xia, Ricardo Jota, Benjamin McCanny, Zhe Yu, Clifton Forlines, Karan Singh, and Daniel Wigdor. 2014. Zero-latency tapping: using hover information to predict touch locations and eliminate touchdown latency. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14). ACM, New York, NY, USA, 205-214. |
 |
U.S. Patent 11,460,951 Transmitting And Receiving System And Method For Bidirectional Orthogonal Signaling Sensors (with Darren Leigh) |
 |
U.S. Patent 11,068,105 System and method for performing hit testing in a graphical user interface (with Bruno Rodrigues De Araujo, Jonathan Deber, Ricardo Costa, and Daniel Wigdor) |
 |
U.S. Patent 10,871,855 Area filtering for low-latency and high-latency input event paths from a single touch sensor (with Bruno Rodrigues De Araujo, Ricardo Costa, and Steven Sanders) |
 |
U.S. Patent 10,691,279 Dynamic assignment of possible channels in a touch sensor (with Darren Leigh, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 10,691,251 Orthogonal signaling touch user, hand and object discrimination systems and methods (with Darren Leigh, Ricardo Costa, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 10,620,746 Decimation supplementation strategies for input event processing (with Ricardo Costa, Daniel Wigdor, Steven Sanders, and Bruno De Araujo) |
 |
U.S. Patent 10,620,737 Differential transmission for reduction of cross-talk in projective capacitive touch sensors (with Darren Leigh) |
 |
U.S. Patent 10,592,050 Systems and methods for using hover information to predict touch locations and reduce or eliminate touchdown latency (with Ricardo Costa, Daniel Wigdor, Karan Singh, and Haijun Xia) |
 |
U.S. Patent 10,572,036 Transmitting and receiving system and method for bidirectional orthogonal signaling sensors (with Darren Leigh) |
 |
U.S. Patent 10,558,293 Pressure informed decimation strategies for input event processing (with Daniel Wigdor) |
 |
U.S. Patent 10,551,985 Fast multi-touch noise reduction (with Darren Leigh, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 10,423,273 Differential transmission for reduction of cross-talk in projective capacitive touch sensors (with Darren Leigh) |
 |
U.S. Patent 10,289,256 Dynamic assignment of possible channels in a touch sensor (with Darren Leigh, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 10,241,760 System and method for performing hit testing in a graphical user interface (with Bruno De Araujo, Jonathan Deber, Ricardo Costa, and Daniel Wigdor) |
 |
U.S. Patent 10,241,620 Area filtering for low-latency and high-latency input event paths from a single touch sensor (with Bruno De Araujo, Ricardo Costa, and Steven Sanders) |
 |
U.S. Patent 10,241,612 Decimation supplementation strategies for input event processing (with Ricardo Costa, Daniel Wigdor, Steven Sanders, and Bruno De Araujo) |
 |
U.S. Patent 10,222,952 Hybrid systems and methods for low-latency user input processing and feedback (with Daniel Wigdor, Steven Sanders, and Ricardo Costa) |
 |
U.S. Patent 10,216,602 Tool to measure the latency of touchscreen devices (with Jonathan Deber, Bruno De Araujo, Ricardo Costa, Darren Leigh, Steven Sanders, and Daniel Wigdor) |
 |
U.S. Patent 10,191,579 Transmitting and receiving system and method for bidirectional orthogonal signaling sensors (with Darren Leigh) |
| |
U.S. Patent 10,168,849 Fast multi-touch noise reduction (with Darren Leigh, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 10,133,400 Pressure informed decimation strategies for input event processing (with Daniel Wigdor) |
 |
U.S. Patent 10,088,952 Systems and methods for using hover information to predict touch locations and reduce or eliminate touchdown latency (with Ricardo Costa, Daniel Wigdor, Karan Singh, and Haijun Xia) |
 |
U.S. Patent 10,019,125 Fast multi-touch noise reduction (with Darren Leigh, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 9,990,696 Decimation strategies for input event processing (with Ricardo Costa, Daniel Wigdor, Steven Sanders, and Bruno De Araujo) |
 |
U.S. Patent 9,933,880 Orthogonal signaling touch user, hand and object discrimination systems and methods (with Darren Leigh, Ricardo Costa, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 9,927,959 Hybrid systems and methods for low-latency user input processing and feedback (with Daniel Wigdor, Steven Sanders, and Ricardo Costa) |
| |
U.S. Patent 9,846,920 Decimation strategies for input event processing (with Ricardo Costa, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 9,841,839 System for Measuring Latency on a Touch Device (with Darren Leigh, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 9,830,014 Reducing control response latency with defined cross-control behavior (with Benjamin McCanny and Daniel Wigdor) |
 |
U.S. Patent 9,811,214 Fast multi-touch noise reduction (with Darren Leigh, Daniel Wigdor, and Steven Sanders) |
 |
U.S. Patent 9,710,116 Frequency conversion in a touch sensor (with Darren Leigh, Daniel Wigdor, and Steven Sanders) |
| |
U.S. Patent 9,632,615 Reducing control response latency with defined cross-control behavior (with Benjamin McCanny and Daniel Wigdor) |
 |
U.S. Patent 9,507,500 Hybrid systems and methods for low-latency user input processing and feedback (with Daniel Wigdor, Steven Sanders, and Ricardo Costa) |
