Another fascinating feature of the Kinect sensing device in Xbox One stems from its infrared sensor, which can identify objects in a completely darkened room. It can recognize people and track bodies even without any light visible to the naked eye. It can identify a hand pose from four meters away, see the fingers of a child, and remember your identity even minus room illumination.

The wider field of view makes it possible for more players to play an Xbox One game at the same time. With the new console, as many as six players can crowd into one scene. A tall adult can play with a small child without either being squeezed out of the picture. Users get a better experience if they’re standing close by, farther away, or on the periphery of the room.

And the improved hand-pose recognition enables users to interact with the Xbox One just by using their hands—no controller necessary. Thanks to the infrared camera, hand activities can be identified at any illumination, or with none at all. Prior hand-pose solutions were able to deliver speed or accuracy, but not both. The hand-pose solution jointly devised by the Xbox team and Microsoft Research can do both.

Based on his experience with both hand-pose and facial recognition, Krupka was able to find solutions that take into account system-level concerns.