Kinect Will Recognise Sign Language

Kinect Will Recognise Sign Language

Patent reveals Microsoft’s device will be able to understand American Sign Language, lip read and even track toe movement.

Kinect will be able to recognize ASL (American Sign Language), a recently published patent confirms. As reported by Slash Gear, this will allow anyone proficient in the sign language dialect to input letters words and phrases. If ASL isn’t known, the system will also be able to lip-read.

"Where the user is unable to speak, he may be prevented from joining in the voice chat." Explains the patent. "Even though he would be able to type input, this may be a laborious and slow process to someone fluent in ASL. Under the present system, he could make ASL gestures to convey his thoughts, which would then be transmitted to the other users for auditory display. The user’s input could be converted to voice locally, or by each remote computer.

"In this situation, for example, when the user kills another user’s character, that victorious, though speechless, user would be able to tell the other user that he had been ‘PWNED’. In another embodiment, a user may be able to speak or make the facial motions corresponding to speaking words. The system may then parse those facial motions to determine the user’s intended words and process them according to the context under which they were inputted to the system."

As well as aligning with Apple’s promotion of iPhone 4′s FaceTime as being ideal for deaf users, Kinect’s sign language recognition could lead to brand new game experiences and is potentially indicative of an industry becoming more aware of disabled users.

The patent also goes into detail on the ‘skeletal mapping’ system Microsoft uses to map players’ bodies and movements, stating that the device can even recognise players’ toes.

"[Within the skeletal mapping system] a variety of joints and bones are identified: each hand, each forearm, each elbow, each bicep, each shoulder, each hip, each thigh, each knee, each foreleg, each foot, the head, the torso, the top and bottom of the spine, and the waist. Where more points are tracked, additional features may be identified, such as the bones and joints of the fingers or toes, or individual features of the face, such as the nose and eyes."