Gloves that claim to translate sign language into speech are gimmicky at best and are not at all capable of actually interpreting a sign language into speech. I'll attempt to explain why they don't work, and why they'll likely continue to fall short into the perceivable future.
NB: In this post I'll be using American Sign Language as my sign language example and English for the spoken language example, though the points are relevant for all signed and spoken languages. Words in all caps are Gloss, what you call it when you write one language in another, and are used to represent ASL in English.
The Technology
At their core, the gloves interpret the movement of the hand joints (and optionally velocity changes, and for the rest of this post I'll assume that they do) to create vector-like patterns that are then matched against a preset database of handshape + movement patterns to find the corresponding English equivalent. This creates a one-to-one* relationship between a gesture and a spoken word/phrase. Therefore if one were to sign I WILL GO HOME then the system will say "I will go home," and if one were to sign WILL GO HOME I (proper ASL grammar) then the system will say "Will go home I." This will be important later.
(* It's possible that an AI system, such as an expert system or neural network, that can use fuzzy logic or contextual information to create a one-to-many relationship, but I've not seen this demonstrated by any such devices and it does not negate the points made in this post. I will assume that these do not exist to any significant extent for purposes of this post.)
What is a Sign?
Signs (as in: Sign Language) are defined by five properties: handshape, position, movement, non-manual markers (NMM), and context. (
Non-manual markers are actions and movements made with something other than the hands to add to or change meanings of signs.) That means that a handshape and movement made on one's forehead, for example, would mean something different than the same handshape and movement made on one's chin or one's chest (see: FATHER and MOTHER and FINE), or a handshape and movement done in the same position but done with or without an NMM would mean something different (see: NOT-YET and LATE).
"You're Sure?"
In spoken language, we commonly use inflection to differentiate sentences from questions. As a simple example, say "You're hungry." and then "You're hungry?" Chances are you'll notice the inflection at the end of "hungry" changes even though the words have remained the same. In ASL, these "inflections" are created using NMMs, specifically the movement of the eyebrows. Aside from the NMM, the signs for "How old are you?" and "You're old." are exactly the same (OLD YOU), but they're obviously quite different in their meaning.
Already you should notice that the gloves are not capturing true signs. Of the five properties they are capturing only two, so the majority of the information is being discarded. These gloves would not be able to differentiate between the examples given above, and so already we see a huge limitation to the devices. But let's continue.
What Isn't a Sign
Classifiers are sign-like gestures that lack one or more of the properties of a true sign and are used in a pantomime-like fashion to convey meaning through common understanding. For example, if I were to extend my hand toward the table in a C-like handshape, pantomime raising something to my lips and drinking, one might reasonably understand that I was indicating drinking something from a glass. If I were to start the same motion, but instead take my hand and invert it, and allow my gaze to fall to the floor as I did so, one might reasonably infer that I was pouring something out of a glass. But because these are not true signs (in these examples, the classifier was lacking a defined movement and position), because they're not strictly definable in a pattern matching algorithm, they're meaningless to a computer. The only reason these two examples would be meaningful to humans is because of our common knowledge of what a glass is and how it's used, as well as our ability to imagine a glass in my hand as I made the gestures.
Classifiers can make up a large part, even a majority, of any signed conversation. As another example, describing how you want your hair cut in sign language would require several classifiers, non-manual markers, and pantomime which would be missed by these devices, as well as contextual understanding, which even a reasonably complex neural network would miss.
YESTERDAY I GO STORE BUY-BUY APPLE CARROT SODA
It needs to be stated because it's a common misconception: signed languages are not manual versions of spoken languages. ASL is not English. Not only are the vocabularies very different, but the grammar is unique as well. The section title is a well structured ASL sentence that would be interpreted to English as "Yesterday I went to the store and bought apples, carrots, and sodas." You can see similarities but you can see some distinctions as well. Sign languages are not verbal languages in the proper sense where words are combined in a specific order to make sentences. They're visual languages, more akin to taking meaning from a painting than from a paragraph. The structure of the language itself allows meaning to be expressed in ways that can't be done in spoken languages, and these significant differences would be completely lost in any such direct translation device.
Final Verdict
Simply put, the technology doesn't exist to interpret a sign language into speech. Frankly, it is almost inconceivable that it would exist within our lifetimes. Even if it did, a pair of gloves would never be able to capture enough information to do a correct interpretation. Even if a device was able to capture the position and motion of the fingers, hands, arms, shoulders, the body shifts, the facial expressions, and all the NMMs, it would
still fall short of being able to interpret sign language because it would need to be able to do what a human does: imagine, empathize, and extract information from common understanding. In my professional opinion, nothing short of the AI singularity would allow a computer to fully and meaningfully interpret between signed and spoken languages. In their current form, these current or similar devices would work to translate, at best, an incredibly small portion of a sign language and only in very limited contexts. Emotion and expression, a giant part of communicating in any signed language, are completely lost. Body shifting would be lost. Indirect noun references would (most likely) be lost. Too much information would be lost for it to make any sense of an actual signed conversation.
TL;DR
While it makes for a neat demonstration and a lot of feel-good articles, the technology does not actually translate sign language to speech in any meaningful way and the practical application for these devices is unfortunately almost nil.
