I'm sorry, but I don't believe in the idea at all. How much do you know about language processing? Have you ever seen a product that reliably can transcribe speech correctly? As hard of hearing I can assure you that no such thing exists. What makes you believe that you could do both correct transcription of movements as well as translating into a completely different language?
Ok, so let's say you capture a pointing movement in a video. How would you know how to translate that? Depending on direction of the pointing it could mean he, she, it, that, this, here, or anything you are referring to, such as a person (Peter, Mary, Claire, or John) or a thing or a place. In Swedish sign language pointing upwards could mean God and pointing downwards could mean basement, but in a slightly different context maybe something else. When signing with people it becomes apparent how the pointing should be interpreted, but I just don't believe you will be able to translate a pointing correctly even if you stay on the project for years. What if the signer is describing what a surface is looking like. How would you automatically describe the signed description?
