vishwajeet
New Member
- Joined
- Jul 7, 2020
- Messages
- 1
- Reaction score
- 0
Hi,
My name is Vishwajeet. Nice to know about this forum and getting opportunity to interact with the deaf community
.
I am currently doing majors in Computer Science from IIT Delhi, India. I along with 3 of my friends are working on enhancing learning of a deaf student in deaf classroom or typical classroom in the absence of any interpreter/ stenographer or real time in person captioner. Specifically we were looking to improve communication between teacher and a deaf student by using cloud services offered by many (we r relying on Google Speech to Text API ) to transcribe the speech from teacher by recording it from either a dedicated mic or using mic of their smartphone and then broadcast it to devices placed at deaf students end.
We have built an app which does this. We also added another functionality to it for enabling the deaf student to ask a question; by letting him type into his device which will be transmitted to teacher's phone, and the phone will spell it out (using dictation tool) on behalf of the student.
We have the application ready but are yet to test in a typical classroom setting. The schools are closed due to pandemic, and so we can't test it anytime soon. Therefore we were looking to have this community's essential feedback !
To summarize our application:
Given: The app at teacher's phone and the app at deaf student's phone; only the teacher's app requires internet to use cloud service.
1. Teachers' app records audio and obtains transcription in real time using Google Speech To Text.
2. The transcription is broadcasted to students' phone/tab that are connected to teacher's phone. (This happens offline, using wifi)
3. To ask a question, the student can write in text their question in their app which will be sent to teacher's phone and spell out by it. (offline)
One of the disadvantage that we see here is lose of visual focus of students towards blackboard and teacher's gestures while focussing on text that they have received on their app. In case of classroom where powerpoint presentation is also used, loss of focus from slides may be a significant drawback.
Lip reading is also disabled, but that may not be a problem if transcribed text is sufficiently accurate.
What do you guys think this solution, should the drawback be a very limiting and how in what ways we can mitigate it ? How likely do you think for deaf students to use this application in view of this drawback?
Some possible approaches that we thought can mitigate
1. Keeping device at a height above desk so that the student doesn't had to head down and can see blackboard/ teacher. But it doesn't look useful as eyes adjust its focus to far and near objects and cant view simultaneously.
2. Installing a cheap projector to display text on a section of blackboard. (We haven't explored the options available here yet)
Thank you !!
Vishwajeet Agrawal
My name is Vishwajeet. Nice to know about this forum and getting opportunity to interact with the deaf community
.I am currently doing majors in Computer Science from IIT Delhi, India. I along with 3 of my friends are working on enhancing learning of a deaf student in deaf classroom or typical classroom in the absence of any interpreter/ stenographer or real time in person captioner. Specifically we were looking to improve communication between teacher and a deaf student by using cloud services offered by many (we r relying on Google Speech to Text API ) to transcribe the speech from teacher by recording it from either a dedicated mic or using mic of their smartphone and then broadcast it to devices placed at deaf students end.
We have built an app which does this. We also added another functionality to it for enabling the deaf student to ask a question; by letting him type into his device which will be transmitted to teacher's phone, and the phone will spell it out (using dictation tool) on behalf of the student.
We have the application ready but are yet to test in a typical classroom setting. The schools are closed due to pandemic, and so we can't test it anytime soon. Therefore we were looking to have this community's essential feedback !
To summarize our application:
Given: The app at teacher's phone and the app at deaf student's phone; only the teacher's app requires internet to use cloud service.
1. Teachers' app records audio and obtains transcription in real time using Google Speech To Text.
2. The transcription is broadcasted to students' phone/tab that are connected to teacher's phone. (This happens offline, using wifi)
3. To ask a question, the student can write in text their question in their app which will be sent to teacher's phone and spell out by it. (offline)
One of the disadvantage that we see here is lose of visual focus of students towards blackboard and teacher's gestures while focussing on text that they have received on their app. In case of classroom where powerpoint presentation is also used, loss of focus from slides may be a significant drawback.
Lip reading is also disabled, but that may not be a problem if transcribed text is sufficiently accurate.
What do you guys think this solution, should the drawback be a very limiting and how in what ways we can mitigate it ? How likely do you think for deaf students to use this application in view of this drawback?
Some possible approaches that we thought can mitigate
1. Keeping device at a height above desk so that the student doesn't had to head down and can see blackboard/ teacher. But it doesn't look useful as eyes adjust its focus to far and near objects and cant view simultaneously.
2. Installing a cheap projector to display text on a section of blackboard. (We haven't explored the options available here yet)
Thank you !!
Vishwajeet Agrawal