Being heard is often a struggle for users of sign language, but a new invention by three students at Christian & Missionary Alliance Sun Kei Secondary School hopes to change that. Sign-translate gloves can translate sign language into text and speech through a smartphone app, bridging the gap between sign and verbal communication.
The team behind the invention are Form Five students Noddy Chung Ho-fai, Jacky Chan Ka-lung and Michael Ng Chak-lam.
After coming top in the Senior Division of the Hong Kong Student Science Project Competition in April, they went on to win the Bronze Award in the Engineering category of I-SWEEEP 2017, an international competition held in the United States which challenges students to tackle global issues.
On their return to Hong Kong, the young innovators explained to Young Post how their invention works.
Wearing one of the prototype gloves, Jacky points at five flex sensors on each of his fingers: as the sensor is flexed, the resistance across the sensor increases and data is collected based on how much his finger bends. At the back of the hand, a gyroscope detects acceleration and angle of the hand. The data collected is then transmitted through a bluetooth chip also attached to back of the glove. Upon receiving the data, a breadboard - used for making an electric circuit - translates the digits into ordinary language, which can be presented on the app as text and even speech.
Clearly, the knowledge applied to this invention exceeds the level of the average secondary school Physics class. Instead, the boys learned from teachers and sifus around them as well as the Internet.
Among the three of them, only Noddy takes Physics as a HKDSE elective; Michael studies humanities subjects while Jacky opted for business and economics. Nevertheless, they’ve found a way to combine each of their strengths in the project. Jacky prepares for meetings with potential investors while Michael oversees the paperwork, designing banners, preparing presentations for competition judges, and making booklets in both Chinese and English detailing every aspect of the project.
“Our team hopes to change the situation of the isolation of deaf-dumb people through these gloves,” reads the booklet.
Having worked out the principle of the gloves, the next challenge for the team was finding the materials. The flex sensors, one of the most important parts , had to be sourced online. Cost was also a factor: just one sensor costs over HK$100. With ten sensors needed for one pair of gloves, the cost adds up to HK$1,500.
The overall budget for the project exceeded HK$5,000. After a HK$2,500 school grant, the boys dug into their own pockets for the rest.
“We just paid for whatever parts we ordered,” says Jacky.
They worked on the project non-stop for three months, staying up every evening after school until 10pm. It hasn’t been smooth sailing.
“We’ve failed at least ten circuit boards,” remembers Noddy. Their app has also undergone more than 50 updates. The team would even pull all-nighters ahead of competition deadlines, and “go home at 6.30am to take a shower before school,” says Jacky.
It all paid off, though, when they completed the first prototype. “It was liking watching the birth of your own child,” Jacky recalls.
Unable to sign themselves, the boys rely on Chinese University of Hong Kong’s Sign Language Browser, which has a directory of thousands of words in one of Hong Kong’s most-used sign languages.
For now, Michael explains, the app’s output to text and speech is only in English, so that it can be used when travelling overseas and generally have a wider outreach. But the team eventually plans to expand their database to include French. Meanwhile, sign language users can also input words themselves, to suit their own vocabulary usage.
Initially, the team manually input all the data, until they decided to write an app that does the work for them. However, it’s still a long process, Noddy explains, because the app needs to be able to recognise slight variations in hand gestures as belonging to the same sign. With more than 5,000 data inputs, there are still only around 20 words in their database at the moment.
“We want to input more words so we can get some sign language users to test it out,” says Michael, with Jacky adding that they hope to improve their invention with user experience feedback.
Jacky also hopes to receive funding from the government or independent groups to lower the cost.
“We’re not looking to commercialise the product. We want to help the people in need.”