Researchers in Canada are developing Typealike, a new technology that uses hand gestures to perform commands on computers.
The prototype works via a laptop webcam with a mirror attached. According to the developers, the program recognizes the user’s hands next to or near the keyboard and prompts for actions based on different hand positions.
In use, a person could place their right hand with the thumb up next to the keyboard, and the program would recognize this as a signal to increase the volume. Different gestures and different combinations of gestures can be programmed to perform a wide variety of operations.
“It started with a simple idea about new ways to use a webcam,” said Nalin Chhibber, a recent graduate student at the University of Waterloo’s Cheriton School of Computer Science. “The webcam is aimed at your face, but most of the interaction on a computer takes place around your hands. So we thought, what can we do if the webcam can pick up hand gestures?”
MORE FROM ELECTRONICS & COMMUNICATIONS
According to the university, this led to the development of a small mechanical attachment that points the webcam downwards towards the hands. The team then created a program that can understand hand gestures in varying circumstances and for different users. The team used machine learning techniques to train the Typealike program.
“It’s a neural network, so you have to show the algorithm examples of what you’re trying to detect,” said Fabrice Matulic, senior researcher at Preferred Networks Inc. and former postdoctoral researcher at Waterloo. “Some people will gesture a little differently and hands vary in size, so you have to collect a lot of data from different people with different lighting conditions.”
The team maintained a hand gesture database with research volunteers, who also conducted tests and surveys to help the team understand how to make the program as functional and versatile as possible.
“We’re always trying to make things that people can use easily,” said Daniel Vogel, an associate professor of computer science at Waterloo. “People look at something like Typealike, or some other new technology in human-computer interaction, and they say it just makes sense. That’s what we want. We want to make technology that is intuitive and straightforward, but sometimes that requires a lot of complex research and advanced software.”
The researchers said there are further applications for the Typealike program in virtual reality, where it could eliminate the need for portable controllers.
The study, Similar: Hands almost like the keyboard for extended interaction with the laptop, written by Chhibber, Matulic, Vogel and team member Hemant Bhaskar Surale, was recently published in the journal for the work of ACM Human-Computer Interaction.
