A Gesture-Based Mobile Game With Machine Learning

Customer

Based in the United States, the company provides marketing services to such eminent customers as Ford, and other. The company had an idea of making a gesture-based mobile game and turned to HQSoftware to develop it.

Solution

The goal for the development team here was to implement Machine Learning into the game to enhance the gesture recognition. The gesture must be precise.

The mobile game offers users to launch rockets by shaking his/her phone. The user has to shake the phone up and down 10 times. As he shakes the phone, the power is generated on a meter. When the meter hits the top, the rocket launches. The objective is to launch as many rockets as possible in 1 minute. There is also an Endurance Challenge with unlimited time.

Users can also play against friends via Game Center and compare scores.

Additionally, the game features an Impossible mission mode. The objective here is to go across the Solar System and to reach Pluto. The user has one rocket, which he navigates by shaking his/her phone. If the user is not shaking fast enough, the rocket sputters and tone of the engine goes out. Now the user has to shake harder, to compensate for the lost engine. The level is over when all three engines go out.

From a technological standpoint, the challenge here was to make sure that the device recognizes movement correctly. The game needs the user to shake his/her phone up and down with a short-range. The game requires a firm shake: just a swing won’t do.

To make sure the game detects the gesture precisely, a team of developers applied Supervised Machine Learning, teaching the system to detect the correct movement.

The game uses a device’s gyroscope and accelerometer to identify movement. The developers tried out different types of shakes, tagging the start and endpoints, the shaking speed, and amplitude.

Then the programmers made a selection of movements of certain speed and amplitude, which the system must recognize as the rights ones. Then it was subjected to this selection to develop an algorithm to better tell apart shakes from swings and sweeps.

As the users play the game, the system will continue to polish the gesture-recognition algorithm.