- A cross-platform AR app that scans body movements
- AI-driven analysis of body movements
- Automatic assessment of functional disorder
Technologies Used: AWS, Cordova, PoseNet, Kotlin, Ionic, TensorFlow.js, Material Design, socket.io, Python, Firebase Cloud Messaging, D3.js, i18next, Angular, React.js, TypeScript, jsPDF
Our customer is a software-as-a-service provider headquartered in Germany. Their flagship SaaS product is a mobile application for health risk assessment.
When the customer turned to HQSoftware, its flagship product was available as a mobile application. It enabled users to scan their bodies in motion via a phone’s camera while performing a certain workout. After an AI-driven analysis, the system would identify body weak spots, rate vulnerabilities by severity, and recommend exercises for improvement.
However, the application had significant performance bottlenecks and critical bugs that affected core functionality. For instance, the scan flow would not operate properly, resulting in an inability to correctly recreate human physiology and identify the weak spots.
Meanwhile, the customer partnered with an insurance agency that served a German industry giant. The partnership entailed that the application would be offered as a service to insured employees.
The customer wanted to optimize performance, resolve existing issues in the application, and enable scalability, so as to provide quality service to hundreds of thousands of users.
The project imposed the following challenges:
- Outdated technologies and a controversial architectural design amounted to a significant technical deficit, causing multiple issues;
- As the application stored sensitive information (personal data, health records, insurance details, etc.), it was crucial to ensure data security;
- The codebase needed to be optimized to raise its maintainability.
The solution is a cross-platform mobile app that scans body movements to assess functional disorders, enhanced by an AI-powered analysis of human mobility abilities. The app evaluates movement patterns using a functional movement test. It automatically examines body system problems by estimating various postures and movements while the person undertakes specified physical workouts. Essentially, the system measures your current movement abilities. The fewer deviations from the standard are found, the better the user’s score. So the score represents a rating of general movement skills.
To help users boost their physical performance, the system applies artificial intelligence together with neural athletics and exercise science. AI algorithms jointly developed with the scientists and researchers from Linnaeus University detect slight movement variations that can indicate a physical imbalance. Where there is the greatest movement deviation, that’s where an imbalance has been found. The app compares all app users to mark average points. The analysis is reviewed using scientific facts and findings.
The app’s workflow is as follows:
- place the smartphone camera at a distance of several meters;
- perform a series of physical exercises, such as squats, push-ups, tilts, etc.;
- the system evaluates, analyzes, and shows where and what the body’s weaknesses are;
- the system provides a general assessment of your physical condition.
The app can be used in three possible scenarios:
- Personal fitness assistant
The user can get basic recommendations on how to carry out various exercises the right way to improve their performance and overall physical fitness. The app also contains a preset of .pdf instructions for user’s convenience.
- Healthcare assistant
If the results of your body movement tests are concerning, you can apply for professional guidance. Via the app, the user can have a consultation with a doctor and get professional recommendations. The system provides the two following options:
- to book an appointment with a doctor;
- to contact a doctor directly using a special form.
- Insurance tool
Insurance companies can use the app as a tool to assess risks. Functional movement tests allow them to offer a specific insurance package of services depending on the physical condition of the person.
The system analysis workflow is the following:
- While the user performs physical workouts, the system records a video;
- the video content is divided into small frames;
- the frames are processed through AI algorithms;
- based on the analysis of frames, the user gets a basic physical condition assessment.
The client’s services have been upgraded with the following functionalities:
- real-time squat counts (TensorFlow.js);
- training programs based on the results of a person’s physical condition analysis. Display and playback of exercise videos. Management and transition between videos, order changes, likes and dislikes, etc.;
- overall codebase improvements, refactoring, performance improvements, and application weight reduction;
- authorization through Google, Facebook, and Apple;
- in-app purchases (subscriptions) for the App Store and Google Play;
- dynamic questionnaires;
- PDF reports and documents;
- refinement of outdated features.
After a thorough examination of the app’s architecture, HQSoftware’s team identified points of vulnerability and worked out a roadmap for improvements. The architecture redesign laid the foundation for improved scalability and high availability. Then, our engineers came up with a strategy to gradually migrate to a modern stack and then update to the latest versions of the technologies without downtime. This would allow for an overall performance boost of 3–4 times.
In the outdated version, the customer used PoseNet to recognize human movement, which wasn’t supported by the latest Java and Kotlin versions. To improve the scan flow, our developers delivered a number of algorithms responsible for smoothing captured movements, thus achieving better precision in recreating human physiology.
By enabling encryption of sensitive information, HQSoftware’s engineers enforced data security and prevented unauthorized access.
By implementing best practices of code reuse and generic component development, our team ensured high maintainability for the app and facilitated extended functionality in the future.
Technologies, platforms, and tools
AWS, Kotlin, Python, Ionic/Cordova, Angular, TypeScript, RxJS, Firebase Cloud Messaging, TensorFlow.js, PoseNet, Material UI, D3.js, jsPDF, i18next, socket.io
1 Backend Developer
1 Frontend Developer
HQSoftware helped the customer optimize the overall performance of the app by 3–4 times and resolved issues involving business-critical functionality. By addressing the technical deficit and following best principles of code reuse, the company laid the foundation for improved scalability and high availability, as well as better maintainability. Altogether, this allowed for the delivery of a service that can be easily scaled and customized for hundreds of thousands of users.
Need more information about our services? Contact us and get a free consultation.
Check Out Other Works
See How We Reach Goals
Let’s talk about your business needs and figure out the best solution. Just fill in this form and get a free individual proposal from your personal manager.