AR/VR: Equipment Maintenance and Service Mobile App with Augmented Reality
A cross-platform mobile app that allows for on-site maintenance of complex industrial equipment without the help of experts.
- Industry:industrial sector
A European provider of workforce mobility solutions for manufacturing and industrial companies.
The Customer turned to HQSoftware to develop a cross-platform mobile app that would allow for on-site maintenance of complex industrial equipment without the help of experts. As the tool was to create and play back interactive repair and service guides with augmented reality, the project team faced the challenge of ensuring image recognition with unmatched precision.
For those who use the app for the first time, the interactive, visualized tutorial comes as helpful, showing the basics of creating a guide.
To begin with, a user takes a photo of the equipment they want to create a repair or service guide for. The picture is then processed by Metaio SDK and turned into an image based on which the app will recognize the equipment during playback. That’s where HQSoftware’s decision to employ Metaio’s library pays off, as the tool offers in-built image recognition capabilities, allowing to save on cumbersome programming of custom markers from scratch.
Once the picture is taken, the user, pointing their tablet’s camera at the equipment, goes on to create a series of scenes with comprehensive, step-by-step explanations of all repair operations.
For each scene, the user can apply – that is, lay over the camera view on the screen – a wealth of animated, drag-and-drop 3D objects, such as a screwdriver, a nut or an arrow showing the direction, to better illustrate how to handle a specific action. All objects are scalable, moveable and can be placed at any angle. In addition to that, the user can attach text, images and videos.
When playing back the guide, all the user needs is to simply point their tablet at the equipment and follow the steps. Should the camera deviate so that image recognition becomes impossible, the app will notify the user to move it to the initial position. There is also a possibility to switch between scenes by using a QR code.
When the process is stuck or some elements of the equipment need replacement, the user can leave a maintenance request, and a relevant technical expert will get in touch to help solve the problem.
To ensure smooth synchronization of scenes and attached 3D objects, HQSoftware’s team implemented a proven combination of PouchDB and Apache CouchDB on the frontend and on the backend, respectively. In addition to their light weight, these databases allow storing data offline – for example, when the user is leaving a maintenance request – and sharing it when going online again.
HQSoftware developed a cross-platform mobile app to create and play back interactive repair and service guides enriched with augmented reality. Allowing for on-site maintenance of complex industrial equipment without training, the tool is already in use at multiple industrial companies.