A Cross-Platform Mobile AR App to Scan Markers or QR Codes

Customer

The customer is a Norway-based advertising company. The client’s major services are design and decor, web, strategy and consulting, social media, copywriting, drone photography, and VR/360. The customer’s client base comprises businesses from various industries, such as multimedia, telecom, technology, and manufacturing.

Challenge

To promote events and ensure high user engagement, the client required a mobile solution for an art festival of ethnic music, to demonstrate various marker-based AR experiences. The customer needed to create a ready-made system that would enable festival visitors to use their smartphones to scan markers or QR codes and receive information about the objects in the form of visual and web content.

Solution

The resulting solution is a mobile application allowing trigger AR experiences using smartphone cameras as AR scanners. The visitor using this application and hovering over this marker gets an AR experience, such as audiovisual content, to get acquainted with the work of the bands and singers. There are two user roles in the system:

• Administrator
• Visitor

The administrator uses the web admin panel to:

• create a new festival object;
• upload a marker image and attach content for the AR experience; and
• review logs.

Visitors can download and install the app from the App Store and Google Play. Next, welcome and instruction screens are shown with some basic information and a Scan button. When users click the Scan button, an AR screen opens up, showing a live smartphone camera stream. 

When the device is pointed at a marker and the marker is recognized, the system runs the corresponding AR experience: audio, video, image, or web content. The application operates only with markers created for the app. 

The app’s workflow includes the following details:

• The mobile application contains no hidden keys;
• All communications go through SSL connections;
• The server is fully set up with a firewall and isolated entry port;
• Admin panel and API are accessible through authorization and can be under a VPN connection;
• Mobile accessibility is available only if it has an authorization token from AWS Cognito;
• Media and assets are accessible only through the server API proxy flow and isolated on S3.

The system consists of several major components:

• Mobile System (Android/iOS application)

The mobile system (Android/iOS application) does not directly store any data, has no access to the administration system, and may only display content from the delivery system via one-time URLs (security based on IP and Cognito session). The content is read-only.

• Identity System (AWS Cognito)

Cognito Sessions is a secure method of creating sessions/tokens for mobile clients and has no known issues.

• Backend Systems (administration and delivery)

Content is managed (created, updated, and deleted) using Node.js CMS. The SSL is used to access the administrative panel, and access is granted to authorized administrators. Additionally, access to the AWS EC2 system is protected by a whitelist of authorized IPs. The read-only access is managed by Nginx, which securely connects to the AWS S3 bucket(s).

• AWS

Administration system (AWS EC2, AWS RDS, Node.js at AWS EC2);

Delivery system (AWS S3, Nginx at AWS EC2).

Most critical content is stored and managed using AWS. It is the top-level critical system, access to which is limited and restricted to maintain the highest level of security.

Process

To advertise the festival and attract more visitors, billboards and posters of the bands and singers were placed inside the festival location and around the city. To begin the project, the customer provided the development team with photos of the posters to put in the app as markers.

The software was developed on Unity as a cross-platform mobile application for the iOS and Android operating systems. After around two months, the application was finished and fully operational. Our IT experts choose the Kanban methodology to effectively manage the work of the team and monitor the deadlines.

The development team proposed to develop the AR mobile application using the EasyAR SDK. It allows users to recognize markers or QR codes and display overlay information in AR mode. Image files were manually uploaded via the admin panel to AWS S3 cloud storage to be used as markers when recognizing and tracking in the app. All markers were loaded into the application (hardcoded).

The system consists of components that are secured at their points of presence (endpoints, inter-communications), and with respect to the security consideration points, all threats are kept under control.

Technologies, platforms, and tools

Unity, React/Angular, Node.js, MySQL, AWS Cognito, AWS S3, AWS RDS, nginx

Team

1 Project Manager
1 Unity Developer
1 Backend Developer
1 Frontend Developer
1 UI/UX Designer
1 DevOps Engineer
1 QA Engineer

Results

The customer received a fully operational solution at the end of the project. With the assistance of HQSoftware, the client delivered the app to the App Store and Google Play. Currently, we are in close cooperation with the company. As new AR experience markers are added, the content and the application are published in a new version.

For the next step, the client plans to work with HQSoftware to develop a universal AR scanner where users can upload different markers themselves. AR experiences for markers can include pictures, photos, music, and any video content. The idea is that such markers would be automatically snapped up in the application, and then the app would recognize and operate them.

Need more information about our services? Contact us and get a free consultation.