Idea

BodyScope is an internal product of HQSoftware. Given our company’s focus on AR/VR solutions, we recognized a significant opportunity to enhance medical learning and diagnosis through advanced visualization. So, we decided to create an app that enables users to view complicated MRI and CT scans in Virtual Reality.

Solution

With our Virtual Reality app, users can upload any series of CT, MRI, or PET/CT images and reconstruct them in 3D. To view and interact with 3D models, users need Oculus Quest 2 VR headsets.

The system includes three components:

• a desktop app for converting DICOM files into a format suitable for VR;
• a VR app built on Unreal Engine 5;
• an admin panel for creating user accounts for multi-user sessions.

To start the experience, a user needs to upload MRI or CT images to the converter, which  prepares them for viewing in Virtual Reality. In this step, we’ve implemented the ability to download files anonymously, ensuring data privacy — a crucial consideration in the medical industry. This allows medical professors to present real case studies to students without compromising personal information.

Within the VR app, users can not only view and rotate 3D models but also:

• scale models to explore specific body parts in detail;
• apply a set of filters, separate for CT and MRI images, to visualize specific tissues by density;
• slice the model at any angle to remove interfering structures;
• measure objects using polygonal or regular measuring tools;
• enhance visibility with a lamp.

Challenges

During project development, our team encountered several challenges.

Medical complexities

As radiology is a complex medical area that requires a deep understanding, our team had to navigate numerous intricacies. For example, organizing raw data from CT and MRI machines, assembling this data into 3D models, determining the final image quality, and more.

Recognizing the need for expert guidance, we invited an experienced radiologist to provide insights into radiology principles, guide us through the diagnostic process, and review our results. By combining the expert’s industry knowledge with our technical expertise, we were able to create a solution that aligns with real-world medical practice.

Visualization

Every study consists of numerous 2D scans from which we construct 3D models in Virtual Reality. The more scans the study has, the more complicated the construction is. So, the challenge was to ensure that the 3D models match the original flat images and are informative and detailed.

It was also important to maintain optimal lighting in the scenario room. We need to create volume and strike a balance between shadows and contrast to avoid issues like backlighting or excessively dark areas.

Data formats

To visualize studies in Virtual Reality, DICOM files should be converted to a suitable format. Since there are many DICOM variations that complicated building a custom converter, we first experimented with a ready-made one. However, it failed to meet all our requirements and made us dependent on a vendor.

Have an AR/VR project in mind? We’re ready to help!

HQSoftware has a team of skilled professionals ready to tackle the project. Let’s talk!

Anna Halias
Business Development Manager

Get a Free Quote

So, we conducted additional research to develop our own converter that takes into account different file formats.

Measurements

The 3D models in Virtual Reality are smaller than those in reality.  So, we configured the scale to ensure that users could obtain informative data reflecting the real-life dimensions when using a measuring tool.

Stomatological scans

Collaboration with a manufacturer of stomatological equipment introduced the need to visualize teeth scans, which differed from the formats we had previously worked with. With additional research and the assistance of stomatological specialists, users can now create teeth models in Virtual Reality.

5 Reasons to Implement AR in Media and Entertainment

How do modern businesses use AR to increase their revenues and outperform their industry competitors?

Download the whitepaper

Overcoming these challenges successfully, our team developed a solution with high-quality visualization and rich capabilities.

Who Can Benefit from BodyScope

BodyScope can serve as an additional visualization tool to enhance:

• education of medical students, enhancing their understanding of the human body and helping them correlate planar CT and MRI scans with 3D body volume;
• training young physicians in residency, allowing them to gain experience through exploration of real-life case studies;
• diagnostics, allowing medical specialists to gain a comprehensive view of a patient’s condition and facilitating shared sessions for more accurate diagnoses; and
• patient education, helping patients better understand their health conditions and treatment options.

Team

• 1 Project Manager
• 1 Unreal Engine developer
• 1 QA
• 1 back-end developer
• 1 front-end developer

Results

Working closely with medical experts in radiology, our team created a comprehensive VR platform for recreating CT and MRI scans in volume and interacting with them. The solution has already been tested in several hospitals as a supplementary tool for educating young doctors, receiving positive feedback. A transplant surgeon noted that in VR, details are more clearly visible, making pathology identification and diagnosis easier.

We are proceeding with the project. Our future plans include enhancing collaborative sessions and delving deeper into multiplanar projection imaging.

Learn more about our AR/VR services.