Virtual reality could soon let your doctor ‘step inside’ your blood sample

By combining the technique, called expansion microscopy, with virtual reality (VR), scientists will be able to enlarge, explore, and analyze cell structures far beyond the capabilities of traditional light microscopy.

The development of these technologies will accelerate researchers’ understanding of infectious and autoimmune diseases and enhance their ability to develop disease diagnostics and prevention and treatment methods.

MAGNIFYING SAMPLES IN VIRTUAL REALITY

Yongxin (Leon) Zhao, an assistant professor of biological sciences at the Mellon College of Science at Carnegie Mellon University, has been developing the expansion microscopy technique to physically magnify a biopsy, allowing researchers to see fine details in biological samples using standard microscopes.

Zhao makes biopsy samples grow in size by chemically transforming them into water-soluble hydrogels. He then applies a treatment that loosens the tissues and allows them to expand more than 100 times in volume. The tissues and molecules within the sample can then be labeled, imaged, and compiled into a complex set of data, to be used to study interactions among cells and their structures.

However, a limitation of the technology is that it extracts two to three orders of magnitudes more data than current techniques are able to interpret. To help solve that problem, the researchers paired expansion microscopy with a virtual reality technique developed at the Benaroya Research Institute at Virginia Mason (BRI).

Through VR technology developed specifically for the purpose, researchers will be able to see and manipulate the originally 2D expansion microscopy images in 3D, giving them a 360 degree view of tissue and protein organizations and interactions.

“At BRI, we’ll prepare the live infectious and autoimmune disease samples,” says Caroline Stefani, senior postdoctoral research associate. “We’ll send those to Carnegie Mellon, where they will enlarge the samples and send images back to BRI to be viewed in VR.”

DATA IMMERSION

“This is the future of how scientists can handle complex data,” Zhao says. “It’s an immersive experience, just like you are sitting inside your data. You have the freedom to explore your data from every angle and every spot.”

Tom Skillman, BRI’s former director of research technology, developed the virtual reality technology and has since founded a VR company, Immersive Science.

“My role in this grant is to develop a software tool that will allow scientists studying disease a way to understand large amounts of data through a computational technique called ‘immersive science,'” Skillman says.

“Bringing all that data into VR not only allows the scientist to see their 2D microscope images in full 3D, but to interact with the data, selecting channels, adjusting the views, colors, and contrast, and grabbing and rotating the images to quickly identify key aspects of the image that are coupled back to the disease under study.”

The eventual goal is for the researchers to share the VR tool, called ExMicroVR, on open platforms with other researchers along with expansion microscopy so that they too can view new details of disease processes and understand larger, more complex sets of data.

The system to convert expansion microscopy data into VR 3D images will be affordable and easily accessible to researchers and physicians in developing countries. It will also allow for up to six people to collaborate and view the same sample remotely at the same time.

Funding for the research came through Grand Challenges, an initiative of the Bill & Melinda Gates Foundation.