Medical Design Briefs - October 2022 - 8

Extended Reality
techniques typically only allow a surgeon
to view a single plane or slice of
the data at a time. In addition, many CT
techniques only allow for limited interactivity
(see Figure 1).
As a result, surgeons must view CT
data during preoperative planning sessions
and form their own mental model
of what lies under the patient's skin rather
than having that data easily accessible
during surgery. Certainly, the ability to
create this mental model is a critical skill
and improves with practice. Yet even the
most experienced practitioners feel the
burden of this additional cognitive load
surrounding what is already a complex
set of tasks.
This is where VR, in particular, can
help. Preoperative planning can be significantly
enhanced by taking the data
acquired through traditional CT, MRI,
and ultrasound scans and visualizing it
using VR.
While 3D modeling has been used for
over a decade as a way to better visualize
a patient's anatomy, it is only in recent
years
that computational power and
low-cost hardware have allowed VR to
make inroads as a preoperative tool.
There have been a number of studies
conducted to prove the effectiveness of
using VR to aid in preoperative planning.
For instance, a study on spinal surgery
conducted by De Salvatore compared
60 patients, where half had
surgery using traditional preoperative
planning methods and the other half
using a technique where CT scans were
composited into 3D models and then
viewed using VR hardware. The results
revealed significant reductions in both
the length of the operations and in the
amount of bleeding a patient experitypically
involves blending 3D imagery
acquired preoperatively with reference
points marked on the patient's body.
This process allows for alignment of
computer-generated imagery with the
patient's body when viewed through a
head-mounted display (HMD) worn by
the surgeon.
With this setup, the surgeon does not
have to constantly look away at a remote
monitor to review preoperative data -
as is commonly done today - and instead
can retain focus on the patient.
enced during surgery when VR was used
in the planning process.
Intraoperative surgical assistance.
Moving beyond VR, AR has also emerged
as a promising technology for in-theater
surgical use. In this scenario, use of AR
Additionally, AR techniques can be used
during surgery to provide real-time insight
into the placement of tools relative
to the area of interest.
"
The market for XR
in 2022 is $290
billion, with medical
use cases in particular
driving demand.
"
- Forbes
Coupling these techniques with multimodal
input methods such as voice control
and gestural interactions further allows
a surgeon to reference preoperative
data in an interactive manner without
relying on assistants or touching outside
equipment. For instance, cardiovascular
surgeons demonstrated a system using
transesophageal echocardiography to
gather data and render a 3D volume
onto an HMD during open-heart surgery.
The system could be controlled
(e.g., zoom, positioning, etc.) in a touchless
manner.3
The discipline of image guided surgery
(IGS) takes this idea ever further by incorporating
instruments that are tracked
during the procedure. IGS also augments
the preoperative images with real-time
data using such techniques as interventional
MRI or ultrasound to provide the
surgeon with an accurate anatomical
view. These applications have been used
successfully in neurosurgery for several
years and are now gaining acceptance in
other fields, such as sinus surgery.
Another option for simulating surgical
training utilizes a digital twin, a virtual
representation of an object that spans
its life cycle; is updated from real-time
data; and uses simulation, machine
learning, and reasoning to help with
decision-making. According to Unity
Technologies - which has developed
the largest platform to power games and
virtual experiences and offers tools and
frameworks to build VR and AR applications
- using a digital twin to simulate
training accelerates the learning curve
of inexperienced new surgeons and can
successfully transfer the burden of the
surgical training learning curve from
patients to technology.
" A mixed-reality simulation allows surgical
students or professionals to practice
and prepare with precision for a surgery
they may conduct on a real patient, "
says Unity Senior Strategist Greg Melling.
" A Unity real-time 3D digital twin
allows for patient data and procedure
guides to be augmented into the surgical
environment lowering patent risk while
reducing procedure time. "
Postoperative care and beyond. XR
Fig. 1 - Data-infused surgical procedures use real-time technology. (Credit: 2022 Unity Technologies)
www.medicaldesignbriefs.com
8
tech can also be used to enhance postop
care. For instance, supplementing recovMedical
Design Briefs, October 2022

http://www.medicaldesignbriefs.com

Medical Design Briefs - October 2022

Table of Contents for the Digital Edition of Medical Design Briefs - October 2022