Augmented Reality Guided Biopsy

Prototype Developed at MIT Reality Hack 2020
for Microsoft HoloLens 2

January 2020

Framework

MIT Reality Hack is a 5-day hackathon held at MIT Media Lab in Boston. It comprises of thought leaders, mentors and creators from all over the world to come together and attend workshops, discussions and collaborative hacking.

Our Project

We found each other as a team of clinicians, developers and designers inspired by the application of cutting-edge visual technology for healthcare use cases. We envisioned an augmented reality guidance system for neurosurgery that improves patient safety as well as the physicians’s workflow.


Technical

Our team had the chance to work with the Microsoft HoloLens 2 augmented reality display, using the Unity3d engine to build the experience. The surgery overlay was derived from patient’s MRI data. Our application required precise mapping of digital objects and physical instruments for which we made use of HoloLens’ hand tracking functionality.