Hacked Kinect Peers Into Your Skull

//

Microsoft’s motion sensing device Kinect promises new frontiers for gamers who want to play just by gesturing or speaking. Now a hacked Kinect could have more serious uses by showing everyone what your brain actually looks like under the skull, in 3-D.

Steer a Spaceship with Your Brain

Using a touch screen tablet, some duct tape, brain scans, and the newest Kinect application programming interface (API) called Fusion, a team from Microsoft Research Cambridge in the UK built an augmented reality system aimed at helping brain surgeons.

Ben Glocker, a post-doc researcher at Microsoft Research Cambridge, recently showed a prototype of the setup to IEEE Spectrum during Microsoft TechFest 2013. In a video, he discusses how the team duct-taped a USB-powered touchscreen device to the Kinect Fusion, which allows them to 3-D model just about anything.

As Glocker demonstrated, the system captures the patient’s skull in 3-D and then incorporates that info with two-dimensional MRI brain scans that doctors took of the patient in preparation for surgery. Those combined images form an augmented reality: When surgeons look at the patient’s head in the operating room, they also see images of the brain underneath.

The team hopes this could help doctors visualize what they need to do. For example, they might see the location of a tumor they need to remove more clearly. To me, and I’m no brain surgeon, a live-action MRI scan in 3-D does seem like it might be more useful than MRI “slices.” However, there are key limitations here.

The most advanced surgical groups already have some impressive imaging systems. As one Gizmodo commenter was quick to point out, neurosurgery tools with electromagnetic tracking that don’t have any line-of-sight restrictions are considered truly next-gen. Modern neuro-navigation systems can even compensate for brain shifting that happens during surgery.

New Google Glass Video Gives Glimpse Of Future

Duct-taping components together also doesn’t inspire a lot of confidence when you’re talking about precise surgical work — even though future tablets will probably have better sensors built in. Given its current limitations, this Kinect Fusion system might still end up being a decent option for teaching. And it’s nice to see Microsoft thinking outside the Xbox.

Photo: A hacked Kinect can show your brain in 3-D. Credit: IEEE Spectrum video.