Try and figure out how this works before you read on.
It’s not hard to understand how it’s done, but more importantly why is it interesting?
Augmented reality is weird because it’s familiar but different at the same time. If you were seeing the keyboard and computer display this wouldn’t be interesting. If the keyboard were a virtual keyboard, projected onto the table then this would lose its magic as well.
So what exactly do we have?
A physical keyboard and virtual letters (under gravity and collisions), projected onto a physical surface (in close proximity to the keyboard).
It’s just the right amount of real and not real.
This project was inspired by the identical demo by Moment Factory. I reverse engineered their process and programmed this using Apple’s Swift and Sprite Kit. The code is here on github.