I'm building what I hope is an educational and entertainment project for when my son is older. It's a cross between Dynamicland[1] and Osmo[2], that combines a projector, camera, and computer vision to hopefully bring programming and creativity out of the monitor and into a semi-real world. I'm just designing the system now and I posted on reddit[3] to ask the machine learning community for advice. I'm also reaching out to computer vision engineers to offer to pay them for a few hours of their time via Zoom to get advice. Some examples of similar systems are [4] and [5].