Hi Valentin,
Nice work here. I've played around with a very similar thing in my apartment before, but instead of using visible codes, I thought about various ways to make them invisible to the naked eye. One relatively simple way is using IR LED beacons. Cell phone cameras often pick up near-IR (in fact, the crappier the cell phone, the better). They can continually flash codes that can be read directly by the phone over 30fps video.
The downside is that, being 30fps, your bitrate is heavily limited, thus your address space is limited, but this can be alleviated by geo-localization or combining IR beacons with BLE beacons, either by using BLE beacons to localize, or by synchronizing the BLE beacon broadcast with an IR pulse picked up on camera.
I would love to chat with you more. I'm a recent MIT alum and still in the Boston area.
This is very interesting and great work on it but I feel in reality the problem being solved does not benefit people a great deal that they will be willing to spend a lot of money or spend a lot of effort trying to configure things around them. The interface is easy to most but from doing home automation it does require a higher skill set to configure physical buttons or input devices to do what you want on another physical device.
How does your team try to combat the cost and the effort required by making this as economical as possible and also easy to use such that a person who is a beginner at technology such as using smart phones, computers, etc. can use it.
With all due respect, I think your comment is a bit misguided. The applied nature of this work might lead some to believe this is a developed product, but ultimately this is research. It's a unique idea and one that may inspire a product that is economical/easy to use. In a few years that is.
You can checkout openhybrid.org for some of these questions.
We do research in the Media Lab not product development. The Reality Editor is a projection in to the future.
But have a look at the costs of SoC's. It's going down so rapidly that in 2 years your question probably will be answered.
The moment real products will be in the marked that make use of the Reality Editor, you will power them up and just point your phone on them. It should not be more complicated.
The openhybrid.org example sharing a timer between toaster and food processor _using a phone, which has its own builtin timer_ is non-compelling bordering on comical. do you have more realistic examples that make meaningful use of the components of a physical object?
I was wondering: Clearly not everything can have its own WiFi radio, it'd be expensive and pollute the spectrum.
Assuming we'll have rooms, buildings, and blocks absolutely packed with low-bandwidth radios like this; what radio standard is most promising at the moment?
I've seen some work out of Stanford (10.1109/VLSIC.2014.6858380) which looks interesting, but I'm not sure passive radios can harvest enough energy to do auth, which is a deal-breaker.
I think the new specifications for bluetooth with ip6 and Mesh networking are interesting.
But the open hybrid platform builds up on the most common ground. If WiFi pollutes the spectrum, jet there are more and more SoC's with WiFi on board pushing in to the IoT marked, maybe the WiFi consortium will take care of the pollution with a LE standard update.
Apologies I haven't looked at the technical stuff yet (so may be an obvious question), but I was wondering:
Are the "API"s (what an object can be programmed to do) for the devices/objects (eg the switches) accessible locally?
As in is there enough embedded within the QR-like code to understand the API (or at least connect to the device, and it can "tell me more") or will it require cloud-based interaction?
Example case: if I'm in a room I've never been to before, no internet connection, and see a switch - would Instill be able to reprogram it then and there? Or do you see the (I)nternet of IOT as necessary?
Yes it would. The Marker (for visually identifying the object) and all interfaces are stored in the object it self. When you start the Reality Editor it finds automatically all objects in the same network.
The Reality Editor does not need to know a fixed API because the interface is basically a webpage that comes from the object it self. In that case you could say, the API is HTML5.
Hey Valentin. I love this concept. Do you have any notions to get rid of The Marker? I can see a lot of potential applications for this where the Marker would possibly interfere with a product's visual identity.
Minor beef, however. The IoT will never take over if it isn't easy, easy, easy to use, and this has that in spades.
Yes, this is a big thing.
Our goal is it to make the creation of objects so easy that it can become a simple part of the workflow of a product + web designer.
Very very very low entry barrier.
This is something we continuously work on with the open hybrid development tools.
On the other side, the marker is limiting the visual shape of an object. The limitation do provoke new shapes as well. It has some interesting properties.
Long story short. Technology is on its way and the marker will disappear. ;-)
As most people have pointed out, this doesn't seem like anything new. If all of those devices are IoT, you can have an app that controls them. Seems pretty pointless to have to use your camera and drag between objects. You could achieve this much easier with icons in an app. Do you personally think this a good product?
How would you connect two objects in physical space with your app centric interface? This is not a trivial task. Its very complex and has a lot of abstract steps. Steps you wold not be able to explain a 8 year old child in 5 minutes. We gave the Reality Editor in to the hands of a 8 year old child and we did not even need to explain how to do it. He just ran with it.
Try to make him do it while watching a 5 year old and an 8 year old while entering a car. It's a more realistic scenario for when you need a knob to work as expected.
That is a good scenario. What you see in the video and on open hybrid is really as much as we can put in to code at this point and I believe that it is a very good beginning.
There is so much more that we want to implement and some of it would give answers for your situation.
I think the Reality Editor will grow incremental and every version will be a bit better then the version before. In some years, when your car actually has this functionality, your scenario will be solved.
It's useful for the same reasons that a 4yo can use the touch screen on an iPad but can't use the track pad on a computer: It removes one additional step of indirection and thereby reduces cognitive load.
The downside is that, being 30fps, your bitrate is heavily limited, thus your address space is limited, but this can be alleviated by geo-localization or combining IR beacons with BLE beacons, either by using BLE beacons to localize, or by synchronizing the BLE beacon broadcast with an IR pulse picked up on camera.
I would love to chat with you more. I'm a recent MIT alum and still in the Boston area.