Reminds me of the iOS app Seene, which is like Instagram for 3D photos made on your phone: http://seene.co/
Ultimately not much of a value prop for me, but was very fun to play with for a few days. I don't think the app could be used to actually export a model (for printing, say), but their website today looks like they have other applications besides their Instagram-like sharing app.
We like Seene but unlike only creating 2.5D pictures we aim for full 3D models one can further use for any purpose (such as 3D printing or creating digital content).
The problem I see with all of these scanners is the lack of quality... Are they all based on the same feature tracking core? Why isn't anyone doing more innovative reconstruction like tracking edge contours, or something else... even with a depth sensor most of the scanned models I've seen are crap.
You're assuming that CCDs are perfect image sensors that don't have noise to cause feature trackers to jitter.
You're also assuming that sparsely featured objects only need simple back/belief propagation to make a good model, finally doing it in less than ten hours on an iphone at anything other than <320voxels^3 is pretty impossible.
Even decent commercial laser scanners only have limited resolution at this scale. Your best bet is either lightfield capture or http://web.media.mit.edu/~achoo/polar3D/camready/manuscript_... (which I've not read fully yet, however looks pretty sexy, even if its not very general.)
Most structure from motion or monocular SLAM algorithms use some kind of feature tracking and then estimate a dense reconstruction after they optimize the locations of all the cameras and all the features.
In general, it's extremely hard to get good quality models from a cellphone camera. Believe me, thousands of innovative researchers are trying their best to make it better.
The critical property of all similar systems is the way how you use it. Modern research systems, both based on depth sensors or cameras can in fact provide very good results if operated by a skilled person in a controlled laboratory environment (no shaky movement, proper light, no reflections, shadows etc.). This is a very different setting to using the system by a wide audience in the variety of all possible situations which might occur every day.
One of our goals is to bridge this gap and make a system which works well for as many situations as possible and can be enjoyed by as many people as possible. Still a very hard task to do. This will happen gradually, one release at a time. Therefore, we will very appreciate your feedback on using the app - both when it works and when it does not. This will help us to improve the app.
Don't quite agree, most recent works produce 3D models of very decent quality. Have a look at this https://www.youtube.com/watch?v=XySrhZpODYs for example, but there are others based on volumetric data-structures that give fine results as well.
They are, the output you're so disdainful of reflects methods even more complex than you're imagining. It's a tough problem.
The best work I've seen recently (from MIT) uses two sensors with different polarization filters to get an additional signal to help with noise rejection and surface estimation.
Right now the scanning works well for a limited number of scenarios (such as the ones shown on the page). We are working on improving the quality and expect it will grow over time to cover more and more cases.
Unlike Kinect this is a passive sensor technology which has it's own advantages (it works outside) and disadvantages (it does not work on completely textureless areas).
Good job to your coworkers with Mementify! There is a number of apps nowadays which work in a similar way (user takes pictures which are then uploaded and processed in the cloud). Our aim is to provide a live-feedback while scanning which is possible only with on-device processing. Without it one simply never knows whether all parts of the model are captured while taking the photos. Moreover, the processing in the cloud usually takes several hours and therefore if the results are not satisfying it might be even impossible to re-take the pictures again.
The advantage of offline methods is though somewhat higher quality. We can add cloud post-processing later to refine final models - especially for the purposes of 3D printing.
It is not that hard. You take multiple photos, add depth information and you have 3d versions of the pictures that you took. I know I over simplified it but in a nutshell this is it
Yes, in nutshell, this is it :-)
But knowing depth information is not enough - you must also very precisely know the position of the camera the individual pictures were taken from. Computing both depth information and position of the camera is in fact a very challenging task.
Unfortunately, we don't have enough time to add Android support at the moment, as we're focusing on improving the quality and disability for the existing app. In the longer term we're definitely planning to release an Android version too.
Thanks! There are some libraries like PCL http://pointclouds.org/ for use with Kinect cameras or things like LSD SLAM and OpenDTAM available for desktop for real time processing and some other structure from motion solutions to reconstruct models from images. They are all really cool, but can be a bit clunky to use for people without computing background, so we're trying to create a new solution that's a bit more mobile and easier to use, hence the iPhone app. Give a try to the sites I mentioned though, maybe you can find something that fits your needs.
Yeah, I'm at the point of running Agisoft Photoscan under wine, no results yet. But of course I won't get the heat-map quality indicator, which is the bit I really like about your app!
Good luck, hope you manage to make it work! Thanks, that's the advantage of having it run real-time. Unfortunately, I don't think there is any other out-of-the-box solution right now that can give you this sort of feedback with a normal camera.
Hi bcks, we're working on it. We'll be releasing a new version that allows you to export to ply. We're also working on adding external
3D printing and delivery straight from the app.
Thanks, Structure From Motion is a broad term covering a number of different methods which were developed over years and our method is also a form of SFM. The main challenge here is to make it work well on a mobile phone in real time instead of hours many SFM algorithms require.
All the processing is done on the device in real time and the app requires enough processing power to work well, so we had to restrict it to newer iPhone and iPad models only.
We don't have any special case processing, everything is handled the same way in the volumetric model, which makes the app really versatile. If you're expecting a scan detailed enough to recognise individual hairs, I'm afraid I'll have to disappoint you so far :) We're currently working on improving the quality and soon we should be able to scan people with good enough quality for 3D printing. Right now though the human scans may lack details.
Ultimately not much of a value prop for me, but was very fun to play with for a few days. I don't think the app could be used to actually export a model (for printing, say), but their website today looks like they have other applications besides their Instagram-like sharing app.