[machinelearning]

Most AR apps today are based on the same toolboxes, either opencv, arkit or arcore. The latter 2 are actually pretty good (in terms of tracking and plane detection) for more simple use cases.

So by gimmicky I assume you’re talking specifically about the use case?

If so it’s probably true that majority of ar apps are games that don’t work too well or ar just demos/prototypes.

However there are a few that are pretty functional e.g. the measuring app in iOS. Was your idea to consolidate all such use cases to create a general purpose app?

[yeldarb]

The magic comes when combining AR with an app that understands what it’s looking at and how to contextually interact with it.

The AR toolkits tell you about the 3D environment but they don’t do anything to tell you about the context.

When your app understands what it’s looking at it can add relevant functionality (eg solve a paper Sudoku puzzle, give hints, check your work, scan into a playable app, etc).

Happy to talk about it more! Shoot me a DM.

[machinelearning]

Generalized scene understanding is still a hard problem in both academia and industry. The big companies are working on such things with access to tons of data.

Assuming you do know what’s in the scene semantically most ar applications that aren’t games do intend to use that information. I’d say the primary hurdle there is still general scene understanding.

It’s easy to identify a sudoku puzzle but not so easy to identify and classify all 3D objects with any sense of precision yet. Seems like a large scale data play to me..

[yeldarb]

Definitely. Not everything I’d like to do is possible at the moment but Machine Learning is getting better so fast that more and more will become feasible bit by bit.

I have a bunch of ideas that seem like low hanging fruit, a bunch that seem hard but possible, and a lot that will be possible someday. And hopefully I can help accelerate that timeline.

[Vomzor]

You’re idea is to build that context layer? It seems the apps you’re talking about will become reality when we have useful AR glasses. But they will be manually built for each usecase.

[yeldarb]

Essentially. I have a list of use cases I think are possible to tackle now so I’m going to work on building out more single-niche apps in the near term.

In doing so I’ll have to build out tools and common code which I hope to eventually release as a platform to help other devs expand the idea out into other verticals.