[RationPhantoms]

Would you trust a ML self-driving algorithm trained on a "digital twin" of a city? I would. I view synthetic training data like a digital twin in which it can provider further control or specified noise to understand from.

[scottLobster]

No, because right now I'm working closely with some EEs to troubleshoot electrical issues on some prototype boards (I wrote the firmware). They're prototypes precisely because we know the limits of our models and simulations and need real world boards to test our electronics design and firmware on.

You're suggesting the new, untested models in a new, untested technological field are sufficient for deployment in real world applications even with a lack of real world data to supplement them. That's magical thinking given what we've experienced in every other field of engineering (and finance for that matter).

Why is AI/ML any different? Because highly anthropomorphized words like "learning" and "intelligence" are in the name? These models are some of the most complex machines humanity has ever produced. Replace "learning" and "intelligence" with "calibrated probability calculators". Then detail the sheer complexity of the calibrations needed, and tell me with a straight face that simulations are good enough.

[Nevermark]

Both are likely to be much better.

Simulations may not be good enough alone, but still provide a significant boost.

Simulations can cheaply include scenarios that would be costly or dangerous to actually perform in the real world. And cover many combinations of scenario factors to improve combinatorial coverage.

Another way is to separate models into highly real world dependent (sensory interpretation) and more independent (kinematics based on sensory interpretation) parts. The latter being more suited to training in simulation. Obviously full real world testing is still necessary to validate the results.

[fennecbutt]

Hey, let's shut down humanity because human behaviour can't be perfectly simulated.

[kjkjadksj]

What makes you assume your digital twin is actually capturing the factors that contribute to variation in the real data? This is a big issue in simulation design but for ml researchers its hand-waved off seemingly.

[fragmede]

Probably due to reports like these where the digital twin is credited with gains in factory efficiency.

https://www.forbes.com/sites/carolynschwaar/2024/12/09/schae...

[joshribakoff]

It either improves the results or it does not, i don’t think i see the problem.

[Corrado]

Isn’t this what Tesla does for their driving data? However it would fall apart if they didn’t have real world days to feed into it, right?

[heavyset_go]

> Would you trust a ML self-driving algorithm trained on a "digital twin" of a city? I would.

No, just as I wouldn't trust a surgeon who studied medicine by playing Operation. A gross approximation is not a substitute for real life.

[fragmede]

Hope you don't need surgery then! Suture training kits like these are quite popular for surgeons to train on. https://a.co/d/3cAotZ0 I don't know about you, but I'm not a rubbery rectangular slab of plastic, so obviously this kit can't help them learn.

[heavyset_go]

This is a reason I opted to have a plastic surgeon come in when I went to the ER with an injury.

I could've had the nurse close me up and leave me with a scar, which she admitted would happen with her practice, or I could have someone with extensive experience treating wounds so that they'd heal in cosmetically appealing way do it. I opted for the latter.

[scottLobster]

The difference being that you have to do a little more than that to become a board-certified surgeon. If a VC gives you a billion dollars to buy and practice on every available surgery practice kit in the world, you will still fail to become a surgeon. And we enforce such standards because if we don't then people die needlessly.

[Nevermark]

How a model learns doesn’t really matter. What works works.

How it is tested and validated is what matters.

There are lots of ways to train on synthetic data, and synthetic data can have advantages as well as disadvantages over natural data.

Creative use of synthetic data is going to lead to many cases where we find it is good enough. Or even better than natural data.

[joshribakoff]

What about a doctor who used a mix of training both on live patients as well as cadavers and models?

[heavyset_go]

Is this doctor able to learn new information and work through novel problems on the fly, or will their actions always be based on the studying they did in the past on old information?

Similarly, when this doctor sees something new, will they just write it off as something they've seen before and confidently work from that assumption?