[zukzuk]

This is a massive problem in healthcare, at least here in Canada. Most of the common EMRs doctors and other practitioners use either don’t have APIs, or if APIs exist they are closely guarded by the EMR vendors. And EMRs are just one of the many software tools clinics have to juggle.

I’d argue that lack of interoperability is one of the biggest problems in the healthcare system here, and getting access to data through the UI intended for humans might just end up being the only feasible solution.

[j45]

I’m not sure how unique or a new problem this is first individually to me and then generally.

Automation technologies to handle things like UI automation have existed long before LLMs and work quite fine.

Having an intentionally imprecise and non deterministic software try to behave in a deterministic manner like all software we’re used to is something else.

[zukzuk]

The people that use these UIs are already imprecise and non deterministic, yet that hasn’t stopped anyone from hiring them.

The potential advantage of using non-deterministic AI for this is that 1) “programming” it to do what needs to be done is a lot easier, and 2) it tends to handle exceptions more gracefully.

You’re right that the approach is nothing new, but it hasn’t taken off, arguably at least in part because it’s been too cumbersome to be practical. I have some hope that LLMs will help change this.

[digitaltrees]

The cost to develop and maintain UI automation is prohibitive for most companies

[asdff]

It begs the question though. If these vendors are so closely guarded of their API to try and shake down people for an enterprise license, why would they suddenly be permissive towards the LLM subverting that payment flow? Chances are the fact the LLM can interact with these systems is a blip: once they do see appreciable adoption the systems will be locked down to prevent the LLM from essentially pirating your service for you.