[haswell]

> I see potential for this to take some jobs, particularly those involving menial/repetitive work on a computer

Robotic process automation has been on the scene for a number of years doing exactly this, and is quite a bit more mature.

I agree that this kind of tool has the potential to take more jobs, but companies looking to do this kind of thing have had a number of options available for awhile now. New tech like this will accelerate the trend.

[dbish]

One of the big problems with RPA is that it's very very specific, and requires less natural tool interactions that we can do with the new models (or will soon be able to do). It should be as simple as having an AI system "look over your shoulder" while you tell them what you're doing once or twice, maybe they ask a question some time in the future, but they can automate it from there like teaching a junior person on your team.

I think one of they pieces to do that is actually being able to explain, not just silently watch your screen, and ask questions,make it a dialogue, even once that you might get pinged on later if they hit a snag or a situation changes and they need confirmation of something.

RPA today is really nothing like that.

[haswell]

Yeah, RPA suffers from brittleness largely due to the focus on repeating clicks on specific regions of the screen vs. letting the system figure out what to click.

Some RPA products have improved this using computer vision so they can more reliably click on the right things.

But I agree that the introduction of natural language is new. But I see that as primarily a change in interface, not outcome. i.e. eliminating tasks that involve systematically doing the same things over and over already has options. This new generation of tech just makes it far easier. I’m sure RPA tools will incorporate it.

I’ll also be curious to see how this kind of thing translates to legacy thick clients where access to the DOM can’t be used to “understand” the interface.