[xorcist]

Please explain to me what the use of RPA is.

Here's my background in the subject: I have met with team of a dozen people that spent half a year doing something that turned out to be the functional equivalent on an SQL one-liner.

They could not within our meeting slot accurately describe what it is they were doing, apart from "automating tedious work" (yes, but what is that work, specifically?) and kept insisting I was simplyfing their work (which was very likely true, but not from a lack of trying). Not a single line of this code was in git, there were no tests written, and there was no migration path concerning changes in the external data sets.

I have seen a lot of visual programming and business process modelling tools, and written my fair share of LabVIEW back in the days, but with these tools everything old seemed to be new again and evey lesson had to be re-learned. I later learned that certain executives had attended a big Gartner event with RPA in the magic quadrant and promptly set up this team upon returning. Of course, today "AI" has replaced RPA, but that team is still ticking away.

Is there something to these tools, or is it just the latest iteration of preying on C-level executives with budgets to fill?

[honey-badger]

Author here. In an ideal firm, where data is handled responsibly, you would not need RPA at all. In large organizations though, data tends to accumulate in monolithic islands that are only exposed to each other with legacy UI based applications. Moving this data tends to make up a large bulk of white-collar jobs in large organizations.

RPA replicates the work that these workers do by automating on the UI of the legacy application and replacing them. It is championed by ambitious business professionals who

1) Don't know that a more elegant solution exists in the traditional IT world

2) Don't receive enough support from the IT org to solve this business problem that is crucial to them.

Therefore, such business professionals collaborate with consultants to spear-head RPA projects in the name of digitization, ushering in AI or similarly varnished back-doors.

However, as long as important data tends to accumulate in disparate islands that IT doesn't have the time to pay heed to, RPA projects will continue to be sold at a price that is a little lower than the salaries of the employees they 'liberate'.

There is, although, another wrinkle to this. RPA is also about democritizing IT by handing (ideally) every white-collar worker the ability to write scripts that can automate pieces of their daily-work. Of course, those solutions aren't likely to be elegant, and their code won't be clean, but since their scope is tiny, that isn't a problem.

[_dain_]

>RPA replicates the work that these workers do by automating on the UI of the legacy application and replacing them. It is championed by ambitious business professionals who

>1) Don't know that a more elegant solution exists in the traditional IT world

>2) Don't receive enough support from the IT org to solve this business problem that is crucial to them.

>Therefore, such business professionals collaborate with consultants to spear-head RPA projects in the name of digitization, ushering in AI or similarly varnished back-doors.

Sounds like Conway's Law taken to an absurd degree and turned into a multibillion dollar industry.

[honey-badger]

That is interesting! It rings true, but I never looked at it that way.

[cosmodisk]

RPAs exist for two reasons:

) Lack of political will/capital to uproot silly/outdated processes.

2) Lack of skills amongst business users to understand development

Many processes in companies are idiotic, outdated and sometimes even unnecessary. Usually these processes are invisible to the decision makers. So whilst the entire HR department keeps sending emails with Excel files to each other all day long, nobody higher up stops and says: well, you know it's bulshit, let's drop Excel and get a proper tool where we could keep records together.

Sometimes the process can't be changed: in my previous job, I was sending Excel files dividing the invoice into 12 different business units in 12 different countries. Guess what, the client was one of top 5 US banks,so we,as a mid size company had zero chances to change it.

Most business users don't understand programming, nor its capabilities or opportunities. For them anything written with code is a black box and ultimately make them feel inferior. So when you go to a business user and say: hey look, YOU can be in charge,YOU can automate. Suddenly they are elevated into the whole new level.

[honey-badger]

What you say here is partly correct. To your list of reasons, I would add

3) The existence of business problems that are valuable in their own right, but not compelling enough for regular developers in the firm to care about

[burnte]

I was just involved in a $50k RPA project. We have an EMR that the buyer of our facilities wanted data from. The EMR vendor gave us two choices, a SQL dump of everything (which we took but would take many months to decode what tables did what, basically reverse engineer the storage process) or let them export the subset of data we needed. The first was free, the second was $60k and would take 6 months. Knowing that vendor, it would take closer to 9-12 due to how backed up they are and how bad they are at time estimation.

Enter a third party vendor I'd worked with before. They specialize in archiving data from EMRs into a standalone searchable database. $50k and two months, they're done. The EMR has an IE only UI, and a newer cross platform UI. We demonstrated how to extract the data required for the new owner, and they created an RPA workflow to "print" to PDF 5k medical records. It's the exact same output the EMR vendor would have given us, but they'd have used their own internal tools to do it, but they'd have taken 3-6 times longer to do it.

RPA is just screen scraping, basically, and sometimes it's still useful.

[honey-badger]

Precisely. In some narrowly defined scenarios where data migration is otherwise like untangling a giant ball of yarn, RPA serves an invaluable need. However, the manner in which it is marketed is out of proportion to the narrower scope in which it is valuable.

[burnte]

I could say that about every technology. :)

[NikolaNovak]

Sometimes it's the unobvious, non-technical things.

FWIW: In my world, we have native tools and methods as part of our ERP application, that would accomplish certain automation tasks more efficiently and effectively than RPA. But that would constitute a "change of application" (because it would be); change management and sign-off on these is so onerous; and the perceived risk and potential impact so high; that they never actually materialize.

RPA sales pitch to executives is "This is not changing your application; this is simply doing what your people are already doing in your application, faster and more repeatable with less errors".

This is a powerful risk-reduction pitch and not an untrue one in the real world with processes and risk aversion.

In the end, the RPA effort... had its hiccups and moments, but it DID automate things in a project/environment/client/process that otherwise would wait until heat death of universe to be automated. So it worked great, with some asterisks and for given definition of worked and great :).

----

P.S. That being said, as far as article goes, I feel it's broadly correct:

1. RPA fails in production a lot - which is counter-productive to its primary goal of reducing risk / increasing consistency.

In our application, there's ultimately a reason why something was a human UI and not a batch job - there's edge cases and circumstances and sometimes decision and value judgment, unusual flows and special conditions, however common or rare. And because you're SO externally tied to a GUI, anything breaks it. Anything. You get into this weird position where updating application to fix something has inherent risk of breaking something else (RPA bot)

2. RPA developers I met were intelligent, energetic, excited, positive, hard working, willing. But yes, also not necessarily "software engineer minded". Experience I think is a significant factor - discipline is new, developers are young, and things you gain with experience such as "how do I eek out requirements and edge cases out of user" weren't there just yet; they will be, but I'll agree with a bit of "relearning things we already knew".

That being said, note that there definitely WAS a text-based script editor in tool that we used as well; not sure how primary it was.

[dragonwriter]

> RPA sales pitch to executives is "This is not changing your application; this is simply doing what your people are already doing in your application, faster and more repeatable with less errors".

> This is a powerful risk-reduction pitch

Is it? It seems to me a powerful risk creation pitch, where instead of dealing with the process problems that make properly controlled change expensive, you just exploit a loophole in your control requirement that are designed to mitigate risk and implement change with no control.

OTOH, since the pitch is generally to the people responsible for the state of the policies, I guess it is “risk reduction” in that it reduces the short-term risk of admitting the larger error since it enables kicking it down the road a bit, and that is something lots of executives like.

[NikolaNovak]

When you buy commercial off the shelf software such as any ERP, it is not "wrong internal policies / incorrect process" or "loopholes exploited" (notion I've seen in couple of comments here) but a design reality of the mode you're operating in.

The more you customize a COTS application, the harder your support gets and the more pain you get when you patch or upgrade.

Building on top of it is practically as well as procedurally safer. It is, legitimately, reduced risk and impact. You keep the solid stable core that is fully vendor supported, and you have the piece that you built and own neatly separated.

In that way you can almost think of rpa as a shell script: You don't customize the kernel every time you need something done. You leave kernel well alone (and reduce both risk, and pain of future os upgrades) and build on an easier language something you own on top of it that interacts with it.

Note I don't do rpa and my team doesn't do rpa, but I've seen it around and seen it's value. It does feel I'm talking To folks who want to reduce this to "management sucks" as opposed to any effort to try to actually understand a technology's Place in ecosystem though...

[xorcist]

So the pitch is that change management hasn't caught up with them .. yet.

The very second one of these runs has some non-intended consequences, and from the nature of the tools that's likely to happen sooner rather than later, change management will latch on to them and never let go.

The tools in question were very similar to Selenium. Any non trivial job is going to have logic in it. Definitively not any less programming involved than writing tests. They were clearly intended for a Microsoft focused audience however, so maybe that's part of it.

[ethbr0]

No, the pitch is:

> automate things in a project/environment/client/process that otherwise would wait until heat death of universe to be automated

Which is to say "things no one else cares about or thinks are important enohgh to spend time automating."

RPA isn't for major corporate priorities.

RPA is for that thing that takes up 25% of a 3-person team's week. Or 15% of every contact center agent's time. Neither of which are ever going to be prioritized.

Or, to put it another way, RPA is the answer to "That has value, but it isn't important enough to spend software developers' time on."

Which is why "but these people aren't software developers" or "but they didn't use git" or etc simplify to "If we had more software developers, this wouldn't be needed."

Yes.

But that's not true. Nor will likely ever be true. So it is needed.

PS: And fundamentally, it's taking corporate computing back from the "ask IT for anything" to "do it yourself" hacker ethos. Computers exist to do work for you. When did we forget that?

[honey-badger]

Amen! RPA is also about democritizing IT by handing (ideally) every white-collar worker the ability to write scripts that can automate pieces of their daily-work. Of course, those solutions aren't likely to be elegant and their code won't be clean, but since their scope is tiny, that isn't a problem.

[_dain_]

>handing (ideally) every white-collar worker the ability to write scripts that can automate pieces of their daily-work.

surely much of this could be done already with a tool like AutoHotKey? I mean it won't have some of the fancier stuff like screen scraping, but it gets you 80% of the way there in a lot of cases. the barrier is not the tools, it's the IT policies that prevent ordinary workers from being allowed to use them.

[honey-badger]

It's also a marketing problem. You need a solution that appears sexy to the worker, the boss and a champion high enough in the organization to overrule outdated IT policy.

I think RPA exists because every organization is plagued by people management problems that developers and IT often fail to acknowledge.

[Foobar8568]

RPA enables the digital transformation of your company! /s

[ethbr0]

"Digital transformation" is a nice way of spelling "Unf@$cking Your Terrible Use of Computers"

[NikolaNovak]

ethbr0 provided valuable perspective, but just to respond:

1. I've updated my post to make it more clear how change management is involved: Using native tools we would be, literally, "changing the application". RPA is not changing application; it is automating what people already do in application. It can get philosophical whether this is a real difference or semantic one; but in corporate world, it's real :). And ultimately in techie world too: ERP is not touched or customized by the RPA (which would have had meaningful and permanent repercussions for troubleshooting, vendor support, upgrades, retrofitting,etc). On Ops side too, we use same process and tools to troubleshoot an error in our application whether a human or RPA bot did it.

RPA bots still go through change management as such; but their risk, cost, implementation time and other profiles are simply vastly different than actually touching the COTS ERP application code. I would not necessarily use RPA to change most of the applications internally made by a company; but they're a valid choice for automating a externally sourced software.

2. I have limited experience with Selenium but I would agree that there are broad similarities. They even remind me of old Mercury/HP LoadRunner or SQA Robot. Just, couple of decades of advancement and different focus.

3. I feel "Clearly intended for a Microsoft focused audience" is some kind of insult, and somewhat uncalled for; but note our application runs on AIX on p-Series, FWIW.

[honey-badger]

All your observations are spot-on. Kudos! You have understood this space much better than most software developers ever will :)

And yes - the exhortation in my article was essentially for RPA developers to max out on the use of the text-based script editor.

[neves]

The use of RPA is to allow companies to contract cheaper people to do development job.

Every 10 years or so you will find the industry trying to sell GUI programming tools. Managers will love it. It will sell a lot. Then people will start to see the brittle applications, that it is impossible for 2 people to develop at the same time, that you can't see what changed from 3 versions ago etc.

Then you will have to point your manager to the the 1986 article "No Silver Bullet": https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&c...

[UK-AL]

Weirdly though. RPA consultants in professional service firms cost more than devs.

[honey-badger]

That is a symptom of hype. In the long run, this will change due to commoditization, as I point out in the article.

[Foobar8568]

Because they add business value....................

[8organicbits]

I've seen this pop up where team A owns a web app (likely legacy) and team B has heavy and increasing use of the app. Team B realizes that they spend so much time shuffling data in and out of the app and realizes, team A should build an API so we can automate process and reporting.

So a tentative plan forms. Team B meets team A and asks for the API. Team B seeks funding to code on top of the planned API. But it falls through, team A doesn't have time/budget for the project, isn't interested in the idea, or otherwise rejects the idea. This is especially true if team A has other larger, more important users.

Team A regroups and tries RPA to make their own programmatic interface on team B's app without their help. The RPA promise that non-programmers can automate their own jobs is also compelling, and is seen as a cost savings.

What I've seen happen next is that nothing really happens. End users are capable of writing RPA, but don't want to. Any automation is flakey, and may not actually reduce costs. Most folks continue getting paid to shuffle data, who'd want to automate away their own paycheck?

Also, since team B didn't want to build an API and isn't part of the effort, there's a bunch of ugly issues: no programmatic contract/interface, no docs, no versioning, no backwards compatibility, no forward notice of changes, no access to test environments. So the automation can unexpected break or begin corrupting data, it needs regular monitoring.

As a developer, that sounds terrible. But if there's no other way to automate, it's attractive.

[nus07]

I worked for a large healthcare system in the past where a lot of the data that I was providing bio-statisticians with required combining with some files from NIH,CMS and other websites. Now ideally there should have been ways to use APIs to get those files' data. However its so bureaucratic that to get API access there were hundreds of permissions and documents required which would take several months. So someone on my team had to login everyday,download those files, unzip it, clean it and then run it to a database. It was annoying so I wrote a python script to do all those tasks and scheduled it. It was great until a few months later management and audit happened and it became a huge issue that an ad-hoc script with my credentials was doing this.Management wanted something that was standardized and 'maintainable'. So they replaced my script with thousands of dollars and hours of RPA based work from a big 4 consulting firm. This is why RPA exists.

[peterangular]

> It was great until a few months later management and audit happened and it became a huge issue that an ad-hoc script with my credentials was doing this. Management wanted something that was standardized and 'maintainable'

In corporate environments it's super common to experience functional success in parallel to social failure, especially like your anecdote. Most corps/management love terms like "intrapreneur" but have zero actual interest in formalizing + celebrating internal innovations/solutions.

[honey-badger]

Great perspective. Now if you were to take the place of Management and tell this exact story, how would it sound?

[dfxm12]

It's supposedly a "citizen developer" tool that records your actions, as a macro, and runs it later, so all those executives got it in their head that if they buy RPA tools, they won't have to spend extra on developers for automation tasks, but then, people decide they don't have the time, wherewithal or just think it is beneath them to learn this stuff and end up hiring "pro" developers to do it anyway, so it usually ends up becoming just as expensive to build and maintain, and usually nowhere near as robust.

If you really do have citizen devs in your org, it's great. I think people are more willing and able to learn Power Automate for Desktop than PowerShell/Python/what have you, then add in the governance, inventory and monitoring/analytics options in an RPA tool like Power Automate, and it can be easier to manage.

[Bouncingsoul1]

Hmm they only context I know it is in context of test automation as "Hardware in the loop". Apply nightly firmware to ble-perpheral via ssh- Install nightly app build on mobile via adb- Test ble connection - Test ble bonding- Test interaction between the devices- control some relais to trigger certain events- curl to see if events land in backend....

would be tedious to do it manually. That said everyting is in git...

[sharps1]

Imagine one person using a web app to click through a list of up to 50 items with multiple clicks to have a letter printed to a person (Which take an hour or so).

That process is done in 20 separate locations by 20 separate people. With RPA we can automate that entire process (We don’t control the web app and have to wait for features from the vendor).

(The process is more complicated than this, but you get the idea).

[DebtDeflation]

>Please explain to me what the use of RPA is.

The short answer is RPA = screenscraping + keystroke recording. Sometimes you need to programmatically interact with old legacy applications that have no APIs or other interfaces apart from the UI of the application, and this is where RPA comes into play.