[HenryBemis]

Probably some job scheduling issue; maybe a job got stuck (dependencies or what have you) and the techie just thought to re-run the job(s) without checking the progress of the "stuck one". I could as well have ran for 99.999% and got stuck on the last account (ZZTop's:) and then it run again "successfully" and the techie went home happy for saving the day. And after all the complains start coming in IT thought just what I thought and went through the support tickets and saw the "Job-Mother-Of-All-Payments" was ran twice.

Now they should be having a chat with their BAs and their Finance on how to reverse the duplicates.

[bridanp]

Unfortunately I was part of one of these at a much smaller bank in the past. A job that "memo posted" checking accounts failed. When we restarted the job, we tagged the restart to the wrong step. We should have hit the restore step first, prior to the posting failure. Instead, we tagged the one right after, duplicating the posting for all accounts that had successfully passed on the first run. It was thousands of transactions that had to be backed out before they were hard posted. It was an awful feeling when we realized it, made more awful to know we caused people so much trouble that day. Anyway, I'm telling the story because it doesn't have to be because Wells Fargo is inherently bad. Their management has made some significant errors lately that could be considered criminal. But in this case, I'd side with it being program, job scheduling, or technical admin errors. I'm almost 100% positive the people responsible are just sick about it right now.

[bpchaps]

Hah! Yeah, probably. No ops person I've met has NOT made this mistake. The more you touch, the more likely you are to break something. It's bound to happen eventually. So, when it does, you buckle down and hope that ALL_CAPS_REG_JOB_X-22 wasn't that important.

A NOC I worked at used to send company-wide pages (in 2008..) whenever something important broke. "Inadvertently" was used maybe once a month!