[PyWoody]

Heh. This reminds me of the time when our newly hired "Salesforce Expert" improved our support queue:

  Every time Support received a new email, a ticket in Salesforce would be created and assigned to Support
  
  Every time Support was assigned a new ticket, Salesforce would send a notification email

The worst part is he wouldn't admit to the mistake and it took us forever to find where he buried the rule.

[bedatadriven]

I can remember something like this a few years ago when a customer emailed our helpdesk with their own internal IT support desk in copy. Our helpdesk at the time sent a complete new email acknowledging the request, which the customer's desk ALSO acknowledged in a new thread...

I think it took us a good hour and a few hundred tickets to get the helpdesks to stop fighting with each other!

[pixl97]

Ah, mailing loops are great.

I remember working for an ISP in the mid 90s. We never really had problems with 1 to 1 mailing loops bouncing back and forth, but we ended up with a large circular mailing loop involving a mailing list, and bad addresses on it getting bounced to the previous server which sent a reply to the mailing list, which got bounced and sent to everyone in the group which caused someone else's mailbox to fill up that was in a forward, which for some reason sent a bounce to the mailing list that really started to set off the explosive growth.

Needless to say the bounces seemed to be growing quadratically and overwhelmed our medium sized ISP, a decent sized college, and a large ISPs mailing system in less time than anyone could figure out how to get it to stop.

[pousada]

I only used salesforce once (was “forced” to use it haha) and it was mind boggling how anyone would ever want to use it or even become an expert in using it.

I’d rather track everything in a giant excel tyvm

[embedding-shape]

> it was mind boggling how anyone would ever want to use it or even become an expert in using it.

As in a lot of cases, the answer is money. If you have expertise in Salesforce, you can get paid a lot, especially if the company you contract/freelance for is in an "emergency" which, because they use Salesforce, they'll eventually be. As long as you get the foot in the door, you'll have a steady stream of easy money. It fucking sucks though, the entire ecosystem, not for the weak of heart.

[GuinansEyebrows]

nobody who actually uses salesforce for daily work chose it. it's sold directly to CIO/CTOs as a one-stop shop for CRM, ticketing, reports and biz dev, who may occasionally use it for reporting (but more often get their staff to provide the reports directly to them). everybody stuck having to use it to actually track work just has to suffer with it.

[wrs]

Or in my case, it was sold directly to the CMO, and as the CTO I was stuck with it!

[GuinansEyebrows]

you won't get off that easily in the eyes of your subordinates :) but to be fair, i should have said CxOs. CEOs fall for this dogshit too.

[consp]

Isn't this the SAP businesscase as well?

[DANmode]

You become an expert in using SalesForce, or SAP, for the same reason you get a medical license in the US.

There’s a limited number of you who are willing to traverse that gauntlet of abuse, so you know you’ll always have work.

[mrgoldenbrown]

For doctors in the US I think the limit is more artificial than that: a cap on how many med school seats are allowed.

[DANmode]

The captive audience after some (mostly) arbitrary grinding,

(not being drawn to serve the niche by any particular talent or interest besides $),

is the comparison being drawn.

[bArray]

Maybe 20 years ago... As a student, the school had an email server that allowed rules to be set. You could set an email to be sent as a result of another email.

IT were not stupid though, and set a series of rules:

1. You cannot have a rule trigger to email yourself.

2. You cannot reply to an email triggered by a rule.

3. You have ~50MB max of emails (which was a lot at the time).

Playing around one lunch, my friend had setup a "not in office" automated reply, I setup a rule to reply to any emails within our domain with a "not in office", but put their name in TO, CC and BCC. It turns out that this caused rule #2 not to trigger. After setting up the same rule on my friend's email, and sending a single email, the emails fired approximately one every 30 seconds.

A few hours later we returned to our email boxes to realise that there were thousands and thousands of emails. At some point we triggered rule #3, which in turn sent an email "out of space", with a small embedded school logo. Each one of these emails triggered our email rule, which in turn triggered an email "could not send message", again with an embedded logo. We desperately tried to delete all of the emails, but it just made way for more emails. We eventually had to abandon our efforts to delete the emails, and went to class.

About an hour later, the email server failed. Several hours later all domain logins failed. It turned out that logins were also run on the email server.

The events were then (from what I was told by IT):

* Students could not save their work to their network directory.

* New students could not login.

* Teachers could not login to take registers or use the SMART white boards.

* IT try to login to the server, failure.

* IT try to reboot the server, failure.

* IT take the server apart and attempt to mount the disk - for whatever reason, also failure.

* IT rebuild the entire server software.

* IT try to restore data from a previous backup, failure. Apparently the backup did not complete.

* IT are forced to recover from a working backup from two weeks previous.

All from one little email rule. I was banned from using all computers for 6 months. When I finally did get access, there was a screen in the IT office that would show my display at all times when logged in. Sometimes IT would wiggle my mouse to remind me that they were there, and sometimes I would open up Notepad and chat to them.

P.S. Something happened on the IT system a year later, and they saw I was logged in. They ran to my class, burst through the door, screamed by username and dragged me away from the keyboard. My teacher was in quite some shock, and then even more shocked to learn that I had caused the outage about a year earlier.

[inopinatus]

You were not the root cause of that outage.

> IT were not stupid

Everything else you described points to them being blundering morons. From an email forwarder that didn’t build loop detection into its header prepending, fucking up a restore, and then malware’ing the student that exposed them into kafkaesque technology remand, all I’m taking away here is third-degree weaponised incompetence

[bArray]

Yes and no. This was the IT of a school, most likely low-paid College/University graduates trying to patch together a working system on a shoe-string budget 20 years ago. Maybe they were fully aware of the issues and struggled to get time to deal with them - try convincing an uneducated management that you need to fix something that is currently working.

I remember IT were continuously fixing computers/laptops broken by students, fixing connectivity issues (maybe somebody has pushed crayons into the Ethernet ports), loading up software that teachers suddenly need tomorrow, etc. Maybe they also have to prevent external actors from accessing important information. All the whilst somebody well above your pay grade is entering into software contracts without knowing anything about software.

Things are likely far more plug & play now for IT infrastructure, back then (XP I think) it was more the Wild West. Only five years ago I know that a University login system used to send username and password credentials via plaintext, because that's how the old protocols worked. The same University also gave me sudo to install/run programs, which provided sudo over all network drives.

You would probably be horrified to know how much infrastructure still runs on outdated stuff. Just five years ago the Chinese trains stopped working because Adobe disabled Flash [1]. I know of some important infrastructure that still uses floppy disks. Not so long ago some electrical testing could not be conducted because the machine that performed it got a corrupted floppy disk.

[1] https://arstechnica.com/tech-policy/2021/01/deactivation-of-...

[inopinatus]

Ah well having operated at all levels of institutional hierarchies I include the hapless/indifferent management within functional and operational scope of the term “IT”, and they are accountable in any case, however understanding you choose to be of the struggling folks at the pointy end. So there’s your root cause.

[direwolf20]

Glad I wasn't the only person who did this.

[trgn]

> and it took us forever to find where he buried the rule.

Salesforce is such an ugly beast

[pinkmuffinere]

lol, that's amazing. Things like this make me both angry (how could they be so dumb!), and empathetic (what is the rest of their life like?)