A user was having a really bizarre problem: They could log in when they were sitting down in a seat in front of the keyboard, but when they were standing in front of the keyboard, their password didn't work! The problem happened every time, so they called for support, who finally figured it out after watching them demonstrate the problem many times:
It turned out that some joker had rearranged the numbers keys on the keyboard, so they were ordered "0123456789" instead of "1234567890". And the user's password had a digit in it. When the user was sitting down comfortably in front of the keyboard, they looked at the screen while they touch-typed their password, and were able to log in. But when they were standing in front of the computer, they looked at the keyboard and pressed the numbers they saw, which were wrong!
I did tech support via phone for a popular consumer computer brand. One particular call, a woman reported that her computer was restarting every time someone in the house flushed the toilet.
Long story short, her home was in the back-back woods with the home powered by a generator. In addition to powering the computer, the generator was also the source of power for a water pump which would kick on to refill the toilet bowl whenever it emptied. And wouldn't you know that that water pump had a beefy coil around its motor and would brownout the entire house every time it started?
I have a similar one, with an automated monorail hoist. The engineer who started the job had ordered the monorail hoist with a control cabinet with Ethernet comms to tell it where to move (instead of just controlling the hoist directly from the main control cabinet.) After days' worth of shenanigans trying to troubleshoot seemingly random comms drop-outs I'd narrowed it down to only occurring when the hoist was being lowered under load, which led me to the Ethernet cable in the hoist cabinet which ran parallel to the motor cables from the hoist's 6kW VSD. Whenever it lowered, the EMI was enough to nuke the Ethernet connection. Re-routed the Ethernet cable and after that it ran fine.
My personal example: VoIP phones stopping after the Asterisk server was up for 3 days.
Reason: the server had IPv6 turned on, and it steadily accumulated privacy IPv6 addresses. These addresses were all sent in a packet describing the supported media endpoints, using UDP.
And yep, eventually it overflowed the MTU and the phones couldn't handle the fragment reassembly.
Thanks. I've never run Asterisk on Ubuntu. FreePBX is CentOS based which is mostly what I've run Asterisk on. I only started to worry about Ubuntu around 2012.
That mad IPv6 address thing must have stuffed up more than just a VoIP negotiation packet. DNS switches from UDP to TCP when responses get too large.
DNS is affected, but differently. It's mostly DNSSEC signatures that cause trouble nowadays.
SIP is special because the signalling and media protocols are separate. So when a call is being established, the parties exchange their media endpoint locations. This necessarily means that the server has to list its IPs (or DNS names) so that the client can choose the best one. And as a quirk of SIP, it sends the entire set for each of its supported codecs.
Yes, I know, just like ftp 8) I do hope that whomever invented putting the control channel in a separate stream from the data is mildly discomforted. Mind you, all that stuff was invented a very long time ago, when trousers were a major trip hazard.
My go to fix is "symmetric RTP", which seems to have become a default over the last decade or two.
I remember one (might have been a hn-er's comment, dunno) about the computer restarting when the toilet was flushed. Turns out it was due to voltage drop when a compressor turned on to refill the reservoir of the toilet.
That's why in rural locales with spotty power it pays to have a UPS on any electronics -- you might not benefit much from 15-30 minutes of extra power in a day long blackout, but it keeps everything happy when the voltage fluctuates.
Thanks for mentioning! I would love more submissions! I have a few stories in my backlog to read and vet, but not enough. I'll be going through this thread and adding more that haven't been added yet.
DNS responses sent over UDP are often truncated if the response is too large. This manifests itself as "machine unreachable if name > x characters" sort of errors when you have really long FQDNs.
https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...