[EvanAnderson]

I find it's important to remember, too, that a failed PING tells you nothing other than your echo request did not receive a response. If the remote host received your request, and if it responded, are both things a failed PING can't tell you, because both of those things could be true but you still end up with a failed PING.

I've seen technicians get tripped up in troubleshooting thinking that a failed PING tells them more than it does. When the possibility of asymmetric return paths is involved it's always important to remember how little a failed PING actually tells you.

[jacquesm]

And that can be a lot more subtle than you might think. I've had a persistent very hard to debug false alarm triggered on pings sometimes not making it and most of the time they did. But very rarely that would happen three times in a row and that was the threshold for raising an alarm. We spent days on this. Finally, the root cause was tracked down to a BNC 'T' connector at the back of a media adapter that filtered out the header of some percentage of ICMP packets. It is one of the weirdest IT problems I've ever encountered and it makes me wonder how much of what we rely on is actually marginal.

[ceejayoz]

> It is one of the weirdest IT problems I've ever encountered and it makes me wonder how much of what we rely on is actually marginal.

Vernor Vinge had a character who was a "Programmer-Archeologist" on a relativistic starship. Feels more and more prescient as time goes on.

[cmehdy]

I work at a company that invented an internal syntax to compile into C++ code, that still relies on c-shell and conventions taken when OS/2 was in use there, and with a web of Jenkins instances and homemade wrappers and DBs to build that stuff.

I can safely say that title exists already. And I value my current experience as a humbling example of what is to come as software becomes an older industry, and not just a world of startups and their freshest languages/frameworks/tools.

[thwarted]

The way you describe this system is exactly how I'd describe a system I worked on in the early 90s at PW (before it was PWC).

[gnerd00]

jenkins did not exist though

[pbh101]

I thought that too up until this GenAI moment, and now I wonder if needing to be an archaeologist will be so valuable if one can get your needs met by a quickly GenAI-written script/program.

[palmotea]

> I thought that too up until this GenAI moment, and now I wonder if needing to be an archaeologist will be so valuable if one can get your needs met by a quickly GenAI-written script/program.

I never have actually read those books (though I read some summaries about them, interesting concepts). My understanding is the "programmer-archeologists" basically had an archive of massive quantities of very high-quality software that did pretty much anything you'd want software to do. So it made more sense to find the software you need and glue it together than write from scratch.

And given GenAI doesn't write high quality software (at least not yet, and hopefully never), I don't think that "GenAI-written script/program" would be a good replacement (though an AI archeologist might make more sense, with such an archive).

[pbh101]

The world in question is ours but later, with direct lineage from Unix systems indicated. So I see these archaeologists as a glorified priesthood of shell scripters, grep still having bugs, and the glue being programs themselves. Not too different than many roles today.

Beyond that, it is an odd hope on your part for GenAI to never be able to write high quality software.

Zooming out, my bigger point was that this was a sci-fi book written by the person who coined the term and concept ‘Singularity’ and the series includes a malevolent murderous sentient AI virus (IIRC) and it included some reference to how programming was accomplished and yet still, given all that, there was no anticipation of even our nascent current GenAI coding capabilities.

[jacquesm]

I've yet to have my needs met by a GenAI-written script/program. Archaeologists tend to be a lot more precise in their statements, especially about what is speculation and what is not.

[toast0]

I mean, if you're willing to accept AI slop, that's fine. But if you're willing to accept AI slop, you'd probably be willing to accept human slop (at least if it claims to be AI) too, and then the job gets a lot easier.

[pbh101]

We’re talking about a sci-fi scenario that presupposed a lot of things but not anything that wrote code for you to the extent that society found value in dedicated code librarians. The state of AI today has nothing to do with re-inspecting that future world in light of the last 3 years of GenAI progress.

[resonious]

I work on regular web stuff and people already call themselves archeologists when doing routine tasks on our 15 year old codebase.

[EvanAnderson]

I feel like an archaeologist working on code I wrote myself 20+ years ago. I can imagine the feeling is stronger when it's somebody else's code.

[gosub100]

I'm a SRE and encountered this recently. To prevent DDoS, there is a buffer setting on the kernel that will limit the number of pings (a few settings actually). So if you have a group of machines that all ping a single destination at once, it's very possible to have some that fail to get a reply.

[pixl97]

It's for reasons like this that ping is one of the worst protocols to use for aliveness.

Even worse is I've had completely dead Linux boxes that will gladly respond to ping and nothing else.

[jacquesm]

Oh, that's nasty. How long did it take you to troubleshoot that?

[gosub100]

Relatively speaking, it wasn't that bad. It took a few weeks of getting trouble tickets with no root cause, and a bit of googling. But management wasn't okay with fixing the root cause, instead they just increased the timeout/retry window.

[jacquesm]

Wow. That's a classic. We were quite motivated because we were the ones that got the automated alerts. I still see them in my nightmares: "chopper is down". The machine was called chopper, I'll never forget, it's been close to 30 years. My buddy Jasper and me spent multiple nights trying to track it and when we finally found it we still couldn't believe that that was it. But a simple swap was proof.

[dylan604]

Did someone yell for you to "Get to the choppa! Do it! Now!!"?? Please say that's not been wasted!

[kulahan]

I've always assumed that in situations like this, a traceroute is better. You can get more information simply by reaching the next stage in the trace, even if you're given zero information beyond "I'm now at the next server".

[EvanAnderson]

Nothing says the Time Exceeded packets elicited by your traceroute have to follow the same path back to you that you initiating packets followed out. It's a convenient fiction to think about IP networks acting like circuit-switched networks and in most LAN and small WAN applications they do. Mostly you can get away with thinking that way in more complex networks, too. When you end up in a situation where path asymmetry is causing you grief, thought, it's nice to have the understanding that each datagram can have a unique routing destiny.

[pstuart]

Traceroute uses ICMP and can encounter the same problem ping does.

This has come in handy instead -- https://linux.die.net/man/1/tcptraceroute

Disclaimer: not a network engineer but dependent on packets going from A to B.

[webstrand]

I had an experience recently setting up a third-party VPN where the echo responses were being delivered to the correct (host,interface) but with the wrong destination address (not the same as made the request)