Hacker News new | ask | show | jobs
by hinkley 2055 days ago
We had enough problems with "hardware can't remote" with server equipment that they made systems to control them. Those systems are useful even if you're in the same building and just don't want to go hang out in the cold, loud server room while you figure out the problem, and everyone wonders if you're even working today (since they can't see you).

Or you might not even have access to that room.

We just haven't had much motivation to do this for a more general class of hardware, and there probably haven't been enough non-pandemic motivators to build such things, and it would be pretty macabre to build a system just in case there's another pandemic, or to buy the thing if it existed.

Time sharing hardware with someone on the other side of the planet, for instance, might introduce too much delay for a Big Red Button. Apparently it's about .25 seconds if you're in the room. You can do a lot more damage in another 200 ms, and packet loss could make that delay bigger.

1 comments

Testing is one field where you already might see fairly automated unsupervised/remote setups that are not that difficult to use for remote development too, although surprisingly often testing is still done manually.
We had some experience with this, and the device under test or even testing harness can fail, so you need a local operator to fix things anyway.

It is more feasible to ensure no such failures the simpler the device is, e.g. with a good hardware watchdog setup. We did it with a rather complex stack.

Of course, you also need someone and somewhere to set it up in the first place after all, but it makes a big difference between "you need the entire team in one place" vs "you need a place that has a test engineer or one member of the team available/on-call". (To continue the parallel from /u/hinkleys comment, lights-out management is useful even though the server still can't swap a broken disk itself)

Seeing that now with Covid at customer sites: only 1-2 people from a larger group in the office, who if needed can go poke devices, while the majority works remotely (which often includes contractors that otherwise would be in hotels etc). Generally not optimal, since many don't have the amount of automation one would prefer, but workable and with room for improvement.