[idiotsecant]

This is wildly overstating the issue. Hackers are not going to break into hundreds of separate sites, compromise inverters, compromise relay protection, compromise SCADA systems, and execute a perfectly timed attack. Even if they did, these are distributed resources, they don't all go through a single substation and I doubt any one site could cause any major harm to any one substation.

Instead, they're going to get a few guys with guns and shoot some step of transformers and drive away.

The problem with infosec people is they tend to wildly overestimate cyber attack potential and wildly underestimate the equivalent of the 5 dollar wrench attack.

[g_p]

They don't need to break into separate sites though - the issue at hand is that a single failure in the centralised "control plane" from the vendor (i.e. the API server that talks to consumers' apps) can be incredibly vulnerable.

Here's a recent example where a 512-bit RSA signing key was being used to sign JWTs, allowing a "master" JWT to be signed and minted, giving control of every system on that vendor's control system.

https://rya.nc/vpp-hack.html

[Gud]

Most(more or less all of them) grid operators can operate their network remotely from a single control room.

I suspect most grids are extremely easy to hack(never tried, don't bite the hand that feed you etc).

Info sec is just a hobby of mine. I install high voltage switch gear for a living.

[applied_heat]

A lot of utilities have their own fibre since they own poles/towers and need it for tele protection anyway so they can have secure a real private network between control room and significant power plants

[TwiztidK]

> I suspect most grids are extremely easy to hack

I’d expect the opposite. All companies controlling equipment that is part of the “Bulk Electric System” have to be NERC CIP compliant and are audited regularly with large fines for non-compliance. Doesn't guarantee perfect (or even good security) but it’s more likely to be a priority.

[ufocia]

How do fines make things better? They confiscate resources that could be used to improve.

[aerostable_slug]

It also perverts incentives such that no utility will communicate perhaps helpful information to other utilities or government when said information can leave them liable for fines.

Until there's some kind of hold-harmless agreement, the various industry & government security information sharing groups can only be of limited effectiveness.

[applied_heat]

The management at the utility doesn’t want to be recognized for being a deficient operator that doesn’t meet standards, so they hire employees to ensure they are compliant

A fine is a black eye for a utility where people pride themselves on the reliability of the service they provide

[lll-o-lll]

Hurray! I have experience that may shed some insights. I worked on SCADA software (3 different ones), for about 15 years, started off as a Systems Engineer for an Industrial Power Metering company (but writing software), built drivers for various circuit breakers and other power protection devices, and wrote drivers and other software for IEC61850 (substation modelling and connectivity standard). I’ve been the technical director of one of these SCADA systems, and in charge of bringing the security to “zero trust”. I’ve been on the phone with the FBI (despite not being an American or in America), and these days I design and lead the security development at a large software company.

I’ve been out of the Power Industry/SCADA game for about 6 years now, and never had huge involvement with solar farms, so please take this with a large grain of salt, but here is my take. 15 years ago, all anyone would say about industrial networks was “air gap!”. Security within SCADA products was designed solely to prevent bad operators from doing bad things. Security on devices was essentially non-existent, and firmware could often be updated via the same connectivity that the SCADA system had to the devices (although SCADA rarely supported this; it was still possible). In addition, SCADA systems completely trusted communication coming back from the devices themselves, making it relatively simple for a rogue device to exploit a buffer overrun in the SCADA. After Stuxnet + a significant push from the US government, SCADA systems moved from “defensive boundary, trust internally” to “zero trust”. However, devices have a long, long service life. Typically they would be deployed and left alone for 10+ years, and generally had little to no security. Security researches left this space alone, because the cost of entry was too high, but anytime they did investigate a device, there were always trivial exploits.

Although SCADA (and other industrial control software), will be run on an isolated network, it will still be bridged in multiple places. This is in order to get data out of the system, but also to get data into the system (via devices, and off-site optimisation software). The other trend that happened over time was to centralize operations in order to have fewer operators controlling multiple sites. That means that compromising one network gives you access to a lot of real world hardware.

Engineers never trusted SCADA (wisely), and all of these systems would be well built with multiple fail-safes and redundancies and so on. However, if I were to be a state-actor, I’d target the SCADA. If you compromise that system, you have direct access to all devices and can potentially modify the firmware to do whatever you want. If there is security, the SCADA will be authorized.

I don’t think the security risks are overblown (they are overblown in what they think the real problems are). I think that as the systems have gotten ever more complex; we have such complicated interdependencies that it is impossible to deterministically analyse them completely. The “Northeast blackout of 2003” (where a SCADA bug lead to a cascade failure), was used as a case-study in this industry for many years, but if anything, I think the potential for intentional destruction is much higher.

[applied_heat]

I’m in this space, but plc io networks from Schneider and Rockwell are still “trust internally”, and some HMI or scada has to have read/write to them. At least Rockwell you could specify what variables were externally writeable whereas Schneider was essentially DMA from the network.

[1053r]

This isn't hundreds of separate sites that have to be hacked individually. This is fewer than 10 clouds with no security to speak of and the ability to push evil firmware to millions of inverters worldwide, where in a few years at the current rate of manufacturing growth, it will be 10s, and then 100s of millions of inverters.

Yeah, the potato cannon filled with aluminum chaff or medium caliber semi-automatic rifle can take down a substation. But this is millions of homes and businesses, which can all have an evil firmware that triggers within seconds of each other. (There will inevitably be some internal clocks that are off by days/months/years, so it's not like it will happen without warning, but noticing the warning might be difficult.)

And the growth in sales is exponential!

[ethbr1]

> medium caliber semi-automatic rifle

Technically, anything that can put a hole in an oil-filled transformer. https://en.m.wikipedia.org/wiki/Transformer_types#Liquid-coo...

You don't need to break it... just crack the radiator enough for all the circulating fluid to drain, then it overheats.

[EdJiang]

Important to point out this isn't just theory, it's actually happened (in the SF Bay Area!) with a regular rifle.

https://en.wikipedia.org/wiki/Metcalf_sniper_attack

https://www.npr.org/sections/thetwo-way/2014/02/05/272015606...

[ethbr1]

Also in the north GA mountains in the 1970s.

[applied_heat]

Any transformer over about 5 MVA will probably be equipped with a low oil level switch that de-energizes it

[jpc0]

If all you wantes was to kill the power I don't see the difference...

Sure the repair is easier/quicker but the economic damage was already done...

[applied_heat]

The lead time on a transformer that size is probably a year, so a year long outage has a lot more damage than a 2 week outage

[lll-o-lll]

Would the switch on the transformer possibly be software controlled? (By software, I am wondering about firmware on a device reading a sensor, as opposed to a physical mechanism). I don’t know enough about the internals of these things, but I wonder if you could maliciously overwrite firmware, whether certain protections could be made to fail.

I’m going to assume this kind of thing is likely covered in FMEA and such, so is unlikely.