What would happen if Russia attacked these power plants directly? Are they built to "fail safe" even when hit by a missile, or is there a big risk of nuclear meltdown?
Big risk of nuclear meltdown. Nuclear reactor cannot stop immediately. Gigawatts of power will damage equipment by high voltage and current, then reactor will melt. Nuclear reactor in Zaporizzhya finally turned cold after more than year, with multiple attempts made by Russians to disconnect it from grid by shelling the grid and targeting repair teams.
> What would happen if Russia attacked these power plants directly
There is zero need to attack a NPP directly.
If you really need to shutdown it then just target the power distribution system - a reactor can't run without providing the energy and if there is no load then you need to shutdown it.