Only the king (at the petition of parliament) can remove a high court or appeal court judge, and that's only ever happened once, in 1830.