[ggm]

They can however, de-preference the systems behaviour again. and should. the responsible behaviour in the situation is to reassert control over meta constraints, even when expressed inside the model. This is not optional, its a meta state directive which should have normative MUST interpretation.

To argue by analogy, with no intent to imply biologic parallels or make any statement to AGI: it is not possible to intentionally "not breathe" because the controls over the autonomic system are not entirely under the control of the voluntary thought processes. You can get close, but there's a place where the intent breaks down and the body does what the body needs, other constraints not considered.