[jt2190]

  > Configuration that can perform arbitrary computation can 
  > put our programs into literally any state...

Considering that it's generally a poor design that expose the full state of an application to all parts of the same application (think: "global state"), then yes, it holds that exposing the entire application state to the configuration system is also a very poor design. I would think that if a user could set three values via configuration, then those three values would be the only state exposed.

  > It also opens up the risk that people won't configure 
  > their software correctly...

We can always find someone who can't configure the system, regardless of how the configuration is done. If we're worried about making configuration "safer", then the system should be more careful about what configuration values it accepts i.e. implement robust validation.

  > But I think one of the biggest risks is fragility. 

Systems that can't adapt to unanticipated needs of their users get replaced by systems that can adapt. It's why many pieces of mature software develop this robust configuration ability (often hiding under the label "api", "scripting interface", "plugin framework", etc.)

  > Software that gets used by lots of people will end up with 
  > towers of complexity built into the configuration, to the 
  > point that removing or refactoring them risks bugs, edge 
  > cases, or breaking a particular feature or misfeature that 
  > someone relies on.

This is not a inevitable. Many mature systems have successfully incorporated fully programatic configuration [1] without turning into "towers of complexity". (It probably says something about the application's architecture if it's so tightly coupled to it's configuration.)

[1] Unix is the poster child for this.