[falcolas]

You can dictate that a service is running, and it will ensure that the service is running. You can also set up more advanced HTML checks in at least some frameworks[1] to ensure that the contents are being served correctly.

[1] http://docs.ansible.com/get_url_module.html & http://docs.ansible.com/uri_module.html for ansible, I'm fairly certain you can build up something similar for puppet and chef, though I haven't done it myself.

[sandfox]

Writing code/recipes/etc for chef/ansible/puppet is no different to writing code for an application. Just because you think you have dictated (in code) that things should be a certain way does not mean that what will be executed or run.

[falcolas]

The exact same caveats could be applied to unit testing an environment. Common sense and practical tests rule any form of automation.