OP here, these are some of the off-the-shelf options we've looked at so far and why they don't quite fit us:
- https://www.flightcontrol.dev (looks great but pricing is out of our league + only support the provisioning of a limited set of AWS services which isn't necessary a deal-breaker)
- https://flexstack.com/ (pricing + also seems to only support the provisioning of a limited set of AWS services)
- https://seed.run/ (pricing but also more serviceless focused that we're looking for)