The LLM doesn't have to know about the critic though. It can just output things and the critic is a second process that filters the output for the end user.