[JackGreyhat]

Do you use this approach?

If so, are you willing to share an example robots.txt file? What bots would be allowed? Google, Bing, ...? Those?

[nuc1e0n]

robots.txt is a standard and has been for many years. It seems that https://www.robotstxt.org is the main repository for its definition.

Any robots.txt I might suggest wouldn't necessarily be appropriate for anyone else. It's a policy that site stakeholders need to decide for themselves I'd say.