[michaelmior]

What I think would be really interesting is to apply distance metric learning (DML) to the problem. You have users tell you what responses are good and bad and use that to learn a metric that will classify responses as good as bad. One of the big challenges is that DML is typically applied to data in some vector space as opposed to strings, but I would expect using some embedding constructed from the output could work well.

[noaflaherty]

Super interesting idea! We already expose UIs and APIs for supplying feedback on the quality of the output, so this could totally be possible once enough feedback has been collected. Thanks for sharing