[ben_w]

With a sample size of 2 because I've not tried before and it's late here, gpt-4-1106-preview seems to work OK at this just by asking for a probability.

Though you will need to post-process the response because it says the usual blah blah blah ${number} blah blah reason blah.

[firtoz]

It will be so expensive :(

But btw now you can enforce function calling and json responses

[ben_w]

Hopefully the smaller models can also be used for spam checks; perhaps even some of the open source models? My insomnia-driven test last night was the complete opposite of an exhaustive study.