Was caught off guard that it rates the following text at only "3.2% chance AI generated":
"As a large language model, I am not able to answer this question."
Interesting. In my experience, ChatGPT always says "As an AI language model..." or lately just "Sorry, I can't help with that." Have you seen "As a large language model..." come out of any of the big LLMs?
We're trained on real ChatGPT data so am interested in hearing your prompts that result in this.
I typed this from memory, so probably not repeating LLMs word for word. If I change that prompt to "As an AI language model..." the detection works as expected. In a sense, a broader point my original comment demonstrates is that even for an absolutely obvious AI-generated text, auto-detection can't work reliably because it's trained on specific responses of specific LLMs that can be altered at any time.
To be clear: not attempting to discourage you. It's a very complex and interesting problem to tackle.
We're trained on real ChatGPT data so am interested in hearing your prompts that result in this.