[joouha]

Not GP, but I too was interested in this slopdetect-minilm-v3 model.

The model appears to be similar to MiniLM-L6 (384 dimension, 6 transformer layers) but uses RoBERTa/GPT-2 style embedding/tokenization (50265 vocab size), used for binary classification, and quantized to INT8.

My guess is that is was distilled from roberta-base, then fine-tuned on freely available pile like artem9k/ai-text-detection-pile or similar.

It's a nice model - I've just used it to create a browser extension which highlights text based on how likely it is to be LLM genereated.

Edit - a quick google search reveals ibm-granite/granite-embedding-30m-english with the same architecture: slap a binary classification head on, fine tune, job done.

[joouha]

Just noticed in the model's metadata:

> MiniLMv2 L6-H384 (30M params) progressively distilled L24→L12→L6 from RoBERTa-large-v8 teacher (neobert-v2 recipe, min_words=0). Byte-level BPE tokenizer. INT8 quantized.