Hacker News new | ask | show | jobs
by mring33621 45 days ago
L O C A L M O D E L S

All data stays on computers that you control.

Same API. Localhost.
1 comments
mring33621 45 days ago
Try Mistral-Nemo-2407-12B-Thinking-Claude-Gemini-GPT5.2-Uncensored-HERETIC_Q4_k_m.gguf. This 7.5GB model runs well in llama.cpp on my 2021 Macbook Pro and is good at both coding and business document analysis tasks.
link
NekkoDroid 45 days ago
> Try Mistral-Nemo-2407-12B-Thinking-Claude-Gemini-GPT5.2-Uncensored-HERETIC_Q4_k_m.gguf.

Thiss sounds like such a shitpost I initially thought you were joking... but this seems to be a real model???

link
cpburns2009 45 days ago
There's a method to the madness:

- Mistral-Nemo: the actual model developed by Mistral and Nvidia.

- 2407: likely the release date of the base model, July of 2024.

- 12B: the model has 12 billion parameters.

- Thinking: the model operates in thinking mode (generates output plan and injests it before producing actual output).

- Claude-Gemini-GPT5.2: I think this means the model was finetuned with session data from Claude, Gemini, and GTP5.2 to replicate their behavior.

- Uncensored-HERITIC: the model was uncensored using the automated Heretic method.

- Q4_k_m: the model is quantized (lossy compression) to ~5 bpw from orignal 16 bpw.

link
NekkoDroid 45 days ago
Yea, I know what the parts individually mean. I just meant as a whole it just seemed so obsurd.
link
mring33621 45 days ago
It is! I like to try the variations from possibly 'interesting' people.

Some of them are good. Others randomly break into gibberish and Chinese poetry(?).

link