[parched99]

I think Powershell is a bad test. I've noticed all local models have trouble providing accurate responses to Powershell-related prompts. Strangely, even Microsoft's model, Phi 4, is bad at answering these questions without careful prompting. Though, MS can't even provide accurate PS docs.

My best guess is that there's not enough discussion/development related to Powershell in training data.

[fragmede]

Which, like, you'd think Microsoft has an entire team there who's purpose would be to generate good PowerShell for it to train on.