[macinjosh]

IMHO, LLMs are better at Python and SQL than Haskell because Python and SQL syntax mirrors more aspects of human language. Whereas Haskell syntax reads more like a math equation. These are Large _Language_ Models so naturally intelligence learned from non-code sources transfers better to more human like programming languages. Math equations assume the reader has context not included in the written down part for what the symbols mean.

[WanderPanda]

They are heavily post-trained on code and math these days. I don‘t think we can infer that much about their behavior from just the pre-training dataset anymore

[ozozozd]

They are not called Context-Sensitive Large Language Models though.

LLMs are very good at bash, which I’d argue doesn’t read like language or math.

[gck1]

SQL - somewhat, python - no. LLMs that write code only work well with proper guardrails. Dynamic languages like Python lack essential guardrails.

Once the project can't fit in model's usable context window (~150k tokens even for 1M models), you need code fighting back and leaving breadcrumbs for model to follow.

[catlifeonmars]

I suspect your probably right, but just for completeness, one could also make the argument that LLMs are better at writing Haskell because they are overfit to natural language and Haskell would avoid a lot of the overfit spaces and thus would generalize better. In other words, less baggage.

[travisgriggs]

I would guess they’re better at python and SQL than Haskell because the available training data for python and SQL is orders of magnitude more than Haskell.

[dwb]

I have found recent models to be quite respectable at Haskell, given a couple of initial nudges on style - but that’s true of anything.