[Yusefmosiah]

I wonder if combinators could be useful for neurosymbolic AI—either in the backward pass (e.g., training models on synthetic data) or the forward pass (e.g., iterative code generation with evolutionary algorithms). Combinators feel alien, making even Haskell or APL seem intuitive, but maybe that’s because they don’t align with human working memory. Language models, with their massive context windows, handle long-range dependencies in sequences well, even if their understanding is shallower in some ways. Could combinators, with their compositional and deductive nature, be a better fit for machines than humans? For example, instead of generating Python functions in an evolutionary approach[0], could we use combinators as the building blocks? They’re compact, formal, and inherently step-by-step, which might make them ideal for tasks requiring structured reasoning and generalization. What do you think?

[0]: https://jeremyberman.substack.com/p/how-i-got-a-record-536-o...

[ocean_moist]

Yeah training data is the major issue. You can try programmatically brute-forcing/generating random well-formed (compilable) data and training on that. Then heuristically pick exogenous programs from the output of the trained model to train a new "generation". The issue is in the possibility of the algorithm generating "exogenous" program and "heuristically" picking them out.

[Vecr]

There's probably not enough training data. People use Python with LLMs because it's what works, not because it's the best language. Anything rarer starts having issues.