[dtagames]

What you're describing isn't how GPT training works. Mostly, they work on next token prediction without having any understanding of what those tokens actually mean. It works well for text and images but it can't lead to a reproducible set of steps.

I wrote an article[0] about it recently that you might enjoy.

[0] Something From Nothing | A Painless Approach to Understanding AI

https://medium.com/gitconnected/something-from-nothing-d755f...

[purplerabbit]

The tokens could describe a sequence of actions and their consequences vs. blog / forum type content

[dtagames]

Unfortunately, what a token describes is exactly what an LLM doesn't understand. As I explain in my article linked previously, procedural steps with determinate outcomes need procedural, traditional code rather than predictive LLM results.

If you just want something to predict the next best step or likely outcome, LLMs can already do that by fine-tuning on the kind of data you're talking about.

FYI, today's LLMs aren't trained on blog and forum type content as you mention but actually contain millions of books, academic papers, and other legit sources. Then, they're fine-tuned by a specific industry or company to include actual papers and data from their industry.