Could Transformers/LLMs take over custom software implementations given enough training? Why have so many custom ERP software suites or anything else related to the backend if a transformer can do the work?
> Why have so many custom ERP software suites or anything else related to the backend if a transformer can do the work?
Because they can't. Transformers are great at probabilistic tasks that traditional computing can't do well, like natural language processing. They're very suboptimal for the tasks that traditional computing is already good at.
The first and most obvious reason is that traditional computing is deterministic and predictable in its behavior. This is great for business systems because errors can be tracked down to specific lines of code that can be demonstrably fixed and regression checked. A transformer doesn't have this property—when something fails you can try tweaking parameters like temperature or fine tuning it longer, but it's hard to prove that the specific bug won't show up again.
The second reason is cost—even saying you can make a reliable transformer-based ERP system, in the long run it won't cost less than the traditional equivalent. The initial training cost can be enormous and you still need highly paid staff to organize that work. Once you have the working system, your business will still need to make changes to the rules over time, so you'll still need to keep some of those staff on hand to make those changes, but you also have to pay for the GPU time to run the application, which will cost much more than the equivalent traditional ERP.
Finally, even if all of that sounds fine, at some point you need to store the data and host a web server of some kind, which means you're going to need to interface with deterministic computation. I guess you could go the tool use route and let the LLM generate its database queries on the fly, but that makes all the reliability problems in the first paragraph much much worse. And if you're going to hire web devs anyway, why not just have them do everything?
Just wondering if we to combine stochastic Transformer approach with its deterministic NLP cousin namely Typed Feature Structure Grammar (TFSG) as implemented inside CUE language (non Turing), is it possible together in combination with Transformer become the generic computing backend [1],[2]?
Before transformer NLP people use deterministic approach like TFSG as being employed by CUE language [2],[3].
Because they can't. Transformers are great at probabilistic tasks that traditional computing can't do well, like natural language processing. They're very suboptimal for the tasks that traditional computing is already good at.
The first and most obvious reason is that traditional computing is deterministic and predictable in its behavior. This is great for business systems because errors can be tracked down to specific lines of code that can be demonstrably fixed and regression checked. A transformer doesn't have this property—when something fails you can try tweaking parameters like temperature or fine tuning it longer, but it's hard to prove that the specific bug won't show up again.
The second reason is cost—even saying you can make a reliable transformer-based ERP system, in the long run it won't cost less than the traditional equivalent. The initial training cost can be enormous and you still need highly paid staff to organize that work. Once you have the working system, your business will still need to make changes to the rules over time, so you'll still need to keep some of those staff on hand to make those changes, but you also have to pay for the GPU time to run the application, which will cost much more than the equivalent traditional ERP.
Finally, even if all of that sounds fine, at some point you need to store the data and host a web server of some kind, which means you're going to need to interface with deterministic computation. I guess you could go the tool use route and let the LLM generate its database queries on the fly, but that makes all the reliability problems in the first paragraph much much worse. And if you're going to hire web devs anyway, why not just have them do everything?