Hacker News new | ask | show | jobs
by horsh1 914 days ago
So with the current state of technologies we can answer all kinds of questions regarding a sentence, and an llm can even invent questions to all different words in the sentence. And we can translate between hundreds of languages live and dead. And we can even ask llm to produce a parse for a sentence.

But we just can not solve the question of mass translating sentences to their AST.

It amazes me. I really hope someone will step up and tell: sure it is possible as a cheap byproduct of the transformer technology, just do this and that.
2 comments
mikewarot 914 days ago
ASTs are the result of parsing source code that fits a specified grammar. No such fixed grammar exists for English or any other human language.

You're asking for the impossible.

link
mikewarot 913 days ago
Further pondering about this.... every word has multiple meanings, and the choice of which is intended can only be resolved into a probability, as are the positioning of the words within a sentence diagram, and the structure of that diagram.

The best you could get, is NOT an AST, you could get a tree of possible sentence diagrams, each one having an overall probability of being the right one.

link