| Another example. I chatted with it for quite some time to define a toy language. I asked it to write small programs like an echo server in this toy language, and it did pretty well (though it sometimes hallucinated changes to the language). I then asked for a tree-sitter grammar, and it always got close, but never could fix the problems. Even after I pointed out the problem, its fixes were always wrong. So instead I decided to try a different approach. > Let's write a compiler in C that tokenizes a single file in this language, parses it into a syntax tree, and emits an object file in LLVM IR. I'd like the compiler to be readable and understandable by a human, but execute as fast as possible. The tokenizer it wrote was good, but when it got to the operators, it implemented + and -, and then contained this comment: // Add other operators and delimiters When I asked it to fill in the operators, it actually did a good job. However, this was the parser it gave me. typedef struct ASTNode {
TokenType type;
char* value;
struct ASTNode* left;
struct ASTNode* right;
} ASTNode; ASTNode* parse(Token* tokens) {
// Example parsing logic, build your AST based on the tokens
return NULL; // Placeholder
} Obviously, it's leaving building the entire AST up to me. So I asked it: > Now implement the complete parser. The result was long, so I won't paste it here, but it contained all kinds of comments: // Skipping parameter parsing for simplicity
// parameters would go here
// Simple implementation: only supports return statements for now
// Simple expression parsing: only literals for now
// Add more parsing functions as needed... So I said: > Seriously, I want you to implement the entire thing, not an outline or a framework. And it gave be a code with fewer, but more complete parsing functions. The expression function still had this comment. // Assume we're only parsing integers and binary +,- operations for simplicity And then at the bottom: // Assume the rest of the necessary parsing functions are implemented similarly |
My prompting style is much more direct - things like this: https://chatgpt.com/share/61cd85f6-7002-4676-b204-0349a72323... - more here: https://simonwillison.net/series/using-llms/