A couple of days ago I was trying to make Chatgpt give me the equivalent s3:// string for an https://... s3 bucket. The stupid thing would just replace http with s3, no matter how much I told it what I wanted (I needed to reformat several links, and search engines were just giving me s3 -> http scripts).
It finally landed on me that I was using gpt3.5 . Once I moved to gpt4 it got it right the first time.
I was trying to get it to write some Elixir code to handle Erlang B calculations and it just couldn't figure it out, but it was pretty confident with the wrong things it was printing out.
The rust borrow checker often requires too much critical thinking for ChatGPT, as well.
LLMs confer a tremendous productivity boost. You just have to understand their limits and know when it's faster to think through and write the solution yourself.
No, because there is nothing I wanted it to do instead. Perhaps what I should have done was to change the system prompt to something like "You are a helpful assistant that will admit its mistakes but never apologise for them."
It finally landed on me that I was using gpt3.5 . Once I moved to gpt4 it got it right the first time.