Hacker News new | ask | show | jobs
by pewu 334 days ago
Once I tried to reverse engineer a simple checksum (10 ASCII chars + 1 checksum byte), gathered multiple possible values and fed it to Gemini 2.5 Pro. It figured out the calculation completely wrong, when I applied the formula in code I got completely different checksum. After debugging step by step it turned out it hallucinated the value for sum of 10 integer values in all of the sample data and persistently tried to gaslight me that it is right. When I showed the proof for one of the sample entries, it apologized, fixed it for this specific entry and continued to gaslight me that its formula is correct for the rest of the values.

This time ChatGPT gave me a much better result.