| From time to time I've tried using ChatGPT for financial modeling, and I have to say my experiences don't inspire much confidence. Just this past week I used it to generate a simple model of a few different scenarios related to an investment property I own. The first problem I ran into is that it was unable to output a downloadable XLS file. Not a huge deal - it suggested generating CSV tables I could copy/paste into a spreadsheet. The outputs it gave me included commas in a handful of numbers over 1,000 (but not all of them!) which of course shifted cells around when brought into Google Sheets. We pivoted our approach to TSV and solved this problem. Big deal? No. Seemingly basic oversight? Absolutely. This is where the real fun began. Once I started to scrutinize and understand the model it built, I found incorrect references buried all over the place, some of which would have been extremely hard to spot. Here's my actual exchange with ChatGPT: - - - - - - - - - - > Can you check the reference in cell F3? It looks like it's calling back to the wrong cell on the inputs tab. Are there similarly incorrect references elsewhere? > Yes, F3 is incorrect, and there are multiple other incorrect references elsewhere: (It listed about 30 bulleted incorrect references) Bottom line -
- Many formulas point to the wrong Inputs row because of the blank lines
- The Sell + Condo section also has a structural design problem, not just bad references. The cleanest fix is for me to regenerate the entire AnnualModel TSV with:
- all references corrected
- all 15 years included
- the condo scenario modeled properly with a separate housing asset column - - - - - - - - - - This was me asking about the exact output I had just received (not something I had made any changes to or reworked.) There are plenty of domains where I have enough faith and error tolerance to use ChatGPT all day, but this just sends a chill down my spine. How many users are really going to proof every single formula? And if I need to scrutinize to that level of detail, what's the point in the first place? |