At least with a Junior Engineer they can learn and grow from your feedback, becoming a more useful member of the team, the generative model on the other hand will not.
I would imagine the context takes up valuable input tokens that you would otherwise need to use for your request. So you'll run out at some point and then you just have a simple model rather than a skilled engineer.
Use Composer notebooks to keep a growing markdown document of context you want future versions to remember.
My Cursor today is much better than my Cursor on day 1.