With a large enough TV at a close enough distance, it can make a difference. On my 55" at about 4m distance I can see better detail in 4K but 1080p is still very good. Anything larger and/or closer, and the difference will be a lot more pronounced.
I'm sure I would be slightly happier with a modern 4K TV (as long as I could avoid getting a "smart" one), but I'm still perfectly content with my 720p TV from 2007. My main gripe with it is that it doesn't play nice with some kinds of HDMI input (it doesn't have good support for adjusting overscan), but even that is no more than a minor nuisance.
It's definitely not worth even the ~$500 we paid for this TV to get a new one just at the moment.