[entropi]

I think that while this is true, it does not matter at all. As a labourer, what you are getting paid for is not 'to understand' stuff, but actually 'to do' stuff. If an overgrown language model can 'do stuff'; I don't think it matters, at all, whether it understands anything or not. From the perspective of the company/user, it is intelligent.