The problem is that when you put weight on your limbs you are creating levers and inertia which get transferred to joints in ways those joints are not good at dealing with.
One usually doesn't do much with their arms while carrying groceries, they just hang at the side or move the bag around when putting it down or picking it up. When performing other movements, especially when they are fast or forceful, the extra weight can add momentum that is potentially hazardous, especially if it bounces or moves around (makes it very hard to compensate for reflexively). If present during repetitive tasks, extra weight can increase the risk of repetitive stress injuries occurring.