>I think the Japanese meaning encompasses any kind of roundish object
As in Chinese, 金玉 also means "gold and jewels" in Japanese, i.e. treasure, so the semantic distance to testicles is not any smaller. Presumably someone might have heard of "family jewels".
>For a Chinese speaker, using 金玉 (gold+jade/precious) to describe testicles probably wouldn't be too strange
金玉 is used in the idiom 金玉满堂, which can either mean full of riches or knowledge,[1][2] so it would be rather strange.