I guess that a 'poison' is a chemical that kills by biological effects... drinking fifty litres of water in a minute will kill you by mechanical means, which is kind of beside the point... the element iron is also a lethal if five hundred kilograms collide with you at a hundred metres per second, but you haven't been 'poisoned'.
Nature does not care about our definitions, and properties are usually on a gray scale, not black-white. Inert gas have very low reactivity [1], but they can still react:
> Nonetheless, low reactivity—instead of no reactivity, as had formerly been thought—characterizes the rare gases. One of the factors governing the reactivity of an element is its electron configuration, and the electrons of the noble gases are arranged in such a way as to discourage bonding with other elements.
That said, I would probably classify noble gases as perfectly safe, since damage from exposure to them will more likely come from mechanical interactions, and not biological / chemical.