Historically it was pure glass but modern retro recreations usually adds that plastic cap, pusher rod and a plastic wrapper for easier manufacturing as well as for corporate liabilities. Those aren't technically necessary.
I have not seen one that is pure glass. The oldest I have seen has a gasket. Every source I see indicates a rubber gasket was used in the first versions.