Is there anything against glass containers? I'm debating what to use for medium/long term storage (replacing the contents) and indeed I'm wary of metal coatings.
Glass is usually harmless, if you don't use lead or uranium glass. But breakage will lead to harmful shards, which is why many locations such as some schools or party places ban glass bottles.
For storage, use glass bottles. The material of the seal is usually not a problem if there is no contact to the liquid and the seal doesn't crumble. When reusing bottles, make sure to only use fresh or at least good-looking caps. And sterilize things properly (taking into account what the seal is suitable for), because mold and bacteria are far more harmful than common metal or plastic contamination.
Cork is oak, which is carcinogenic (in different amounts in different parts of the plant). Of course for some things one doesn't usually care about that because drinks such as wine are also stored in carcinogenic oak barrels and contain carcinogenic ethanol among other harmful things, so it doesn't really matter.
Also, corks promote microbiological contamination of various kinds, leading to e.g. the common failings in wine (there is actually a multitude of "cork" faults, not just one): https://en.wikipedia.org/wiki/Wine_fault
I'd prefer any metal or plastic to cork actually. Just because it is "natural" or "traditional" doesn't make it any good.
For storage, use glass bottles. The material of the seal is usually not a problem if there is no contact to the liquid and the seal doesn't crumble. When reusing bottles, make sure to only use fresh or at least good-looking caps. And sterilize things properly (taking into account what the seal is suitable for), because mold and bacteria are far more harmful than common metal or plastic contamination.