Pretty sure freezing is exactly it — Greenland and Antarctica contain lots of water stored well above sea level. These fluctuated a lot worth sea levels.
Several people are saying this so I’ll reply to just this one. It wouldn’t make sense for water to freeze up over Antarctica over time as a way to reduce sea level. Water freezing on top of water doesn’t affect water height. And water freezing on top of submerged land would just raise the water levels because it’s more voluminous. There needs to be a mechanism for the ice to accumulate on top of itself far above sea level for sea level to go down beyond just water freezing. As another poster explained, it’s snowfall accumulating semi-permanently in the same spots that moves water to these areas. Excuse my pedantry.
Not pedantic, I think people just didn't realize you were narrowly thinking of seawater freezing directly when you said "freezing wouldn't explain it".