I think that LGPL for the encoder is exactly the right choice. A format's strength is in uniform support; taking a MIT-licenced encoder and making an improved incompatible format won't be great for end users.
Not explicitly, but in practice, it does. It is often not viable to jump through the hoops required to do it. That is, if your lawyers will even let you try.
Having it as a dynamic library is not enough. It has to be a dynamic library tgat you can replace. This doesn’t work when, for instance, distributing through many app stores.
Also, a GPL-licensed encoder will in no way stop incompatible extensions.