You don't have to redo the whole thing, but there are nuances, just like switching between any framework. I was actually able to use a much shorter prompt with VertexAI than OpenAI.
One of the big problems we will have with these large, hosted models is version changes. They don't support them for very long because they are expensive to run and they only have so much capacity. You typically have to redo prompts with each new version. This is why I think self-hosted will eventually win. The upstream vendor will break your pipelines and there's not much you can do about it. With models you control, you decide when they update and can test / update thoroughly before you do.
One of the big problems we will have with these large, hosted models is version changes. They don't support them for very long because they are expensive to run and they only have so much capacity. You typically have to redo prompts with each new version. This is why I think self-hosted will eventually win. The upstream vendor will break your pipelines and there's not much you can do about it. With models you control, you decide when they update and can test / update thoroughly before you do.