The party was forbidden, the symbols were forbidden. They hung the main leaders, quite publicly. It became a huge taboo, the ideology effectively died (for decades). A strong democracy was established, older democratic parties took over.
Yes a bunch of previous nazis made it back into power and politics, but they didn't call themselves nazis or acted like nazis. But also, the country as a whole took a very different path after wwii.
A lot of symbolic actions were taken, but the majority (not "a bunch") of Nazis continued to hold positions of power in both the GDR and FRG.
Justice was never served for what the Nazis did. Both the US and the USSR scooped up Nazi scientists (Operation Paperclip), and with the advent of the Cold War, the West quickly decided that it cared more about contesting Europe with the Soviets than seeking justice.
If you ignore Berlin (which, I think, kept its four occupation zones) it were first four, three from January 1, 1947, and two from from August 1, 1948 (https://en.wikipedia.org/wiki/Bizone)