[kijin]

Restart is fine. It only takes a fraction of a second.

But APT doesn't do restarts. It stops services and doesn't start them back up until it's finished with all the other packages. This can take a few minutes if there are kernel updates that trigger multiple initramfs rebuilds, or indefinitely long if the update process throws up a dialog to confirm something.

OP makes it clear why this is happening. Packages have no concept of an upgrade, so APT stops services before removing the old version, and starts them back up after installing the new version.