I didn't realize it was only activated in private windows. Now that you've made me notice I consider this misleading, on the settings page it says nothing about being private windows only.
It most certainly does say that this only applies to private windows, "Use Tracking Protection in Private Windows" [1]. If you want the always using tracking protection you can set privacy.trackingprotection.enabled to true in about:config or install the disconnect extension.
[1] https://support.mozilla.org/en-US/kb/settings-privacy-browsi...