[bluGill]

If you could build a caste (Expensive, even if you use slave labor) that also meant you had the ability to hire and train knights. In fact most castles were built because the king (who often wasn't very powerful, thus the need to make these deals) offered you rule over some area of land in exchange for raising those knights.

The Castle was built as a place to live on your land so you could keep it. If there was a peasant revolt the castle meant they couldn't do anything about you, while your army could leave anytime the peasants were busy elsewhere to harass them. Thus the castle enabled war as you said, but the castle generally came after raising the army in the first place.