Not quite. It's the 'risk-free bond' term that I dropped from my explanation of the put call parity.
To replicate the stock, you'd buy the call and sell the put. The risk exactly balances out in the sense that a total portfolio of 1 stock short, 1 call long and 1 put short would have zero risk and behave like a risk-free bond.
(If the risk premia of the long call and the short put would not exactly balance, you could make money with very simple arbitrage trades.)
To replicate the stock, you'd buy the call and sell the put. The risk exactly balances out in the sense that a total portfolio of 1 stock short, 1 call long and 1 put short would have zero risk and behave like a risk-free bond.
(If the risk premia of the long call and the short put would not exactly balance, you could make money with very simple arbitrage trades.)