Not really, since the Treasury still honors the interest of the bond. Just because someone values the purchase value lower doesn't invalid the interest rate. You just lose money if you sell it.
Exactly. You're mark-to-market red on your bonds, since if you tried to sell your 2%-yielding bonds in a market where new bonds yield 4% obviously your bond will be worth less than what you bought it for; but you'll still continue to receive those coupons at 2% for the lifetime of the bond.