Although from a transfer learning perspective, chopping off last layers and retraining… You are prob right