If an entry has been replicated to a majority of followers, then the new leader is guaranteed to have that entry and therefore it won't be rolled back.
That is correct. The solution to this is given in section 5.4.1 (election restriction), section 5.4.2 (Committing entries from previous terms) and section 8 (Client interaction) of the Raft paper.
Roughly, a newly elected leader will have all committed entries (guaranteed by the "election restriction", 5.4.2) but it does not know precisely which are committed. The new leader will commit a no-op log entry (section 8) and after it has received replies from a majority of the cluster it will know which entries have already been committed.