You could subtract 365.25 days, but then you're left with a new problem: just because you can amortize a leap day over four years doesn't mean that you get an extra 6 hours each year.
And as I just learnt elsewhere in this thread, it would actually be three four-hundredths less than that anyway, i.e. 365.2425, since only one in four centenaries is a leap year.