Judging from the downvotes, a good portion failed at thinking about it. Answer:
In the 20th century, we commonly expressed the year by its last two digits, you know this is why we talk about "70s", "80s", "90s". It's a small number, not ambiguous, and it also happens to be "the number of years since 1900". So of course a computer in the mid to late 20th century would choose this base. IT means the number is convenient for display, but also works for the machine, because "years base 1900" is a valid machine model as well. And it became a popular time format others picked up.
Lo and behold, though, the world didn't end in 2000. What do we do? Well, let it go over 99 and we add "1900" to it for display purposes, so we can maintain BC.
There. That's why we didn't choose any other base from 4.3 billion years or whatever.