IE5's box model was the good one (* { box-sizing: border-box; } is the vestigial "please use the IE box model" in almost every CSS file today). It was Netscape/early Mozilla's box model that was hard to reason with.
But also, using Tables for page layout had less to do with box layout problems and more to do with the lack of something like CSS Grid (finally, decades later). Tables let you define regular grids in a way that was closer to designing a grid (thinking in terms of spans across cells instead of widths/margins). Tables for layout didn't truly go away until around the Bootstrap era with its column grid helpers (and their complex CSS math to make that work before both flexbox and native CSS Grid).
As a Firefox partisan since back when it was still Phoenix, of course I'd argue it was IE 5's box model that sucked. But the real problem is that there wasn't one box model that behaved identically across all the popular implementations.
The old Netscape/Phoenix/Firefox box model had borders take up half-inside half-outside and doing any sort of math with it and getting things to align at all was terrible effort in CSS.
I love Firefox to this day, but mathematically the choices that box model made made sense as easier on the renderer but was awful to work with for the web developer.
It's great that we have the choice between box models today and their behavior is better standardized. I know if you surveyed CSS in the wild you'd find most CSS is written for the IE box model (border-box) and yet it is funny that that is the one you have to opt-in to because that adds to the overall impression that CSS by default is broken and needs reset stylesheets and boilerplate.
All true, but the point at which web dev more generally moved to CSS over tables was considerably later, not least because early CSS was pretty hard to find resources on and work effectively with. Firebug helped a lot from 2005 on, as of course also the browser-native devtools it inspired, but in 2007 it remained far from a surprise to see sites built in what was then still a current, if not the latest, style.
But also, using Tables for page layout had less to do with box layout problems and more to do with the lack of something like CSS Grid (finally, decades later). Tables let you define regular grids in a way that was closer to designing a grid (thinking in terms of spans across cells instead of widths/margins). Tables for layout didn't truly go away until around the Bootstrap era with its column grid helpers (and their complex CSS math to make that work before both flexbox and native CSS Grid).