[nickspicer1993]

Do you have any books that you would recommend on the subject? I'm all for people being informed but I have looked for information myself sporadically over the last 10 years and it's a very sparse landscape from my point of view.

[ohthehugemanate]

Start with Kahneman and Tversky's work on the Planning Fallacy [1], and follow the rabbit hole to Reference Class Forecasting [2]. I don't know about popular science books on project management, though, sorry. There's some good, readable material about Agile estimation as a group of practices, but you have to avoid the dogmatic stuff from so many of the specific implementations (I'm looking at you, scrum fetishists). Any of the Agile Manifesto authors are a good bet, since they've been around to watch various implementations come up. Martin Fowler, Uncle Bob, etc. Whatever you think about their code advice, their estimation advice is worth listening to.

[1] https://en.m.wikipedia.org/wiki/Planning_fallacy

[2] https://en.m.wikipedia.org/wiki/Reference_class_forecasting

Relevant papers:

Buehler, Roger; Dale Griffin; Michael Ross (1994). "Exploring the "planning fallacy": Why people underestimate their task completion times". Journal of Personality and Social Psychology. 67 (3)

Kahneman, Daniel; Tversky, Amos (1982). "Intuitive prediction: Biases and corrective procedures". In Kahneman, Daniel; Slovic, Paul; Tversky, Amos (eds.). Judgment Under Uncertainty: Heuristics and Biases. Science. Vol. 185.

Kahneman, Daniel; Tversky, Amos (1979). "Prospect Theory: An Analysis of Decision under Risk" (PDF). Econometrica. 47 (2)

Flyvbjerg, Bent (2006). "From Nobel Prize to Project Management: Getting Risks Right". Project Management Journal. 37 (3)

Hope this helps!