| Thank you for the feedback. I chose (for better or worse) to cite the most recent article as opposed to maybe the least controversial one. Other papers I've read have suggested similar ranges; nobody presents a stddev so it's hard to compare. And how you classify "debugging-related tasks" is kind of subjective, as you correctly point out. So linking other papers might or might not be useful anyway. In "A framework and methodology for studying the causes of software errors in programming systems" (Ko & Meyers 2005), they cite a NIST publication as saying 70-80% of time is spent debugging. I parroted this off in a talk once, and then I looked at the same publication ("The Economic Impact of Inadequate Infrastructure for Software Testing" RTI 2002) and the only 80% I found was time spent in testing software in the early days of software engineering. In the 1990s, it finds coding and unit testing to comprise about 30% of the time. This is as far as I can tell, at least similar to the (1) and (2) in the CVP survey. But again, indirect comparisons, who knows? In tables 6-4 and 7-6, the RTI study finds different numbers for time spent on bugs, and different frequencies of where bugs end up being discovered. Whatever the rate, it is clear that post-production discovery bugs are the hardest to fix, and I think this is because you only ship code when you have high confidence in its correctness. I think this is a hard measurement to make. It's hard to find participants, hard to guarantee they have the same idea of what "debugging means", hard to quantify. So thank you for calling me out on this. But I think other studies have evidence that this isn't too far off from a correct measurement. For example, Gugerty and Olson in 1986 did some novice vs expert study at debugging LOGO programs. (I don't like the Dreyfus model for classifying debugging, but whatever.) They found that novices took about 18 minutes to fix bugs, testing an average of 3.6 hypotheses per program, with the first hypothesis being correct only 21% of the time, and re-executing code every ~3 minutes. Experts took 7 minutes, testing an average of 1.6 hypotheses per program, with the first hypothesis being correct 56% of the time, re-executing code every 2 minutes. So guesses are only right maybe half the time if you're really good. Basically any novice vs expert study compares how much time people spend on the problem (they're trying to reduce debugging time), but it's hard to extrapolate this into percentage of development time. For example, some people spend 100% of their time debugging because they're on QA or sustaining engineering teams. To be quite honest, there _fundamentally is not_ enough research into the practice and pedagogy of debugging. I have about 60 papers in my "Debugging Papers" folder that I've read over the past year, and basically everything up until the Dweck-inspired research in 2008 isn't great. There are only 9 papers in all of computer science that cite Dweck that I've found as of maybe 6 months ago (I'm not a researcher and I don't look that often). From memory, two of them were recommending further research based on her work, maybe five of them were from people in CS departments helping Dweck create games and programs to perform her research, and then only two were actually applying her work to debugging. I would agree with you if you said this needs more research, and that the claim isn't strongly supported by the provided evidence. I greatly appreciate your feedback. |