| 1. Many frameworks create boilerplate when you create a new application. Do we count those LOC? Yes if it's code someone could edit. Someone unfamiliar with the framework needs to read these lines of code. Boilerplate may be "pre-typed" for you, but it's part of "your" app, and it's optional. That's not the same as a lib or the framework itself. You could start your app differently. The new hire has to deal with these LOC regardless of who typed them (you or a wizard), so they're significant. 2. Many frameworks on dynamic languages copy their entire corpus of functionality as source files into the application's root. Certainly we don't count those. Check out our Github repo's colored language bar at the top right. Agree this should not be counted. Only what's part of the app in question. Put another way, if those directories could be "hidden" from the new hire, and he could do his job, don't count them. 3. Do build scripts count as LOC? No. 4. Do configuration files count as LOC? Yes. The Spring framework's pom.xml or any dependency injection "configuration" file is crucial, so yes. On configs, though, I'd guess kbenson's idea about diff is reasonable. - - - However, about diff on #1 above, I'm concerned that with diff you're really just measuring how different the test is from the boilerplate. That says more about the test than about the framework. Someone could also optimize their framework's pre-typed boilerplate to match your test. (As we see on browser rendering benchmarks, for example.) Really, if it's code that's typed (by the framework or by you) into an application, it should be counted. I'm coming to this from the standpoint of hiring a new developer onto a project. More time is spent maintaining projects than jumpstarting them. At some point really soon after jumpstarting the app, boilerplate and edited code are going to be intermingled and indistinguishable. The new guy has to wrap his head around it all. So I would like any code typed by you or for you to be counted. And beyond LOC, it matters how those lines are distributed. If you have to open five nested files to find out what one hello world is doing, even if each file only has a single 3 line function in it, that's complexity that matters. Number of files and number of directories both make a framework feel very different. Most benchmarks are for weekend MVPers. Your benchmarks are moving into "real world" territory. In the real world, being able to wrap your head around someone else's existing project matters. LOC, files, directories, and dependencies, factor into that heavily. |
That's true. Maybe a two numbers, one with and one without boilerplate? That way someone could get an idea of what they are looking at. (I know, I'm just stacking more and more work up...)