[jonhohle]

I’m currently looking at some slow python scripts and libraries that take about 30% of the total build time to generate 20 linker scripts for a project that builds 1,300 C files. Every dev, every build, every CI run is waiting on slow python. So the devs that wrote the tool are happy, but everyone else is slowing dying waiting around for this thing to finish. Relevant [0]

0 - https://www.folklore.org/Saving_Lives.html

[igouy]

Where's the second part of the story: where someone else profiled and discovered 19 of the linker scripts were generated really fast, and that re-working generation of the slowest script only took 15 minutes.

[jonhohle]

I have been profiling and it’s really just python being slow.The first ⅓ of the run of 12 cores is all python. Fortunately these are mostly independent so there is currently no blocking, but there eventually will be.

[igouy]

Thanks. I don't have any expertise to share. I'm just curious. Might PyPy be faster?

[makeitdouble]

> Every dev, every build, every CI run is waiting on slow python.

Parralelize the build, buy more resource for the CI. It might cost more but it will be "saving lifes" after all, right ?

The question is usually whether those scripts would have existed if it wasn't for Python. I assume if it was trivial to rewrite them you'd do it in 2 hours and go on with your life.

[jonhohle]

Besides the python scripts, everything is parallelized and is CPU bound on as many cores as it can be given. Because of licensing throwing more CI at it isn’t an option. This is an open source project, so there’s not really money for buy moar bigger.

The tools possibly wouldn’t exist, but there are options now that provide better ergonomics and are not slow.

[makeitdouble]

I think that's the position most "slow" scripts are: They could have been written faster in the first place, but they weren't intended to be kept for so long and/or that wasn't a priority at all in that moment. And now that people are looking at them again they could be fixed, but there's usually still not enough merits to do so.

I assume something will done at some point, but IMHO it's one of these very nice problems to have, as there is a working version that will only be replaced by something better, and only if it's worth it.