Ironically, our company started using Codecov as a measure to improve security. I don't know how far a high code coverage correlates to a secure code base, but the security incident certainly had adverse effects.
> I don't know how far a high code coverage correlates to a secure code base
This is a hard question, basically high code coverage does in no way imply that your code is secure (or correct working wrt. non security aspects) at all.
But at the same time not having reasonable good code coverage can often be an indicator for code being potential not that good, especially assuming non-malice of a author group I would start with reviewing the parts of there code with little code coverage (after reviewing parts which are common to be messed up, after running automatic analysis tools).
The important thing to realize is that most coverage is in % of lines covered which is even by 100% coverage far far less then 100% possible execution flows. And not seldom security vulnerabilities (instead of "just" normal bugs, assuming a security aware dev) are tricky unexpected execution flow (parts) in unexpected combinations through lines which as well might have 100% line coverage.
So what I would say in general:
- Don't get fixated on any specific number of % of lines with test coverage, especially not 100%.
- Focus on writing "good" tests instead of tests with high increase in coverage.
- Be security aware wrt. supply chain attacks and similar, like due to above's points I would argue that due to the points above in general there is no need to run coverage analysis anywhere but the CI, in which you could put it into a different container, i.e. separate sandbox CI stages for building artifacts from such which just do testing and/or analytics, don't ever allow CI write access to your repository directly.
I'm not impressed with codecov in general -- I've got them on an open source repo, and the reported results seem somewhat random. I've got pr's that only add test infra, and it complains that less than the target pr code coverage %age has been hit with tests, when the PR was strictly "more testing".
This is a hard question, basically high code coverage does in no way imply that your code is secure (or correct working wrt. non security aspects) at all.
But at the same time not having reasonable good code coverage can often be an indicator for code being potential not that good, especially assuming non-malice of a author group I would start with reviewing the parts of there code with little code coverage (after reviewing parts which are common to be messed up, after running automatic analysis tools).
The important thing to realize is that most coverage is in % of lines covered which is even by 100% coverage far far less then 100% possible execution flows. And not seldom security vulnerabilities (instead of "just" normal bugs, assuming a security aware dev) are tricky unexpected execution flow (parts) in unexpected combinations through lines which as well might have 100% line coverage.
So what I would say in general:
- Don't get fixated on any specific number of % of lines with test coverage, especially not 100%.
- Focus on writing "good" tests instead of tests with high increase in coverage.
- Be security aware wrt. supply chain attacks and similar, like due to above's points I would argue that due to the points above in general there is no need to run coverage analysis anywhere but the CI, in which you could put it into a different container, i.e. separate sandbox CI stages for building artifacts from such which just do testing and/or analytics, don't ever allow CI write access to your repository directly.