That's always a risk! One of the strengths of the proposal is that if a maintainer slacks on defining a solid API testset, users can submit the tests they think belong. At that point the responsibility is baked in: once a test is added, you either keep it green or bump the version, enforced by the registry.
If a maintainer staunchly refuses to define an API, that's useful information, the kind you can't get with standard SemVer, where the only mechanism is trusting strangers to do the right thing. Which, to be fair, works ok, some of the time.
> you either keep it green or bump the version, enforced by the registry.
assert(true); is a thing. I don't think this solution would actually work. Tests might be refactored or improved and that shouldn't trigger a major release.
Malicious compliance is in fact a useful escape hatch here, a maintainer can release a 1.0 where the entire API is "test 1 + 1 == 2". That, too, is useful information.
But the package registry checks all API tests against the last version and rejects the registration if they change at all. That can be relaxed for non-semantic parts of the test, like a description of what the test means, but none of the code is allowed to change. It would be better if this were based on the AST, so that whitespace tweaks don't trigger a build failure, that's practical to achieve in most languages.
Refactoring an API test isn't worth losing the guarantees a system like this provides, and it's only the API tests which come with any restrictions, maintainers may do as they please with the rest of the test suite. An improved API test has to be provided as a new test. Part of the proposal is that users can refer to API tests in their own code, as a way to determine if tests they rely on break in a major release, so the tests need unique names, which means they can be rearranged in the file. It also means that if there's a typo in the name, or the name sucks, well, you're stuck with that until the next major release, and even then it goes on to live in infamy, forever, in the obsolete-API portion of the test suite. Not ideal, but it can't be avoided.
hmm. Interesting. So, most likely, "stable" software will likely release with a major version somewhere in the hundreds instead of 1.0? Since initial development usually means lots of breaking changes while details are discovered/built, I can't see 1.0 having any useful meaning.
It's not all tests, it's just the API tests. I'm not sure why that was unclear to so many people. You can have hundreds of tests, thousands even, only the API tests are special.
If there's no stable behavior because the software is still at that stage of development, it's 0.x software still. That's true in SemVer as well as this refinement of it.
Contrariwise, if you think software is ready for 1.0 and you can't come up with any tests which display guaranteed behavior which won't change without that major version bump, then no, it's not ready.
That’s what I’m saying though. At some point, you have to write those tests and there will be bugs. There will be things that aren’t ideal. It’s like the first time you write the CI/CD pipelines and you have to commit 100 stupid things directly to main to test it and make sure it works.
If a maintainer staunchly refuses to define an API, that's useful information, the kind you can't get with standard SemVer, where the only mechanism is trusting strangers to do the right thing. Which, to be fair, works ok, some of the time.