[the_gastropod]

Property testing is awesome, but it does significantly slow down test suites. Are there any standard practices for striking a balance of getting the added confidence/safety of using property tests without introducing large delays into your CI pipeline?

[mullr]

We like to run the tests as part of CI with a relatively small number of iterations, and then turn the knob way up in a nightly or weekly scheduled test job.

[Smaug123]

Yup. We've got some stuff set up so that in the IDE your tests finish in ~100ms; in precommit testing pipelines you get several seconds; in nightly tests you get two hours.

[hinkley]

My understanding, shallow as it is, is that property testing goes great with pure functional code, because without side effects you can run tests in parallel, taking advantage of all of these cores and build servers we have.

If your tests are coupled, you're already in a bad way whether you know it or not. Dumping property testing on top of that without addressing the underlying cause sounds like a recipe for misery.

It's probably a great stick and carrot if you're pushing a tech debt reduction agenda though.

[Jtsummers]

Item 5 in TFA is one that I found related to performance most. Using filters to toss out bad input means you're potentially generating thousands of inputs for dozens of tests. Aggregate this across all your tests and it's an incredible waste of time. Constructing proper inputs directly can get you closer to the minimum time required to execute the tests and help achieve that desired performance threshold. (NB: Not used professionally, only got one office to even entertain them but they wouldn't take it and run with it even after I showed them a dozen errors I'd found in an hour of using them. Consequently my experience is still limited.)

[k__]

You could fan-out tests to multiple FaaS instances.

Cloudflare Workers, for example, have no cold-starts. But they are JS/WASM only

[amw-zero]

That’s a lot of buzzwords you got there