| I think property tests work well on larger things, with a super basic assertion they're essentially just fuzzers. You can point hypothesis at a swagger spec and let it test your API like this. I built a property testing library back in the day when I made a library for creating UIs. I was then able to write tests like * Given an arbitrary UI * And an arbitrary up/down/left/right list of user direction (this was the only way of navigating) * If they press a direction and the focus moves, pressing the opposite direction takes them back to where they were before This uncovered bugs in interfaces like this with 3 items, bottom left is in focus and you press right. Now press left, users probably want to go back to the bottom left rather than top left. ┌─────┐ ┌─────┐ ┌─────┐ ┌──────┐
│ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │
└─────┘ │ │ └─────┘ │ │
│ │ ────► │focus │
┌─────┐ │ │ ┌─────┐ │ │
│focus│ │ │ │ │ │ │
│ │ │ │ │ │ │ │
└─────┘ └─────┘ └─────┘ └──────┘
Another one was* Given an arbitrary list of API calls to add/remove/change the UI and user direction presses * There is always only one item in focus, or no items at all This actually uncovered a specification bug. We had two requirements 1. Always have an item in focus if there are any that can be in focus 2. If you delete all the items, then add one back in, it's not focussed Those conflict, but we never noticed, and even had passing unit tests for both cases. I think property tests can map very nicely to the level of system description that we typically want. I'd love to see larger integration with BDD tools. |