Exactly. You really should avoid doing performance optimizations without measurements that show improvement as well as provide some coverage against performance regressions.
Here's an example, Thrift (as used in Hector, a Cassandra client), had someone make a performance improvement:
The discussion has a lot of "shoulds", and one measurement of latency distributions, but no measurement of typical workloads or bulk inserts. Turns out, that caused at least a 30% performance regression:
Here's an example, Thrift (as used in Hector, a Cassandra client), had someone make a performance improvement:
https://issues.apache.org/jira/browse/THRIFT-959
The discussion has a lot of "shoulds", and one measurement of latency distributions, but no measurement of typical workloads or bulk inserts. Turns out, that caused at least a 30% performance regression:
https://issues.apache.org/jira/browse/THRIFT-1121