It is absolutely a correctness issue. Panics do not provide safety guarantees that generalize enough that it is safe to arbitrary recover from them. The statement in the previous sentence is not a subjective opinion, it's a statement of fact. I'm not sure how else to convey this information.
When some bit of code invokes `panic` it is saying that there is an error which is unrecoverable, and the default expectation is that the process will terminate. There is no way to assert that panics do not violate runtime or memory model expectations. They can.