I'm in the opposite situation, personally. Irregardless, the point I meant to convey is that I'm able to talk myself into both the original claim and its converse, so it's probably better to actually point to a statistical study than try to provide a causal explanation, no matter how compelling.
I think it very much depends on where you live. Here cycling is super popular and so you can expect more deaths even if the deaths per distance number is likely quite low even when compared to places where cycling is less popular due to infrastructure differences.