Pretty much all of these "most" findings are explainable by the distribution of the installed user base, i.e. they are not real results but just artifacts of the population sizes.
Yes, you’re right! The report covers data from Instabug users only. We extracted data from 30K apps with a range of user base sizes, locales, devices, etc. I definitely agree with you, it’s not a definitive representation of the market, but we believe we have a good enough sample and that the findings are valuable for app developers.
We couldn’t find any other data on mobile bugs like this, so we decided to share what we have for the app dev community to have some benchmarks and insights.
Sample size isn't the issue here, although a large heterogenous sample is good.
The complaints are e.g. "Most bugs are reported from iPhones" because they are a very popular type of phone with the customers more likely to report bugs. It doesn't necessarily mean the iPhone is buggier than others.
Right, we're definitely not saying the iPhone is buggier than others. I guess the issue is with the wording of the claim. It would be more accurate to say "Most bugs reported through Instabug are from iPhones"
That's also why we included the bugs/user data since it shows a completely different distribution across devices.
I think you're right for the most part, but there are some interesting gems in there which it would be real interesting to get more information on, such as the "Bugs/User vs. Device Manufacturer" graph.
Part of that is explained in the comments here by an employee saying they assume Google (and to a degree iPhone/iPad I'm sure) get increased numbers because devs might use them for testing and thus more bugs are seen, but that does raise interesting questions about why LG leads them all in that metric.
Almost all the graphs that are for total bugs instead of normalized to number of users show very little that is useful. One exception to that I noticed is the bugs to battery level, and that was only useful in that it's a reminder that mobile devices spend a lot of time running while plugged in at full battery, which is just as easily said than inferred through a graph like that.
We couldn’t find any other data on mobile bugs like this, so we decided to share what we have for the app dev community to have some benchmarks and insights.