[electrograv]

When casual readers see results titled "Bloomberg Innovation Index" (myself included, a while ago), there's a tendency due to brand reputation (Bloomberg!) to believe that this is some scientifically and statistically rigorous analysis of a well-defined "innovation" that can be trusted. This is dangerous, and IMO disappointing (I know, I was naive and optimistic) regarding the statistical credibility of many "reputable" sources.

In reality (as you describe), these are completely arbitrary human-designed heuristic scores with most likely no statistical significance.

I really wish we could qualify these "rankings" with a more honest term , like:

"statistically useless, arbitrarily rated average of multiple human designed score scales, meant to loosely relate to some quality we want to measure, but in reality is more a game of politics and adversarial score optimization."

But that doesn't have the same 'ring' to it as "top country rankings in innovation".

[didibus]

It sounds a little like denial. I mean, Bloomberg does a great job, its pretty simple: "The 2018 ranking process began with more than 200 economies. Each was scored on a 0-100 scale based on seven equally weighted categories. Nations that didn’t report data for at least six categories were eliminated".

So its an equally weighted average of 7 categories. The data was reported by the nations themselves.

Now the key takeaway is that the US dropped out of the top 10. Comparatively, you can tell something is changing in the US causing a drop.

Metrics are just indicator, and can sometimes misrepresent the reality, but more often, there's truth in the metrics also. Its hard to say what the impact of this innovation score is, is it economic, or is it social, but clearly the score change is due to real realities changing.

[totalZero]

> Comparatively, you can tell something is changing in the US causing a drop.

The reported numbers from other countries, or the reported numbers from the US, could also be flawed. Just because numbers have changed relative to one another, doesn't mean that countries have done so as well. You are assuming a particular causal relationship when there are several plausible alternative theories available.

> more often, there's truth in the metrics

Metric isn't the right word. It would make sense for tangible qualities like area, population, and even GNP. But the measured quality here is "innovation," with an amalgam of other characteristics taken as the one true proxy thereof. It's a ranking, but it's not actually measuring innovation.

> clearly the score change is due to real realities changing.

It could also be a change in reported numbers. Or even random noise that tends to revert. These numbers may not even be relevant to the true seeds of innovation.

[toomanybeersies]

You've hit the nail on the head I think.

It's the change in relative rankings that is the useful data from these rankings, rather than the absolute ranking.

[oh_sigh]

Presumably OP is calling into question any such ranking, not just the recent ranking where the US dropped out.

> Comparatively, you can tell something is changing in the US causing a drop.

Yes, one of the factors they used to come up with this rating. Now - the question is, are those factors actually correlated with innovation?

[muninn_]

Sorry but not buying it. The selection of even 7 categories, let alone suggesting that such categories should be equally weighted is unreliable at best. The nations self-reporting the data is completely irrelevant and, again, at best unreliable.

Now, maybe the US isn't as innovative as Sweden is. Ok. What exactly does that mean? Why do I care if the percentage of graduate educated people is higher? How does that actually affect innovation? Are those people releasing new, globally-changing products and services? What are some examples?

What does it mean if Samsung has more US patents than any other company besides IBM? Is IBM more innovative than Google?

It's fun and popular to bash the U.S. (has been for some time) but I really don't see much meaning behind these rankings. It's not an in-depth study. Amazon has more criteria for picking a HQ. Do you really think Bloomberg can look at these '7 criteria' and come up with a meaningful estimation? No.

[watwut]

> Now, maybe the US isn't as innovative as Sweden is. Ok. What exactly does that mean?

It means exactly the same thing it meant when US was among top. There were people who were interested in it and sometimes happy about it. Those very same people are still interested, but this time wonder whether it means something is changing for worst.

[muninn_]

So it was meaningless still. There’s nothing to be interest about here unless unfounded clickbait is a hobby.

[rxhernandez]

You should apply this kind of logic to scientific papers too. It's infuriating that some papers manage to get published that have the type of errors that would at the very least gotten a letter grade taken off(if not an F) for papers I wrote in undergrad physics labs.

[electrograv]

Oh I absolutely do, and IMO this scientific irresponsibility is far more dangerous within fields that actually call themselves "sciences" -- at least these "indexes" don't claim to be scientific (even if some people erroneously assume otherwise, such as my former self). There are certain fields with the word "science" in the title that are notoriously awful in this respect. (I won't name names, just for the sake of simplifying the discussion, but it's not hard to find out.)

Even the notion "hard science" vs "soft science" is a very slippery slope, IMO. We should not seek to speak of "hard" vs "soft" science; instead, we should focus on distinguishing good from bad science. Even science performed on incredibly difficult and complex topics (with immensely numerous confounding variables, near impossibility of controlled trials/experiments, etc.) can still be done correctly! One simply needs to adhere to statistical rigor and qualify conclusions from data with appropriate levels of uncertainty, withhold conclusions with no predictive power or statistical significance, and publish meaningful negative results just as frequently as positive results.

This movement of lax scientific rigor within fields that call themselves "sciences" is incredibly dangerous and threatens to erode the credibility of all results/fields that call themselves "scientific" in the eyes of the general public -- the vast majority of whom do not have the time, energy, or ability to review each field and/or publication to understand how rigorous and honest it actually is.

When you read about problems of anti-intellectualism and public distrust in science, the first thing we should do is look to the "sciences" (and bad journalism) that justify this distrust.

[CandidlyFake]

You forgot the elephant in the room. Money. Everyday these writers have to make up the most outlandish "story/fake news" to get you to click to sell you ads.

"statistically useless, arbitrarily rated average of multiple human designed score scales, meant to loosely relate to some quality we want to measure, but in reality is more a game of politics and adversarial score optimization." while honest and true doesn't get the masses clicking. Honest and true doesn't get stories bumped to the frontpage of HN.

I do feel for these journalists. They are like daily vloggers who have to deal with the constant pressure of generating content every single day to make money. That's like clickbait is so rampant in both the traditional and social media.

[jandrese]

In other words it is about as rigorous as any other Internet listicle[1].

[1] https://en.wikipedia.org/wiki/Listicle