[ahaucnx]

Achim from AirGradient here. Good to see that my post has been submitted here. Happy to answer any questions you might have.

I spend quite a long time writing this post and it actually helped me to see the bigger picture. How much can we actually trust tech reviews?

I am already getting very interesting results from the survey I posted and already planning to write a follow up post.

[edent]

Ultimately, you shipped a broken product.

That points to a lack of QA on your part and, I think, it is fair for a reviewer to point out.

Even if you have an exemplary warranty process and easy instructions, that's still a hassle. Not everyone has the confidence or the time to repair simple things.

As for the objective/subjective nature of reviews. Are your customers buying air monitors for their 100% precision or for "entertainment" purposes / lifestyle factors?

I have a cheap Awair air monitor. I have no idea if it is accurate - but it charges by USB-C and has an inconspicuous display. That's what I wanted it for.

It is perfectly fair for a reviewer to point out their personal preferences on something like this. They aren't a government testing lab.

[16bytes]

If you sell a physical thing, some percentage of them will have defects. That's just a fact of manufacturing.

It seems unfair to move to "not recommended" due to a single instance of a hardware failure, especially if the manufacturer made it right. And repair-ability is one of their core values!

At most this should've triggered a "this happened to me, keep an eye out if this seems to be a thing." note in the review instead of moving to not recommended.

[edent]

If you got food poisoning from a restaurant, would you recommend it to your friends? After all, food-borne pathogens and poor hygiene are just a fact of life.

How about if they gave you a voucher for a free drink to say sorry?

Reviewing products is like interviewing people. You have to go by what you see on the day. Your can't review (or interview) based in what could have happened; only on what did.

[toomuchtodo]

Yes, if it happened once. If I get food poisoning every time, probably not. Perfection is impossible, I am reasonable and mindful of the challenges of consistency.

Hardware device arrives damaged or non functional? I’m just going to call and ask for another one. If it’s a critical need (I cannot wait for a return and delivery cycle), I’m buying more than one upfront. Spares go in inventory.

[16bytes]

Did the product result in physical harm? No. A better analogy would be ordering a med-rare steak, but it coming out overdone.

If that happens once and the restaurant makes good on the mistake, I wouldn't hold it against them.

Sending something back to the kitchen is a way better product analogy than food poisoning.

[kadoban]

> If you got food poisoning from a restaurant, would you recommend it to your friends? After all, food-borne pathogens and poor hygiene are just a fact of life.

There are standard practices that avoid the vast majority of food poisoning. Poor hygiene is not a fact of life, it's a failure of process in a restaurant.

There are no known standard practices to avoid all faulty electronics at anything like a reasonable price. From the sounds of it, this unit worked initially but failed over time. That's what warranties are for, this is why they exist. As a society we've decided that it's kind of okay if _some_ products fail early, as long as the companies make it right when they do. Which it doesn't sound like the company had any lack of intention in doing that here.

There is no corresponding societal understanding for your analogy.

[shkkmo]

> Ultimately, you shipped a broken product.

They shipped a working product that failed after several months, was covered under warranty and also repairavle at home (which the review doesn't even mention.)

> It is perfectly fair for a reviewer to point out their personal preferences on something like this.

But that isn't what happened. The product was included as "not recommended" not just "this wasn't my favorite due to X but would be good for this type of person".

They make up a category to list every other unit as "best for" but decided that nobody should want to buy this one because the Author got annoyed.

The review is poorly written and doesn't do a good job of actually comparing units. It is the kind of review article that mostly fluff with very few details about actual differences revealed during testing that I have learned to ignore when looking for information about what to buy.

[ahaucnx]

A few thoughts from me on the discussion so far, which I find incredibly insightful. Many thanks to everyone sharing their perspectives. I truly appreciate it.

On subjective reviews: I think there's absolutely nothing wrong with reviews based primarily on an author's subjective opinion. However, such reviews should be appropriately labeled. For example, "My Favorite Air Quality Monitors" rather than "The Best Indoor Air Quality Monitors". The title sets reader expectations for objective evaluation with consistent methodology.

On the defective display: Important clarification: we did not ship a broken device. The display issue developed during the review period—this wasn't a QC failure on our part. Hardware can fail during use (as it can with any electronic device), which is exactly why we immediately offered replacement parts, a new unit, and detailed repair instructions when we learned about it.

On the tiny display and lessons learned: We're well aware that opinions on our display vary significantly, as evidenced by this discussion. Some users love it, others find it too small. We actually have differing opinions within the AirGradient team as well. We're planning a refresh of our indoor monitor next year and are currently testing around 10 different display types—including e-ink, colour OLED, touchscreens, and others. So far, we haven't found the ideal replacement, but we're planning to involve our community later this year to gather feedback on the various options.

[pwg]

> For example, "My Favorite Air Quality Monitors" rather than "The Best Indoor Air Quality Monitors". The title sets reader expectations for objective evaluation with consistent methodology.

Unfortunately, that ship has sailed. There have now been so many review articles for so very long titled "Best X" when the nature of the review is "... in the subjective opinion of the review author" that it is unlikely anyone views a "best X" article as having any objective evaluation or rigor behind it at all.

Your suggestion would be nice to enforce, but there's no way to get that ship back to port to change its course now.

[redbluered]

eInk.

I hate blinking lights in a bedroom.

[mind-blight]

Super smart move. I hadn't heard of you folks before, but I'm interested in your product - open source and repairability are high on my list for home monitors. I'm lying in bed awake right now sir to an air quality issue, so it's top of mind.

The only thing you're missing for me is radon detection. I just bought a house and tests came in below remediation levels, but the report showed a lot of spikes and variance. So you have any plans for a model with radon detection in the future?

[jzellis]

I think your concerns are legit, but it's not necessarily the reviewer's fault that they're on three deadlines and don't have the time to give your product the care and concern it deserves. It's probably the editor's or the publisher's.

I'm a world class writer but I stopped doing it for a living a long time ago. Why? Because as media moved from print to online, the work was devalued. I've worked for 25 cents a word sometimes, which was pretty decent when one 1200 word piece could pay rent back then. Nowadays, writers are offered $25 per article flat with no compensation for rewrites. Staff positions pay badly for too much work but are as coveted as C suite gigs are in the tech world. Maybe more so.

So if the reviewer is staff, they might be assigned three or four reviews in a given week on top of other work. If they're freelance, they might have to take on more just to make their rent. This is because your average magazine staffer who's not management pulls about as much as a Starbucks manager, and was ever thus, unless you got in at Vanity Fair or The Atlantic back in the Before Times.

It's like when I was reviewing albums for $50 a pop: I'd get a stack of them to review and cue up track one and if I didn't get hooked pretty quick, I'd just pop in the next one.

Your device arrived damaged, which is absolutely no one's fault, but your reviewer doesn't have time or honestly impetus to give it a second chance. Not for whatever they're getting paid for that review, which is not much at all.

It's just bad luck, is all. And yes, it's not fair and, yes, you're right to complain, but it's not as simple as "tech writer lazy".

(And if anyone's response is "They accepted the job, they should do their best at it no matter how little it pays", I'm guessing you've never had to duck your landlord to try not to get evicted before the freelance check you've been hunting up for three weeks arrives. There's a reason I'd rather make a living as a mediocre coder than a very good writer these days - at its worst, the tech industry is more renumerative and stable than the publishing industry is.)

[shkkmo]

> I think your concerns are legit, but it's not necessarily the reviewer's fault that they're on three deadlines and don't have the time to give your product the care and concern it deserves. It's probably the editor's or the publisher's.

You wrote it, you don't get to dodge the responsibility like that. Professional integrity still matters.

> It's like when I was reviewing albums for $50 a pop: I'd get a stack of them to review and cue up track one and if I didn't get hooked pretty quick, I'd just pop in the next one.

That seems unethical.

> There's a reason I'd rather make a living as a mediocre coder than a very good writer these days

As coders, we also have an ethical responsibility to pushback against code that will harm people.

We need more writers to say no to writing stories with insufficient time/resources to do it ethically same as we need more developers who push back against building unethical products.

[tobr]

> I'm a world class writer

This is awkward, but I think you mean ”I'm a world-class writer”.

[handoflixue]

Be kind. Don't be snarky.

[redbluered]

One of my least liked HN rules. I'm from a culture where good- natured ribbing (snark) is not considered unkind.

I rather enjoy snark, whether by me, at me, or just reading.

[skylurk]

Snark is well tolerated here, when it's not boring.

[redbluered]

HN rule #1: Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes

Click 'site guidelines' when posting. You'll see.

I don't like that rule, but it is a rule.

[pwg]

From your "pop out" in the article:

"is that this review is ... pretty much purely based on the personal preferences of the author."

You've found the core takeaway about nearly all "product reviews" in nearly all publications. They are almost all simply "the personal preferences of the author".

These authors have neither the time, nor the science skills, for anything even beginning to look like a rigorous scientific review, and so the "best" vs. "ok" vs. "not recommended" tags applied result because the author liked the particular shade of pink used on a trim piece on one, or liked that another one looks like the Apple computer they are using, and so forth.

But they are never based upon any objective criteria, and are never (nor ever were intended to be) reproducible in any scientific fashion.

Yet, as you say, they have "great power" to influence buying decisions on the part of folks who read their reviews.

[bryant]

> But they are never based upon any objective criteria, and are never (nor ever were intended to be) reproducible in any scientific fashion.

This is also why review aggregators exist: if I'm just getting into a thing, such as watching movies or buying appliances, I probably need a general sense of how people collectively feel about a thing. But if I'm keenly aware of my preferences, it helps me to find reviewers who align with how I think. People routinely seek out specific reviewers or specific publications for this reason.

For instance, someone reading this review might conclude "I really appreciate that ease of use is a topic that's front of mind with this reviewer." Another reviewer's focus might be customizability, and they might recommend AirGradient. And that reviewer's audience follows that person likely because those concerns are front of mind.

...to be honest, if AirGradient had responded more along those lines ("we prioritized X and Y. We understand if this isn't the best fit for customers looking for Z, but we're working on it"), it would've felt more empathetic and less combative to me.

[philipwhiuk]

> Another Monitor: Recommended, despite having no display at all

Isn't this an outdoor one. Outdoor ones aren't expected to have a display because you want to check them without going outdoors.

This seems reasonable.

[ahaucnx]

No, this is actually the purple air indoor monitor which has no display.

[axus]

Is Wired really the first place people think of for this kind of product review?

Which sites and publications would you recommend, and which have are biased for financial reasons?

[cestith]

I for one would reach for Consumer Reports well before Wired for recommendations. I might still read the review at Wired for the details and decide for myself what’s important rather than their recommendation.

[jjulius]

First, the thing I'm not really seeing mentioned anywhere here in the HN comments is that a separate AirGradient sensor was #3 on the list of "recommended" sensors, and was specifically called "Best Budget Quality Air Monitor". I also can't seem to find this mentioned in the piece that you wrote, either. Why not highlight that success?

You write:

>How can a product be penalized for a failing display when another recommended product has no display?

This is an incredibly perplexing take. A display is subjective - whether or not the customer wants one is up to the customer. What the customer does want is a functional product, so regardless of what another product's features are, if that product functions as intended and yours does not, of course it's going to be recommended over yours.

>How can an indoor monitor without CO2 sensing - essential for understanding indoor air quality - be recommended over one that includes this crucial measurement?

Again - the products without CO2 sensors functioned as intended. It is indeed accurate that CO2 is one of the most critical metrics for assessing indoor air quality, but it goes back to my previous comment - perhaps the customer is more interested in PM2.5 indoors than CO2 for a specific reason. We don't know. Ultimately, the CO2-less sensors functioned as intended, whereas yours did not.

You go on to say:

>And specifically for situations like this: How would you want us to handle it? Should companies stay quiet when review methodology breaks down? Should we be more aggressive in calling this out? Or is transparency and open discussion the right approach?

Maybe focus less on one review and more on improving the product? As another comment states, you shipped a broken product and it suggests that there's a problem with your QA process. Further, you state early on:

>Let me be clear: this was a legitimate hardware failure, and we take full responsibility for it. As soon as we learned about the issue, we immediately sent replacement parts and a new unit, including repair instructions, as repairability is one of our core differentiators.

Let's maybe hear more about that. How/why did the hardware fail? Did you examine your QA process and make any improvements to it? Highlight these steps, as well as the "core differentiator" that is your repairability, rather than asking perplexing questions about why one reviewer didn't like your product.

As an "average Joe" customer in this area, the whole response feels excessive and... whiny (driven by the fact that you don't highlight that you did, in fact, have a product on the list that was well recommended). I don't say that to be terribly mean, it's just a bit off-putting. You're not necessarily wrong about product reviews like this in general, but like... who cares? Put the effort into making a solid product, not taking umbrage with one person's opinion.

There will be more reviews, and some of them will be negative. You're not going to be able to control perception and opinion, and nobody will ever get perfect marks from everyone. Learn to be OK with that.

Edit: I just saw your response about this not being a hardware failure when shipped. Still, the general concept of my point remains - detail what you're doing to determine how this happened and prevent it in the future, rather than griping about the review process. If "transparency is how [you] operate", lemme hear the deets about this issue!

[ahaucnx]

Yes, I did not mention our outdoor monitor being featured because I don't see this as a success when I critique the whole methodology of the review.

As I mentioned above, we are working on a refresh of the indoor monitor. The display is also under discussion but so far with 10s of thousands of the indoor monitor sold, I am only aware of a single digit number of cases of a failed display.

[liminal]

Will your device measure hydrogen sulfide levels?

[redbluered]

I have your product. I like your product. I like your company. Fundamentally, I can't disagree with the review. You got unlucky, and the reviewer was looking for a different product. It happens.

It's not a universal product. Competitors have upsides and downsides, and different people want different things.

I think this is the time to move on, and focus on people so like and appreciate you, and not dwell on those who don't. Success brings more of both, and if you can't handle a few haters, you probably don't want to be too successful.

And reviews are imperfect, but a lot better than no reviews. Accept life isn't perfect.

Thanks for a great product and for running a company with integrity.

[no_wizard]

The fact you didn’t include a tired / wired meme feels like a missed opportunity