[cody3222]

"People said they were 14.2% more likely to buy this chocolate bar when they were shown the version with a cost breakdown."

What people say they would do and what they actually do are 2 very different things.

The conversion rate of actual behavior needs to be tested for this to mean something useful.

[tdmckinlay]

Author of the summary here:

You are right regarding the chocolate bar experiment.

That's why the researchers ran multiple experiments, some of which measured actual behavior:

- People were 16.1% more likely to bid for a gift card for an Everlane backpack (vs a J.Crew one) when they saw cost information about it

- Sales of chicken noodle soup bowls ($4.95) in Harvard’s campus canteen increased 21.1% when costs were disclosed

[shoto_io]

Hi Thomas, thanks for your work. I am a fan!

Just a minor comment: I find using decimals places (like 16.1% and 21.1%) in human experiments pretty irritating. It feels like false precision.

After all, these experiments must have confidence intervals. If I had to guess, I’d assume at least a +/- 5 ppts variability in all these numbers.

What’s your view on that?

[tdmckinlay]

Thank you so much!

I agree that the figures can vary for many reasons and we shouldn't expect them to be exactly the same (some things we don't end up controlling for).

At the same time, if we take the experiment of the soup for example:

They measured 9,227 sales of it so the 21.1% increase is quite robust and I'd expect the error margin to be much lower than 5% either way - so in some ways the precision is warranted.

I also feel that if I were to round a 21.4% to 20% I'd be miscommunicating the findings of the research :)

[lupire]

You are confusing confidence intervals (used to say that you are confident the increase is positive at all) with error margin (the false precision in the 3rd significant figure).

[tdmckinlay]

You are right, my bad! Thanks for catching that in the comment above, I've updated it

[vntok]

That does not bode well for the rest of the methodology...

[boomboomsubban]

Mixing up terms in casual conversation happens to everyone, and is not a sign of a meaningful problem.

[johnsmith4739]

I don't know why people downvote you:

//> we finally have an absolute number of sales measured, but no way of knowing - representing all sales within a period or just cherry picked?

//> for the rest of the population, did it reduced sales?

//> was there any randomised test or not, because in the latter case there could be other biases we're unaware of

//> the increase is compared to what exactly?

//> any WHY is purely speculative as what was measured was WHAT people did. Internal motivation is in this case unproven, there is just a potential correlation.

//> confuses me to hell people taking about error margins and confidence intervals for something measured directly.

[rahimnathwani]

"if I were to round a 21.4% to 20%"

Right, but you'd round 21.4% to 21%, not to 20%.

[infogulch]

You're assuming that you can only round to the nearest 1% which is not true at all. If you round to the nearest confidence interval and the CI is 5% then you would round to 20%. That said, I would prefer both the precise number and CI communicated together like sibling comment mentions.

[pushrax]

Or ideally keep 21.4% and include the 95% CI. There’s nothing misleading about the extra precision when the CI is included.

[yowlingcat]

Agreed. Actually, I wish the norm was to communicate percentages with confidence intervals by default, because I feel like the tacit implication is colloquially "100% CI unless communicated otherwise."

[shoto_io]

Thanks for explaining! Your point about miscommunication makes a lot sense to me!

[bachmeier]

> I find using decimals places (like 16.1% and 21.1%) in human experiments pretty irritating. It feels like false precision.

Whether you say 21.1%, 21%, or 20%, you still have a single number. You could make an argument that decimal places like 21.147258% add clutter, but without an actual measure of uncertainty, all you're doing is reporting a summary of the data in the sample with different amounts of arbitrary rounding. That's not particularly helpful as a substitute for the full distribution.

[yuliyp]

It's not just a number. It's a string of characters conveying information about a measured value. The way it's specified conveys information about the number of significant figures (https://en.wikipedia.org/wiki/Significant_figures).

[phreeza]

Significant figures are not a universal standard, they are an imperfect way to in-band information about confidence, by sacrificing precision.

[shoto_io]

21.1% is implying a different confidence interval than 21%. At least to me.

[ceh123]

Yup, 21% and 21.0% do implicitly convey different information even though they’re the same number.

[djrogers]

> - Sales of chicken noodle soup bowls ($4.95) in Harvard’s campus canteen increased 21.1% when costs were disclosed

What sales decreased in relation to this? Would overall sales increase if all food items had costs listed? Or would people gravitate to the meals with the lowest margins, figuring them to be relative bargains?

It's really not simple to draw a conclusion from the limited data set here.

[CodeWriter23]

It's kind of lazy of you to disclose the Backpack and Chicken Soup cases in a declarative way. You present no references, no drill-down on whether all other effects were equal in these cases. For example, did chicken soup sales continue for an extended period of time and were not driven by other factors such as "a cold going around" or low temperatures? Trust is certainly a factor in consumer behavior, but it nowhere near the sole factor in decision-making that your 'summary' tries to present it as.

You're also lacking cases where the markup is high, 500-2000% is common among a wide range of products, from Fashion to SaaS.

Edit: I'd also add, in the case of food products, if all vendors adopted the transparency strategy, once consumers see typical margins in that industry are in the 5-10% range, suddenly that not-unreasonably-priced organic chocolate bar looks like a high margin item...

[johnsmith4739]

Sorry for the downvotes you get, it's appalling. The whole piece is superficial and riddled with inconsistencies. I guess we can take it as entertainment? It's what non-marketers think a growth marketer does, just put in text form. I'm not here to make friends, but if you state that you got 21% increase, you better show the work. In a world where everybody lies, you better have proof.

[renewiltord]

Honestly, I downvoted because most of the utility in these articles to me isn't the precise number, it's the idea. After all, I don't really care that chocolate bars work this way if I'm selling a SaaS product. I'm going to run the numbers myself.

It's the idea that this could work.

I want to encourage people to honestly communicate ideas to me and I want to discourage people who would discourage those first people.

I explicitly don't want to restrict only the highest-quality research. I want to permit some amount of scamming me.

[schnable]

Text form, with emojis!

[taneq]

> You're also lacking cases where the markup is high, 500-2000% is common among a wide range of products, from Fashion to SaaS.

The article does cover those cases explicitly:

> Extremely high profit margins (>55%) could trigger a negative reaction, although this was not tested.

Basically they didn't even bother because it's pretty clear that someone selling a commodity for a massive markup is not going to benefit from this approach. Of course, that doesn't mean this hypothesis isn't worth testing...

Disclosing your costs if you have a modest profit margin and also modest absolute costs is good signaling on multiple fronts (as long as you're credible):

- Higher parts/ingredients costs are a signal indicating good quality

- Low profit margins make customers feel like they're getting a good deal

- The appearance of transparency signals your own confidence in all aspects of your business

[dimitrios1]

It's also equally lazy for you to demand such an in depth and thorough argument and sources when you can pop into a search browser and do some fact checking yourself. No one owes you anything. This is a discussion board, not a dissertation defense.

[readflaggedcomm]

FWIW, the paper "Lifting the Veil: The Benefits of Cost Transparency" doesn't mention confounding factors like weather or temperature, only the possibility of revealing labor costs. "Fact checking" this study would require reproducing it, not searching the web.

[marcinzm]

>No one owes you anything.

and no one owes the author page views or their time either.

>This is a discussion board

Exactly, we're having a discussion on what the article is missing and why it matters.

[CodeWriter23]

I came up in a time where merit meant something. And this summary literally has no merit at all.

[robinj6]

I’ll bet the Harvard students saw a new fancy chicken bowl and bought it out of novelty.

[bbarn]

What about the "guilt effect" of showing that the high price is mostly because of the labor costs involved. Wouldn't that potentially dirty your experiment of just showing "cost" vs. involving a human element?

[Wassimo]

Chicken Nood Soup bowls for $4.95? Don't they usually sell for a buck at the grocery store? Unless it's a high end brand.

[smeyer]

Most of the cost was labor, according to the picture in the article. I don't know which dining hall or cafe the "canteen" they're referring to is, but much of the food sold on Harvard's campus is produced by unionized dining hall staff with decent pay, hours, and benefits, which affects the costs.

[klipt]

Personally I could see that information back firing, some places are very stingy with ingredients like chicken, and I'd feel doubly cheated if they did that after admitting it's only 5% of their total cost!

[istjohn]

This was in a cafeteria.

[edoceo]

At Harvard

[giu]

Good point! Sadly, I don't have access to the Harvard study (https://pubsonline.informs.org/doi/10.1287/mksc.2019.1200), so maybe someone with access to it could check it, but the linked article might be misleading in some aspects (depending on the results of the study shown in the paper).

From the study's abstract:

> A preregistered field experiment indicated that diners were 21.1% more likely to buy a bowl of chicken noodle soup when a sign revealing its ingredients also included the cafeteria’s costs to make it.

From the linked article's sub-title:

> Sales of a chicken noodle soup increased 21.1% when people were shown the costs of making it.

The study's abstract mentions that they were more likely to buy a bowl of chicken; it's not mentioned that they actually bought it.

[tuukkah]

Here's open access to the full study: http://ssrn.com/abstract=2498174

[tuukkah]

"21.1% more likely" is how articles phrase a 21.1% increase in observed frequency.

"Sales increased 21.1%" is equivalent as long as the unit price remained the same.

[vntok]

The article is very confusing on this front.

> People said they were 14.2% more likely to buy this chocolate bar when they were shown the version with a cost breakdown

Surely that cannot be correct. Possibly 14.2% of the people who were asked said that they would be "somewhat" more likely to buy X with more data stuck to its label (does any data improve sales? Did they A/B the label by adding random info?). This is very different from them acting upon it though.

[tuukkah]

Actually, the figure 14.2% does not even appear in the research article. The experiment was between two groups of random Mechanical Turk workers: one group was not shown the cost breakdown while the other group was. The workers answered how likely they were to buy the product on a scale from 1 to 7. If you calculate the average increase from one group to the other, sure enough the number you get is 14.2% but it is not a probability.

[yunohn]

Exactly this. Plus, I didn’t see “profit” mentioned anywhere in the costs, and the article also touches upon the (mostly obvious) fact that for high margin goods this would impact them negatively.

[tdmckinlay]

The effect was tested on profit margins as low as 17% and as high as 55%. Outside those ranges it's unclear if it will still work.

[yunohn]

I understand, so maybe my critique was a bit of my personal views.

Rather I meant that it’s a bit odd that costs are clearly specified, but the consumer has to calculate the profit themselves. In my mind, that actually makes it less transparent as high profit margins are what drive the crazy capitalist market we have today, and associating small values to material and labor doesn’t make me feel better when the profit is the “artificial” portion of my purchase price.

[mitchdoogle]

I think the assumption is whatever the difference in what the item costs vs the total listed on the package is the profit

[edent]

As it says in the article "Sales of chicken noodle soup bowls ($4.95) in Harvard’s campus canteen increased 21.1%"

[tyingq]

Interesting. I'm curious how they got to $3.23 of labor for 16 ounces. And I suppose that drops as they sell more, assuming it's made in large batches.

[lupire]

There's no reason to believe the costs are accurate at all. Obviously a business would lie if they applied this theory to their pricing. The experiment was based on the presence and magnitude of numbers in the advertising.

[chiefofgxbxl]

Is there an official name for this sort of effect? I've been looking for something to call this. For example, we've had this effect at work when trying to organize social outings; people say they're interested in some activity X (e.g. hiking, board games, etc.), but then the day of that event comes and nobody shows up. The bar is very low to say you're interested in something, but doing actually takes effort.

[johnsmith4739]

Social Conformity - it would be costly for the participants to say no, from a social standpoint, but when the wallets have to open, there is always an excuse.

[benjohnson]

We fight this by giving everybody a token bit of public labor - tell someone they need to bring cookies, another needs to bring party-hats and someone else to bring punch and you'll find that they don't want to let others down.

[porb121]

expressed vs revealed preference

[AlwaysRock]

Sure. You could call pretty much all research into question using this argument. A huge amount of research is done based on questionnaires.

[elliekelly]

It’s been a while since I worked retail but unless something has drastically changed most people don’t read product packaging very closely. I would constantly have people walk up to me with an item and ask a question that was not only clearly answered on the package, but the information was _highlighted_ for them. I just don’t think most people would even notice the cost breakdown let alone read it and think about what the information means.

[gpm]

You're sure it's most people? The people coming to ask you questions are a self selected set...

[darkerside]

People have been ignoring the highlighted bullshit on the package since long before they were pop-up ads on the internet