[wsb_mod2]

The profit motive is being blamed here for optimizing for engagement.

But how do you assess progress when you remove all desire to monetize?

I run r/WallStreetBets and "quality" is an extremely nebulous term.

I look for things like "novelty", "thought-provoking / well reasoned commentary", "original content", "authenticity", "self-awareness" or "primary research". But these are human assessed metrics.

Some more easily measurable metrics might include, "length of submissions", or "number of outbound links excluding blacklisted domains". Or even "number of tickers or quality-correlated keywords mentioned".

All these metrics have very clear downsides, and if generally well-known, become useless. Interestingly, a score too high can also result in something being unlikely to be authentic.

Another challenge is your relationship with users. Surprisingly, moderators are not innately adversarial to users, they can also promote content through other channels (discord, twitter) or sticky threads for a viewership boost.

So, even without a profit motive, what do you do?

[JohnFen]

> But how do you assess progress when you remove all desire to monetize?

If you ignore the profit motive, presumably you are doing it for other reasons. You assess "progress" based on those motives. Or perhaps you don't assess "progress" at all, beyond "I like what I'm doing here and where this is going".

I've run numerous internet services over the years without the goal of them generating an income at all. I've run MUDs, websites, discussion groups, etc. I didn't objectively measure anything about any of them, because my goals were not ones that could be objectively measured. And honestly, even if there were some metric that could be used, I would have avoided using it because then it becomes about maximizing the metric rather than the purpose I started the activity in the first place. If I was happy with how they were going, that counted as "success" to me.

> So, even without a profit motive, what do you do?

I honestly don't understand this question. Without a profit motive, you do it for other motives. If you have no other motives, then why are you doing it at all?

[the_snooze]

>because my goals were not ones that could be objectively measured

This is something that technically-minded people can take to heart more. There's a lot of value in things that can't be objectively measured. For instance, I'm in a bunch of group chats with various friends. I don't always interact with it (i.e., my "engagement" is fairly low), but I derive a lot of enjoyment from those. Why? Because I like those people. Can I put a number on how much I like those people? No, because I'm not a socially stunted robot.

[tialaramex]

> There's a lot of value in things that can't be objectively measured

Or maybe there isn't much value, since apparently you wouldn't be able to tell. Except, you can tell, and so actually they can be measured. You don't want to measure them, but that's not the same thing.

> Can I put a number on how much I like those people? No, because I'm not a socially stunted robot.

You don't want to put a number on it. That's fine. But can it be done? Of course it can. In fact even if we insist on trying never to think about it that way, we make decisions based on how much we like people all the time. I've never measured how far it is from where I live to the supermarket. Why would I? But it would be silly to claim I can't put a number on it, I just chose not to.

[atoav]

The point is that putting a number on it might not capture the meaning and the context humans put into it. There is so many ways to do this wrong, that sometimes it is just better to have someone who loves the topic and tells you: "yep it is going good", if it is indeed going good.

[mhall119]

So many things are qualitative, but not quantitative. They can be measured, but not counted. Happiness is one of those, you can say that you are "more happy" or "less happy" from one day to the next, but you can't count happiness, you can't put a number on it.

[nonameiguess]

I don't think you're taking this the way it was intended. The person who posted is a moderator who does not have a profit motive. Reddit doesn't distribute profit to subreddit moderators (as far as I know?). But even without a profit motive, he is still left without any simple way to evaluate the quality of submissions and decide what should be promoted and what should be killed. The point being "engagement" is often used not because of profit motive, but even in the absence of profit motive, it is a good default metric that is easy to measure in trying to figure out what your community actually wants to see.

He's not asking what motives you should have if you don't have a profit motive. He's asking, even in the absence of a profit motive, what better measure of content quality is there than how much the community engages with it? You gave an answer. In communities you ran, you just asked yourself if you were happy with them. That's an acceptable answer. But assuming someone who runs a community wants the community to be happy as well, that is a lot harder to assess, as you no longer have direct access to their mental states the way you have to your own. Now you're left again with the need to estimate their happiness via some proxy measure. If not engagement, what should that be?

I'm not saying there is no answer and all communities should throw up their hands and either just use engagement or use no measure at all, but I do think it is legitimately a hard question.

[scythe]

>I didn't objectively measure anything about any of them, because my goals were not ones that could be objectively measured. And honestly, even if there were some metric that could be used, I would have avoided using it because then it becomes about maximizing the metric rather than the purpose I started the activity in the first place.

There is a middle ground here. Choose a basket of metrics, impose the natural partial ordering, and intuit your way through pushing up one or the other. If this sounds crazy, keep in mind it's roughly how the Fed (inflation + unemployment) runs the monetary system.

[empathy_m]

Historically, the immortals on MUDs used to optimize for thing like "fun" and "clout" - how much do you enjoy interacting with the players and being seen as the person in charge of their game? How much do you enjoy telling people you volunteer your time to work on it?

If WSB members were doing bad stuff like, making mean jokes about the leadership of companies Ryan Cohen divests from in; or coordinating real-life harassment of family members of employees of the Depository Trust Company or something; you'd probably want to put a stop to that and focus them somewhere else. Because it wouldn't be fun.

If Fox News interviewed you and you sounded silly and all the community members got mad at you, you'd probably also quit the volunteer job.

But as long as you make choices that make the community a fun and engaging place to hang out and feel like you're part of a big secret treasure hunt, that seems awesome, and you would probably want to make choices that maximize that.

TBH most MUDs were not data-driven and kinda govern by feeling -- they'd sort of add steering they thought was interesting and keep it in place unless there's a backlash. The tedious administrative stuff (bans, moderation, player requests for item reimbursement) would always have a backlog and you'd recruit junior imms to help out and they would feel like they were part of the fun too.

[JAA1337]

Just want to give you a shout out for using the acronym MUD. I havnt seen it used in like 25 years :). Thank you!

[smeyer]

What does it stand for?

[yoyohello13]

Multi-user Dungeon: https://en.wikipedia.org/wiki/MUD

[solardev]

Multi user dungeon, basically like early MMORPGs, usually text based. MMORPG are massively multiplayer online role-playing games, like World of Warcraft.

[jrochkind1]

> But these are human assessed metrics.

OK, and? I could guess what the problem with this is, but let's spell it out... maybe that they are subjective so different assessors can disagree... and this is a problem why?

What if there are some cases where it's actually important to use human-assessed metrics as one component, or even as the entire thing, cases where no appropriate 'objective' metrics are available? (In scare quotes, because these quantitative metrics are seldom _quite_ as 'objective' or independent from human judgement as assumed. There is usually human judgement involved in how the metrics are defined and measured, where different people might define and measure a metric different ways resulting in different numbers...)

[zinglersen]

There are definitely some cases where it's important to use human-assessed metrics, like cooking a meal.

But if you are the size of Facebook, Instagram, Tiktok, etc. it is too risky (and expensive) to let individuals define what success looks like - you need something measurable that you can spread to multiple projects or even across the complete company culture.

I fully agree with your point that metrics are typically not objective, like any data once it's used to communicate something.

So the question should be what metric(s), if not engagement in itself, should we measure for to create a healthy online community?

I wonder if it's less about metrics and more about principles; how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control? The UI/UX of the site? Something else?

[jrochkind1]

> how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control?

My guess is that dang's moderation is a large part of it, and I think dang moderates (creates moderation policy and executes it) based on things that are "human-assessed" and not quantifiable....

> But if you are the size of Facebook, Instagram, Tiktok, etc. it is too risky (and expensive) to let individuals define what success looks like

You're probably right, but I think it's worth challenging this conventional wisdom. (Not necessarily here, we're not going to work it out, but still, I'll ask some rhetorical questions) Why is it too dangerous? Even if you have quantitative metrics, are you _sure_ this isn't still "letting individuals define what success looks like", since individuals decided how the metrics were defined and measured? So then, what's the difference? Why, specifically, is it too "dangerous" to have non-quantitative metrics defining what success looks like? Danger of... what? I would say facebook as it is, is actually incredibly dangerous to, like, human society. So... what kind of danger are we talking about? Danger to facebook's profits instead? Or what?

> So the question should be what metric(s), if not engagement in itself, should we measure for to create a healthy online community?

Sure. I don't know! I think that's the question the OP is meaning to ask too, if not completely answer. The OP suggests:

> At the start of this, I said that people join your community for support, connection, opportunities to give back, and meaningful relationships. Those are the things they value, and those are the things you should value. And if you value them, you should measure them.

> They’re not as easy to measure as engagement, sure, but they can be measured. The best part is that maximizing these metrics is always going to be good for your community.

My point is that i'm not certain this should be short-circuited with "Well of course whatever metrics these are, they need to be quantitative and have the appareance of "objectivity".

[bombcar]

> I wonder if it's less about metrics and more about principles; how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control? The UI/UX of the site? Something else?

HN is successful because it can be moderated by one dang. Maybe a bonus helper can appear.

But once you get so big that your dang can't do it all anymore, it will eventually fall apart.

[marginalia_nu]

Why does it need to be measurable?

[kelseyfrog]

HN in general isn't aware of operationalization[1]. It's an interesting quirk of computer science being a STEM field, but CS research is qualitatively different from physics, sociology, psychology, biology, zoology, and ag science. Rather, the typical CS student has a pre-theoretical conceptualization of quantification and data collection as it pertains to research methodology.

Instead of relying on operationaluzation, the phenomena is considered strictly immeasurable. It leaves a lot on the table. In the examples above, for instance, there are a number of unsubstantiated assumptions as well as pros and cons that implicitly do not account for operationalization and it shows.

1. https://en.m.wikipedia.org/wiki/Operationalization

[jholman]

Off-topic, but learning about classic examples of operationalization is when I realized that mostly psychology isn't a real science (yet). I believe the canonical example involved me learning the phrase "bobo doll".

[kelseyfrog]

How interesting, same with operationalization in physics too?

[marginalia_nu]

This still doesn't answer my question of why.

[runnerup]

Measuring community engagement makes sense when your primary mission / core goal is to build a strong community. When I was building a large online third-party Discord community for a university during COVID, I was very glad that Discord exposed a lot of metrics for admins/mods to track. It's obviously important to not "game" the metrics by falling into short-term gain traps like over-pinging users, or encouraging high-volume but low-quality participation, etc. But as long as you are laser-focused on "providing real value" to members, then long-term or multi-cyclical trends of community engagement is a useful benchmark.

But it doesn't make sense to focus on these metrics if your mission is to create a great product -- focusing on community engagement should be just one small aspect of your overall marketing strategy, which should be a relatively constrained part of your overall budget (both in labor and money). And even within marketing, there are many other important metrics besides community engagement.

Soylent had great community engagement but not enough focus on scientifically-guided product engineering. Tons of businesses that get this wrong.

Generally I interpret overcommitment to community engagement as a signal that the founders are overly narcissistic and usually apply a discount the expected future value/evolution of their product.

[JohnFen]

> Measuring community engagement makes sense when your primary mission / core goal is to build a strong community.

But measuring community engagement doesn't tell you anything about the nature of the community. Only that it "engages" people. If your goal is to build a strong community, isn't it more important that the community be a good one?

[MartinCron]

Lynch mobs are strong and engaged communities, after all.

[wussboy]

Strong does not mean good. Those are two separate goals. The most important developments of the next 100 years will be in determining ways to make strong and good communities that scale.

[runnerup]

The most important things (quality of community) are not generally quantifiable and proper “measurement” depends on the leadership ability of those who choose to foster the community.

[AndrewKemendo]

So, even without a profit motive, what do you do?

I think there are two options here:

1. If the community that you have does a "foom" and grows like crazy BEFORE it has a set of values, then you're at the whims of the mob and where certain vocal and rallying members decide they want to take the community. Usually this ends in internecine conflict and the inevitable "break offs" into other smaller groups

2. If your community starts with a value vector then you structure, nudge and contain the community to a MDP (Markov Decision Process) with the value vector as the reward, for which the moderators are the critics and the community members are the actors in the actor-critic model.

It seems like the vast majority of organizations/movements etc... fall into category 1

For me the learning here is, if you are a leader in your community it's never too early to nail down what the core value and benefit of your community is and stay obsessed with just that.

[hammock]

Hard to scale, but farming out content samples to a panel of your target market for them to rate on the attributes of "novelty", "thought-provoking / well reasoned commentary", "original content", "authenticity", "self-awareness" or "primary research" might be an approach

[wolfofcrypto]

Mark Cuban recently launched a community at https://biztoc.com as sort of a WSB for business people. I wonder if that will/could work without an umbrella topic.

[RealityVoid]

It seems to be a news aggregator rather than a community.

[tstrimple]

> All these metrics have very clear downsides, and if generally well-known, become useless.

Good ol' Goodhart.

"When a measure becomes a target, it ceases to be a good measure."

[civilized]

> without a profit motive, what do you do?

We could pose the question even more generally. Apart from making money, what is the value of a human life or community?

[groby_b]

> So, even without a profit motive, what do you do?

You acknowledge that human interaction and metrics are not compatible, and apply actual judgment. You're a human being, not a paperclip optimizer.

Your goal isn't to stripmine anything of value out of a community, but to provide a vibrant place people like coming to. So, for feedback, you ask them what they like, what they miss, what they wish for.

Qualitative assessments instead of quantitative ones. Lean into the power of the humanities.