[sayhar]

Oh hi! That’s me! (And a bunch of other folks). Happy to answer any questions.

I also really like this founders note that we wrote. It’s in our own words, etc: https://integrityinstitute.org/founders-letter

We are recruiting new members! If you work on integrity / trust and safety / antispam / content quality / etc, let’s talk.

[dogman144]

Many PMs and engineers and laymen knew FB was a rotten product for a long time. I also assume a lot of people at this institute didn’t take the Alex Stamos (ex, short term FB CISO) U-turn either. So…

How is this effort not a virtue signaling by people that made their fortune on the back of this generation’s cigarettes, and now hoping to get traction on being listened to for fixing the mess they created?

[shug2k]

Integrity folks across the industry work on solving these problems, not creating them. It is more of a balance of whether to tackle the problems inside the organization, or outside - depending on which is more effective.

What is not in question is that we should be spending time on tackling these problems.

[hn_throwaway_99]

> Integrity folks across the industry work on solving these problems, not creating them.

Solving these problems is pretty trivially simple. Get rid of the news feed, or perhaps make it just friends, reverse chronological order. All of these "integrity teams" only exist because Facebook isn't content just making boatloads of money, it needs to make 10x boatloads of money by getting people to scroll infinitely.

People try to pretend this is some hard, difficult problem, when it's only a problem because Facebook is just as addicted to their users' mindless scrolling as their users are.

[SrslyJosh]

> It is more of a balance of whether to tackle the problems inside the organization, or outside - depending on which is more effective.

I think you answered your own question already.

[dogman144]

Of course.

Regarding internal solutions: it’s impossible to overlook the significantly misaligned incentives facing integrity teams. They try to solve problems inherently created by the profit model of the company and what also pays their salary. The parallels to the internal scientist teams at Phillip Morris are uncanny.

As I said, this paradox, and the company kicking the can down the road, has been obvious for years externally and also seemed internally per the Slack (or w/e) leaks post 1/6.

So, for these teams, if they’re staying longer than a quick in and out once realities trumped ones idealism at “going where the problem is” (look, I really do get that impulse), continuing to claim the moral high ground in the way this Institute is so tone deaf. Sorry buddy - I’m on levels.FYI as well, we all know what you got paid to (increasingly ineffectually) support this product. Like there’s been leaks for years of PMs discussing Myanmar genocide inflammation via FB/WA for a few years now. Was that not enough to leave?

This is inescapably how a lot of tech is going to judge this period/company. It went on for too long to claim ignorance otherwise.

Edit: You know what, no it's not at all about an external vs. internal balance, that very much misses the point. It's likely that a meaningful change which preserves what FB (and related) intrinsically is will have to come from an internally-driven fix. The technical abstractions are just too much of a problem for outsiders to grok, then suggest a fix for, and etc. etc. ("Senator, we sell ads...").

It's about understanding where the moral and character capital that leadership, especially transformational leadership, comes from, and understanding how the founders of this have none of it. People that have stayed at FB and profited immensely from the experience, integrity team or not, stayed silent about it, and then start coming out with this publicly now vs. years ago (see: Alex Stamos' example), don't have that capital. And it's so distasteful to see them think that they do via efforts like this. That's the problem.

[shug2k]

While at the company, we were busy trying to solve problems during a critical period. The criticism has been fierce for Facebook - but make no mistake, our teams made enormous difference within the company, and the world would look very different than it does today without the work of integrity workers within the company. This continues to be true to this day.

Coming out publicly over the years has done very little. Even now, with all the attention, it is questionable what will actually change. We are much less interested in virtue signaling, taking the high ground, etc. than working with folks who are interested to carve a path forward. We are dedicating our lives to this work to find solutions over the long-term, whether it is in the spotlight, or not.

[dogman144]

Someone else in this thread commented it well - “this is the problem talking to FB engineers.”

I’m not sure how you can read that founders letter, and not see a spotlight grab/virtue signals by people who contributed to and profited from the problem they’re trying now take a leading role in solving. It’s like there is just total refusal to be seen as part of the problem. Blows my mind.

“Coming out publicly has done very little” attitude sort of says all you need to know. Pretty sure the article said also ~”now that Frances came out publicly, we can start this!” Mental gymnastics.

[dogman144]

Correct, the Integrity Institute's cofounder:

"Frances is exposing a lot of the knobs in the machine that is a modern social media platform," Massachi said. "Now that people know that those knobs exist, we can start having a conversation about what is the science of how these work, what these knobs do and why you would want to turn them in which direction."

It seems public discussion does help, and is even a critical input to this effort. It's just better when others do it first.

[homo_ergaster]

>now hoping to get traction

Not even just that, they also want us to donate to cover the costs for the good work they’re surely going to be doing

[rajin444]

Are there any examples of sticking to your principles by standing up for socially and morally reprehensible groups? My initial impression is this is just another group pushing left leaning elite American values as "integrity".

Things like this: https://www.aclu.org/other/aclu-history-taking-stand-free-sp... go a long way.

[zestyping]

I have a hypothesis. It goes like this.

Today all the major text-based social platforms work in about the same way: we type text into an empty little box that's threaded under another box. There are variations in ranking and flagging, but the basic mechanism is unchanged.

It is striking to me that, in all these years, we have explored only a tiny little corner of the design space. There are no mechanisms to lower the temperature when arguments get intense, for example. Nothing to help keep us from misconstruing comments out of context. Nothing to assist us in feeling compassion for the people we're talking to, or understanding their intent as they mean it to be understood. And so on. It's almost as though we sold millions of cars without brakes, everyone is crashing into things and hurting each other, and our response as a society is to throw up our hands and say "Welp, guess humans are just too stupid to drive cars safely."

Many people have suggested eliminating engagement as a metric and going back to purely chronological feeds. That sounds pretty reasonable given where "engagement" has gotten us so far.

But what if there were such a thing as "healthy engagement"?

The hypothesis is that healthy engagement is achievable. I don't know whether it is, but there's a huge range of design possibilities that we have yet to explore.

Do you know of anyone working on healthy engagement? Is it something you want to work on, or do you have any recommendations on starting an effort in this direction?

[rhizome]

>Do you know of anyone working on healthy engagement?

Wikipedia is one, by not "working on" it.

>It's almost as though we sold millions of cars without brakes, everyone is crashing into things and hurting each other, and our response as a society is to throw up our hands and say "Welp, guess humans are just too stupid to drive cars safely."

No, they have brakes, it's just that half of the car owners are punching people who work at Jiffy Lube and screaming that brakers should be killed.

>But what if there were such a thing as "healthy engagement"?

"Engagement" can not ever be healthy because "working on" it relies on manipulation. Curiosity implies an absence of manipulation and a maximum of arbitrary connections.

You want healthy engagement? Create something well-loved that doesn't depend on or require any additional action from the appreciator for them to experience its complete impact.

Oh shit, that's hard! "Well then how about I scare them into clicking on a link that pays me money when they do so, then use an image and/or words to call them inadequate in some aspect of their lives so that they send me more money?" Tomato-tomahto?

[sayhar]

Hey this is a really good question. And I agree with you 100%! We have only explored a tiny corner of the design space.

Since you asked for links, I think you'd like this talk I gave at Berkman a little while ago: https://cyber.harvard.edu/events/governing-social-media-city

It's about: if we think of social media as a new city, what are the alternatives to hiring tons of cops/censors? What about urban planning, bike lanes, etc?

As for healthy engagement: I think there are a few people in this space. I honestly don't know as many as I'd like. This will be a learning experience for everyone. I think New Public might be doing good work, but I'm not sure!

https://newpublic.org/

Hope that helps!

[SrslyJosh]

> There are no mechanisms to lower the temperature when arguments get intense, for example.

This is a second, third, or fourth-tier concern after stuff like white supremacists using platforms for organizing/harassment, governments using them to facilitate genocide, scammers using them to profit off of the pandemic, etc., etc.

"When arguments get intense" completely ignores the incredibly low-hanging fruit: banning networks of bad actors. (They have the data to do this, they just don't want to.)

> Do you know of anyone working on healthy engagement?

Twitter is trying to warn users about "intense" conversations. Unsurprisingly, they suck at it:

https://twitter.com/angryblacklady/status/144754672404668416... https://twitter.com/chadloder/status/1446879163307028481

[Fnoord]

> Twitter is trying to warn users about "intense" conversations.

Discourse, too. It gives a statistic that X% of the posts are yours, and asks you to consider letting other's voice their opinion. It says that after you typed your post. I'm not going to not post, then, but I admit that yes it does remind me to consider spending less time on the platform in general (!).

HN has it, too. Deep threads hide the reply button if posts are made in quick succession.

The orig. comment claimed:

> [...] There are no mechanisms to lower the temperature when arguments get intense, for example.

Addressed above.

> Nothing to help keep us from misconstruing comments out of context.

I don't agree with this. Factchecking occurs, thumb-up/heart on factual content helps. Also, quoting, logic, and linking sources allows to dispute (such) fallacies. These tools are available.

> Nothing to assist us in feeling compassion for the people we're talking to, or understanding their intent as they mean it to be understood. [...]

Bingo.

Because polarized people don't want to. They want to 'win' a discussion instead of learn from it.

[neural_thing]

It's understandable that people go to work for Sauron to get paid big money. But starting a "human-orc ethics think tank" because you have relevant experience in the field is a bit rich.

[labster]

We need to be able to forgive those who have done evil, if they truly repent. If they’re trying to do good now, we shouldn’t hold it against them that they worked for a monopoly spreading lies that incited violence, created online addicts, and took the pieces of silver happily. No, it only matters do good now.

[hn_throwaway_99]

I wholeheartedly agree. But, to be frank, I really don't see any amount of contrition in the Founders' Letter here, https://integrityinstitute.org/founders-letter. Primarily, I don't see anything here that says "You know what, we're sorry, because we were part of the problem, despite working on 'the inside' trying to improve things." I'm not asking for the founders to fall on a proverbial sword, but I don't see any acknowledgement of how the entire Facebook "machine", which causes all the problems they point out in their letter, is only possible because legions of really smart people choose to work there.

[dogman144]

We can forgive them but not listen to their thought leadership on solving the post-Sauron world.

[smsm42]

Repenting is indeed a major step on the way from evil to not evil, but I do not see any sign of repentance in their materials. Repenting means recognizing you did wrong, recognizing and admitting, sincerely and honestly, what was wrong in what you did, and then building a corrective action on top of that. I do not see any of that. I only see a bunch of platitides, which I could find in any corporate "our values" binder, we're all for everything good and against everything bad. "Strive to be a good, honorable person" - ok, sure, who doesn't? Are there many people striving to be bad and dishonorable? That's not doing anything.

[LurkingPenguin]

It's also a bit rich to tell people "from the start, the Institute has been funded out-of-pocket by its founders. But that can’t last forever." -- even though you were only incorporated a month ago on September 29 -- and start asking for donations before you have received approval for tax-exempt status[1].

[1] https://integrityinstitute.org/donate

[sayhar]

Jeff and I have been working on this nonstop since January-February. Formal incorporation happened way way after the work started. This has been my full time (unpaid, volunteer, only) job since February, and I have burned down much of my savings in that time.

[smsm42]

How do you ensure that "integrity" does not morph into "ban everything our groupthink says is wrong"? How do you ensure you're not coopted into an ideological or partisan propaganda/speech control efforts - a trap that so many "fact checkers" gladly fallen into?

[sayhar]

One thing we focus on is not really looking at content at all, but instead, behavior. Mostly -- spam.

One way to see what we're doing is glorified spamfighting. Except the spam sometimes isn't fake ray-bans, but is instead doctored videos that call for lynchings in India.

Here's more on the subject. (My ideas, not the full institute): https://cyber.harvard.edu/events/governing-social-media-city

[wolfram74]

One content agnostic approach to improving signal to noise is the tendency for deep chain sharing to be mostly false/misleading content. Introducing a forced copy-paste hurdle after the chain gets, say, 4 deep, would be content agnostic, but probably improve the quality of what get's spread.

[smsm42]

> the tendency for deep chain sharing to be mostly false/misleading content

Does it rely on some specific research? I mean I'm sure there's a lot of viral stories that are false. But there's also a lot of viral stories that are true - or at least no less true than what you'd commonly see on network TV or read in major newspapers, like NYT. I am not sure why it'd be obvious that something that is shared a lot is necessarily false.

[sayhar]

This is a great example of the kind of stuff we want to be talking about and looking at. Design changes that apply across the board.

You could imagine us swapping notes, and doing some research as a group, to see if this is indeed an idea that would work. And then spreading that information to everyone, including the platforms themselves.

[shug2k]

Transparency goes a long way towards addressing this concern. This is one area we are focused on - asking companies to be more transparent, both in their overall metrics and samples of public content, so we can have those debates as a society rather than behind closed doors.

[smsm42]

Transparency is not enough. If the decision is "we ban everything that the Troika we appointed finds not to its liking" - it's transparent, but not helpful and has nothing to do with "integrity". Transparency is good, but not enough.

[btown]

What are the institute's thoughts, if you've formed them, on end-to-end encryption, especially as it applies to social media where the line between group and group-text blurs? I feel it's an incredibly nuanced topic that's become incredibly polarizing in recent days with some of Haugen's comments.

On the one hand, in favor of E2EE, companies can and will use the content of messages, if they have access to them, to micro-target suggested content to users, and this can lead to increased levels of misinformation being promoted to people who have engaged with misinformation. And of course there's the government surveillance angle, which is an entirely separate story!

But if you remove the signals in that content by encrypting in a way that is opaque to the platform, do you substantially reduce the ability to microtarget? Very possibly not, given the amount of graph data the social media company has anyways about group members independent from the content itself. And encryption gives the social media company the ability to wash its hands of any responsibility or awareness of content.

Assuming it were easy to technically achieve (which is a huge leap, to be fair!) do you think it better serves the definition of integrity you've adopted, that a social media platform have the majority of its content end-to-end encrypted, or not?

[sayhar]

Hi! Thanks for the constructive question. I agree with you that it's nuanced, and also I'm sad that it's getting polarized/simplified in some venues.

I don't think we have gotten an institute stance on very much relating to E2EE. Our community advisory board (composed of integrity workers) is the moral core of the organization. So far, when we say "the institute has a stance on X", that has meant "the advisory board signs off on X, and that X represents a good faith consensus view of workers in the industry".

We don't yet has a doc that lays out why we think integrity and privacy can coexist nicely. Speaking only for myself, I think the answer lies in careful design. As an example, you could see, via research and experimentation on FB Messenger, that messages that are forwarded in chains of > N are just empirically overwhelmingly likely to be bad faith, spammy, etc. You could then take that finding to WhatsApp, Signal, etc, and then bake in changes to the UX that make it slightly more annoying to forward messages if they've been on a reshare chain of N/M. That kind of stuff.

There's also some consideration to group size -- if a group is 5000 people large on, say, Telegram, it might be encrypted, but it's no longer really private. Maybe it should be treated differently? Unclear, let's think and research about it.

I think a rough consensus we might move towards is treating messaging differently than broadcast, and also treating broadcast features inside of messaging apps differently than straight up messaging themselves.

But again, those are just some of my more idle thoughts. There are members and fellows who are better experts on this particular subject than I am.

Does that make sense? Is that helpful?

[btown]

Absolutely, and thanks so much!

To the point about research and experimentation, it wouldn't be surprising to me if certain high-level people at various platforms are having confidential discussions about "are there technical means to prevent message-level research and experimentation from being possible in the first place."

And it's vital not only that the public/media recognize when there are conflicts of interest at play, but also that well-meaning employees at various platforms have the ability to see "here are the bright lines that a consensus of your peers across the industry believe shouldn't be crossed, and here are constructive talking points that you can use for internal advocacy if you are in a position to 'nudge the path' towards a sustainable way for integrity, privacy, and business/legal priorities to coexist."

It's really, really heartening to see people working through these tough questions and working towards a brighter future. You're doing incredibly necessary work.

[sayhar]

I appreciate you. Thank you.

[piva00]

Not working on integrity but really, really interested in seeing this coming alive. I've had a very deep curiosity about this topic for a couple of years, allegedly Facebook's products might have influenced the elections on my home country (Brazil), which has very directly impacted the quality of life of my family still living there.

Looking forward to see what comes out of this and wishing you and the team all the best luck, thank you.

[ChrisMarshallNY]

I am not working on integrity (I have a feeling that the integrity offices of many companies have brooms and buckets in them), but I do write software that Serves a constituency that has a very vested interest in the matter, and wish you well.

I also have a personal code of ethics, and hold myself to a very high standard of Personal Integrity.

In my experience, talking about that in the tech community does not end well.

Ethics and Integrity do not seem to be popular topics for discussion in SV.

[kwertyoowiyop]

Corporate departments you do NOT want to be in: Innovation, Ethics, Trust… unless of course it’s corporate doublespeak and the department is doing the exact opposite of what its name implies.

[Nevermark]

Exactly. If "Innovation, Ethics, Trust" are being addressed by a department:

1. They have no profit and loss, so are going to get sidelined by the higher impact (in any short or medium term) concerns of every department that does.

2. They have no direct control over serious corporate activities, so are going to be considered a distraction at best, interference at worst, by others under their own pressures. If they are noticed at all.

It isn't an accident when universal values are silo'd into departments from the rest of the company. It means leadership doesn't want the rest of the company to waste time or focus on them. But feel like the values need token recognition.

--

It gets even worse if these were governmental departments.

Department of Innovation: enforces acceptable methods and subjects of innovation, i.e. anti-innovation.

Department of Ethics: Reinforces the governments views by justifying them as ethical, and demands lip service to those fictional high ethics.

Department of Trust: You must trust us.

Worst one of all: Department of Truth. Here it is. Don't look elsewhere.

[sayhar]

Thank you! I helped set up the Brazil Election War Room -- the first election war room inside of FB. It was intense! There is a special place in my heart for your country <3

[malandrew]

This sounds like election interference by a foreign power. What business do employees in a US company have influencing the elections of a foreign country?

[blitzar]

More or less intense than the Myanmar Genocide War Room? Was it the first genocide war room at facebook or were there ones before it?

[gwittel]

(1) Thank you so much for doing this. Facebook has been after me for years (> 15 years in anti-spam/email abuse). I keep putting it off, but its time for a clear "no thanks". As an outsider saying "no" - is there anything I can convey to them to drive the message home?

(2) Minor - your "Join Us" link from the Founders Letter page is 404 -- https://integrityinstitute.org/join-us

[sayhar]

Thank you!

(1) - I'm not sure. I don't think I'm an expert here. But if you've got 15 years of anti-spam experience, we'd love to have you join us as a member :-)

(2) - Thank you. Fixed!

[tmule]

What’s the agenda here? (Fit the description, but am very skeptical of your motives :))

[sayhar]

I think this should answer your question: https://integrityinstitute.org/founders-letter

But also, our values: https://integrityinstitute.org/our-values

Also, succinctly -- this is a real, grassroots thing. We gathered a bunch of friends and coworkers for this big idea of "what if we had best practices and a professional association for integrity work, just like we do for cybersecurity" and then worked for 10 months to do it.

We've tried hard to put all kinds of pieces into place. We're making it a place that is both independent of companies and also safe for current employees of those companies to join.

It's also cool that we can draw on this community to give expert advice to stakeholders (policymakers, journalists, companies, academics, etc). Mad that congress doesn't understand how Instagram works, or whatever? We can explain things -- as people whose training was in looking at the total information ecosystem of a platform.

A goal is to have integrity work be at least as prestigious, valued, and essential as cybersecurity, or software engineering is. A thing where quality matters, and if you do shoddy work you will be called out on it.

Does that help?

[tmule]

Thanks! The values resonate strongly (rarely practiced in industry though - power, the pursuit of glory, and partisanship can and do form a toxic combination).

[malandrew]

a statement of values is meaningless. What organizational procedures do you have in place to prevent some faction from hijacking the organization and morphing it into an ideological echo chamber. The values on that page are as mutable as the html they are written in.

For example, in the past 10 to 20 years or so we've seen both the ADL and ACLU morph into institutions that would be unrecognizable to ADL or ACLU staffers from 10 to 20 years ago. No one ever imagined that they would abandoned their classically liberal values and replace those values with the illiberal "liberal" values they practice today.

[wutbrodo]

Was the ADL ever classically liberal? I don't mean that snarkily. I just don't recall much of a shift in their values in the way that's clearly visible for the ACLU and others.

[malandrew]

Probably not fully classically liberal like the ACLU once was but certainly far more classically liberal than they are today (which is pretty much not at all).

[beebmam]

The ACLU was founded by Hellen Keller, who was a lifelong socialist, among others.

[wutbrodo]

I don't think I follow your point. Are you suggesting that the ACLU was also never classically liberal because the one of the dozen+ members of the founding committee was a socialist?

[commandlinefan]

> morphing it into an ideological echo chamber

Assuming that wasn’t the point from the very beginning.

[spywaregorilla]

How do you define integrity in this context. (you, personally)

[sayhar]

This is a great question! I'm still trying to find a top-down definition instead of a "I know it when I see it" one.

To me it's probably something like this: we can think about an information ecosystem or social platform as a system. "Normal" hacking of the system happens through finding loopholes code. (That's cybersecurity). "Integrity-related" hacking of the system happens through finding loopholes in design and rules.

For some easy examples, that covers things like realizing you can post to 1000 groups in an hour. Or using sockpuppets to give artificial boosts to posts. The attackers aren't hacking code, but are hacking a system of rules, norms, and defaults on that system. (And often, finding the holes between where one part of the system was soldered onto the other).

That's the technical part. Integrity also has a sort of ethical component. I think that's meaningful too.

[dogman144]

I know I'm hammering you in another thread, but will press pause for a second.

What you're basically describing as "integrity hacking" is already around as a discipline "social-cybersecurity." If not the same thing, it's likely a very close peer discipline.

Also, coopting a word like 'integrity' vs. 'platform integrity' or 'cyberspace community integrity' or w/e is a tough call. By doing that, you end up with PR-destruction sentences like "integrity also has a sort of ethical component," which at face value is quite a read w/o the context of how "integrity" is being re-defined here.

https://sites.google.com/view/social-cybersec// https://link.springer.com/article/10.1007/s10588-020-09322-9 https://socialcybersecurity.org/

[sayhar]

dogman, thank you for the heads up about this social-cybersecurity group. It does seem related, and I'll be sure to check them out. Lately, someone has also mentioned m3aawg.org, which also seems adjacent.

Personally, I do think the field has a lot to learn from cybersecurity. From my POV, integrity work has three different 90's ancestors: web forum moderation,email/search engine antispam work, and cybersecurity mindsets like risk mitigation rather than risk elimination.

As for the PR stuff -- fair enough! We're all trying to do the best we can with the skills we have. Not every decision will be the right one. Maybe the name was one of those.

I hope you check back in with us in 6-12 months. Once you can judge us more by our work over time, I hope we will have earned your respect.

[spywaregorilla]

What you describe sounds like you're there to counter user fraud. It's definitely not the first thing people imagine when they hear you're there to support integrity imo. Doesn't that feel misleading to you?

[sayhar]

We spent a good amount of space on the website laying out different types of work that are covered in the term "integrity".

I also do very strongly think there is a moral/ethical dimension to the work. That's one reason I shy away from "Trust and Safety" label. Integrity work may have started with connotations of "structural integrity", but it also must be done carefully and ethically. It's not just engineering, it's also about doing the right thing. Hence the Hippocratic oath we all take. [1]

[1] https://integrityinstitute.org/our-values

[_bfhp]

Have you spoken or do you plan to speak out about the most normalized form of state-sponsored propaganda in the U.S., namely hasbara [1]? For example, Hasbara Fellowships and Israel on Campus Coalition [2]

[1] https://en.wikipedia.org/wiki/Hasbara

[2] https://en.wikipedia.org/wiki/Israel_on_Campus_Coalition#Mis...