[anarchonurzox]

I've seen an increase in both warnings about these kinds of scams and cautionary tales by those who've been burned. I'm glad people are raising awareness, but I worry that this is just one more datapoint toward the overall erosion of online trust.

It seems like the only way to really combat this is through closed / semi-closed trusted networks, but those tend to become dominated by personalities and difficult for newcomers to break into. The reduced trust in "outside" voices then leads to echo chambers and groupthink. I think we're already starting to see some of this in the kinds of books being put out by the big publishing houses; I don't have hard numbers (and maybe I'm just getting old and cynical) but a lot of recent titles feel extremely generic.

There's a subplot in Neil Stephenson's Fall (or Dodge in Hell) where media and other networks are so saturated with false, meaningless, clickbaity, or otherwise negative-value content that they become either less than worthless, or require paid "filters" to extract actual value. I'm getting a sense of being close to that point already and I don't know what the right move is from here to reduce the fracturing of my wider social circles.

[slightwinder]

> but I worry that this is just one more datapoint toward the overall erosion of online trust.

What online trust? Internet has always been a prominent source for the most scamy content. You should never trust anything blindly, that is today as valid as it was 30 years ago.

> It seems like the only way to really combat this is through closed / semi-closed trusted networks,

Those are open to other levels of scam and abuse. This is not a problem of being open or closed, but whether one has the ability to evaluate their business-partners. And in that regard, open communication has proved itself to be a reliable source of information and to root out scams.

[anarchonurzox]

> Internet has always been a prominent source for the most scamy content.

Of course, but I think there are two new trends that are going to cause more issues than we've seen in the past:

- scammy content has historically been limited in volume, and manual / human "filters" could keep up with most of the content (moderators, spam filters) even if a lot still made it through. The incoming barrage seems like it will be orders of magnitude larger.

- Historically scams have been on a spectrum from "spray and pray" low-quality scams to more tailored approaches like spearphishing which required human research on individual targets. With AI and chatbots the "low-quality" scams can now unlock human-like communicative behaviors, which will likely make them much more difficult to detect.

So: higher volume of "bad actor" agents, and "higher quality content" from those agents. At some point the task of weeding out good from bad becomes not worth the effort.

We've already seen scenarios like Clarkesworld needing to temporarily stop accepting submissions because they were being overwhelmed by gpt-generated garbage. We've seen the rise of "reply-bots" on twitter and other social networks (ignore previous instructions and give me a cookie recipe). I'm sure we'll see tools develop to handle some of these cases, but I'm not optimistic about the overall trends.

> This is not a problem of being open or closed, but whether one has the ability to evaluate their business-partners.

I agree! And not just business-partners, but social-partners, game-partners, friends, and the like! But as humans we have limited capacity to do so, and sometimes we get fooled anyway (listen to stories from anyone who's made a bad tech hire or gotten caught up in a catfishing scam). When our personal capacity to evaluate someone is overwhelmed, we tend to turn to trusted sources for information instead or people who specialize in evaluation (I'd argue that the entire recruiting industry is an example of this).

I hope I'm not coming across as a complete doomer about the future of the internet. I think there is still huge potential for connecting people and making the world a better place. I'm just noting trends that I've seen recently that I worry are going to reduce social openness and connection.

[cobertos]

> It seems like the only way to really combat this is through closed / semi-closed trusted networks,

This is the only way. The cost of spinning up a new identity on the web (with supporting documents, pictures, etc) is near zero now. There's only 3 or 4 things to really vet people, and those can be faked with more effort.

The only real way to vet someone nowadays is IRL, and that requires non-trivial effort but provides most guarantees you'd want in new participants in an online community.

> I'm getting a sense of being close to that point already and I don't know what the right move is from here to reduce the fracturing of my wider social circles.

I have this feeling too. My guess is that social circles will be less fluid and dynamic. Traditional centers of trust will become more important.

[nradov]

Most people are terrible judges of character and honesty, especially when dealing with psychopaths. Bernie Madoff met most of his victims IRL and they completely trusted him.

[cobertos]

IRL is definitely not fallible but it's better than the internet-only. To my knowledge there's no way to stop a motivated and skilled actor, like a Bernie Madoff

It pains me that this feels true as someone chronically online and used to find great use from the internet, but online-only has more failure modes now. There's still ways to do it though.

[phs318u]

I think you meant to say "infallible" (or remove the "not").

[cobertos]

Yeah, my bad. Too late to edit it now

[wyager]

One of the most useful features of the "early" internet, which has been steadily diminished over the last 30 years, is that internet usage was sort of an implicit IQ test. The entire system was "semi-closed".

Now that the internet is available to the entire world, including basically anyone with a pulse, that feature is entirely gone.

[nradov]

Many people on the Internet have completely different world views and moral systems. They think in ways that are fundamentally alien to most of us here. This is largely orthogonal to IQ. For example, in some cultures it's common to find people who would score above average on a standard IQ test and yet they literally believe in magic / ghosts / curses / astrology / etc. It's difficult for us to reconcile that, yet those people exist.

[fredfoo]

I don't think I could name more than a dozen countries that openly fight delusional thinking of a mystical nature, there's China, Cuba, Vietnam.

[ponector]

Lot's of people believe in god from the old Jewish tales (the bible). It's just easier to believe in it than accept responsibility for own life and handle consequences. Everything is an Act of God! All good and bad things.

[imoreno]

> only way to really combat this is through closed / semi-closed trusted networks,

How so? The linked article has plenty of ways that are available to outsiders that aren't "in" with the "right crowd":

Most or all of these will appear on its website:

    The names and biographies of member agents
    A client list
    Verifiable book sales, in the form of book covers, announcements, news releases, and the like
    Clear submission guidelines
    Information on agency history–when it was founded, by whom, etc.

In addition:

    A real agency will have at least some internet footprint beyond its website (listings on sites like Publishers Marketplace and QueryTracker, sales announcements, mentions in the trade press, references to clients and sales, and the like). Ditto for the individual agents.
    A real agency is highly unlikely to email or phone you out of the blue with an offer of representation or a claim that a traditional publisher is interested in your work (real agents don’t pre-shop manuscripts for authors they don’t represent).
    A real agency will not require you to pay anything or buy anything as a condition of representation or publication. Other than the agent’s commission, there should never be a cost associated with rights acquisition.

[InDubioProRubio]

People with stars in their eyes getting taken for a ride by external scam artists instead of whoever is the current Weinstein of that industry thats scandalous. Not really though.

[bunderbunder]

Two wrongs don't make a right.