[pen2l]

Every day that passes I grow fonder of Google's decision to delay or otherwise keep a lot of this under the wraps.

The other day I was scrolling down on YouTube shorts and a couple videos invoked an uncanny valley response from me (I think it was a clip of an unrealistically large snake covering some hut) which was somehow fascinating and strange and captivating, and then scrolling down a few more, again I saw something kind of "unbelievable"... I saw a comment or two saying it's fake, and upon closer inspection: yeah, there were enough AI'esque artifacts that one could confidently conclude it's fake.

We'd known about AI slop permeating Facebook -- usually a Jesus figure made out of unlikely set of things (like shrimp!) and we'd known that it grips eyeballs. And I don't even know in which box to categorize this, in my mind it conjures the image of those people on slot machines, mechanically and soullessly pulling levers because they are addicted. It's just so strange.

I can imagine now some of the conversations that might have happened at Google when they choose to keep a lot of innovations related to genAI under the wraps (I'm being charitable here of their motives), and I can't help but agree.

And I can't help but be saddened about OpenAI's decisions to unload a lot of this before recognizing the results of unleashing this to humanity, because I'm almost certain it'll be used more for bad things than good things, I'm certain its application on bad things will secure more eyeballs than on good things.

[lelandfe]

I saw my first AI video that completely fooled commenters: https://imgur.com/a/cbjVKMU

This was not marked as AI-generated and commenters were in awe at this fuzzy train, missing the "AIGC" signs.

I'm quite nervous for the future.

[superfrank]

I know there are people acting like this is obvious that this is AI, but I get why people wouldn't catch it, even if they know that AI is capable of creating a video like this.

A) Most of the give aways are pretty subtle and not what viewers are focused on. Sure, if you look closely the fur blends in with the pavement in some places, but I'm not going to spend 5 minutes investigating every video I see for hints of AI.

B) Even if I did notice something like that, I'm much more likely to write it off as a video filter glitch, a weird video perspective, or just low quality video. For example, when they show the inside of the car, the vertical handrails seem to bend in a weird way as the train moves, but I've seen similar things from real videos with wide angle lenses. Similar thoughts on one of the bystander's faces going blurry.

I think we just have to get people comfortable with the idea that you shouldn't trust a single unknown entity as the source or truth on things because everything can be faked. For insignificant things like this it doesn't matter, but for big things you need multiple independent sources. That's definitely an uphill battle and who knows if we can do it, but that's the only way we're going to get out the other side of this in one piece.

[jasinjames]

I agree. Also, tangentially related: I use a black and white filter on my phone, and it is way harder to distinguish fake and real media without the color channels to help. I couldn't immediately find anything in the subway clip which gave it away.

[yieldcrv]

I've definitely seen skin blurring filters that everyone already uses to make it really hard to know

[lelandfe]

Hijacking this top comment to say that I found the AI video creator: https://www.instagram.com/bugugugugu_aigc/

[n1b0m]

I agree. Apart from the text appearing backwards it all looked pretty real to me.

[lelandfe]

My assumption was the uploader wanted to make the creator's "AIGC" less obvious. It definitely did that to me.

[KeplerBoy]

Yeah, that's a weird one. I doubt the video was generated that way. I assume someone flipped the video for "artistic" purposes.

[ukuina]

Reversing text is a known loophole to getting around copyright guardrails in image-generation models.

[KeplerBoy]

How does that work? Would you prompt the model to write "hello Kitty but in reverse" on the train so the resulting image isn't flagged?

[nihil2501]

I'm beginning to write off most images as AI. I actually think that's where this is all headed.

[thih9]

There are projects like https://contentcredentials.org/ . If we want, with some effort we could distinguish between real and ai generated. If.

[jprete]

No individual actor - human or corporate - stands to benefit enough because "trust in reality" is neither easily measured nor financialized.

[thih9]

Some do care, e.g. some camera manufacturers or some news agencies. Surprisingly some social media platforms[1] want clear labels for AI generated content.

[1]: e.g. tiktok https://newsroom.tiktok.com/en-us/partnering-with-our-indust...

[netdevphoenix]

that's the easiest position imo. It's AI unless proven otherwise. No one has the time to place this much detailed on a random video when the purpose of the video is just entertainment. What this might lead to though is people losing (or not learning) the skills needed to separate real content from AI generated content

[jacobr1]

And even if it isn't AI, it is quite possibly deceptively edited. Content provenance will be important in the future.

[cess11]

A precondition is likely that one has mainly watched CGI-heavy movies for most of one's life. Compared to old school analog movies or fairly raw photography that looks as fake as the Coca-Cola Santa. There's a rather obvious lack of detail that real photography would have catched.

[Quekid5]

> A precondition is likely that one has mainly watched CGI-heavy movies for most of one's life.

Indeed, a great (if counterintuitive) example of this is The Wolf of Wall Street. I bet a lot of people would be surprised at just how much CGI is used in that just for set/location.

[lelandfe]

Wow, you were not kidding https://www.youtube.com/watch?v=pocfRVAH9yU

[sbarre]

The OG film for that was Forrest Gump. It is often lauded as one of the first movies to use CGI heavily but in completely, and intentionally, unnoticeable ways...

[Quekid5]

True, but in that case you knew it had to be CGI because Kennedy didn't talk to Tom Hanks in any capacity.

[cess11]

Sure, it's like a weird dream where sometimes shadows don't come from the sun and the scenery has this absurd, acutely unreal polish.

[fennecbutt]

A) also true that many people don't put a lot of thought into very much at all. They'd never consider actively thinking if a video is fake or not. These are the targets of short form content.

[Cthulhu_]

B is / will be huge; the largest amount of "mindless" content is consumed on phones, with half attention, often with other distractions going on and in between doing other stuff, and can be watched on older / lower fidelity devices, slower internet connections, etc. AI content needs high resolution / big screens and focused attention to "discover".

The truth is... most people will simply not care. Raised eyebrow, hm, cute, next. Critical watching is reserved for critics like the crowd on HN and the like, but they represent only a small percentage of the target audience and revenue stream.

[KennyBlanken]

You can see the perspective/angle of the objects changing slightly as the camera moves in a way that makes it pretty obvious they're CG, AI or otherwise. That's always been a problem with AI generated imagery in video/animation; it changes too much frame to frame. If researchers figure out how to address that, yeah, we've got a problem. Until then - this looks worse tha

Then there's the usual giveways for CG - sharpness, noise, lighting, color temperature, saturation - none of them match. There's also no diffuse reflection of the intense pink color.

[lambdaone]

Yes. The lack of diffuse reflection from the pink train is the clearest giveaway, and AI videos in general have problems with getting shadows and radiosity right. There's also the existence of the real-world Hello Kitty Shinkansen and the APM Cat Bus in Japan that makes this image more plausible.

[Cthulhu_]

That last point is also important; if it's not surprising, people will just accept it without being too critical about it. And since these AI tools are trained with real / existing content, creating realistic-enough content will be the norm. I think the first big AI generators - dall-e and co - had their model trained on more fantastical / artistic sources, and used that primarily as their model, also because realistic generation (like humans) wasn't yet good enough, or too uncanny. But uncanny and art work well together.

[jacobr1]

Also consider one of of the reasons AI generated video has CG like artifacts is because it is trained on CG video. Better CG generation, and more real video for training will reduce these over time.

[ecmascript]

Honestly, stuff like that could also be because of compression. We're all used to see low quality videos online.

[dagmx]

Most people have terrible eyes for distinguishing content.

I’ve worked in CG for many years and despite the online nerd fests that decry CG imagery in films, 99% of those people can’t tell what’s CG or not unless it’s incredibly obvious.

It’s the same for GenAI, though I think there are more tells. Still, most people cannot tell reality from fiction. If you just tell them it’s real, they’ll most likely believe it.

[dmazzoni]

> I’ve worked in CG for many years and despite the online nerd fests that decry CG imagery in films, 99% of those people can’t tell what’s CG or not unless it’s incredibly obvious.

I've noticed people assume things are CG that turn out to be practical effects, or 90% practical with just a bit of CG to add detail.

[dagmx]

Yep I’ve had that happen many times , where people assume my work is real and the practical is CG.

Worse, directors often lie about what’s practical and we’ll have replaced it with CG. So people online will cheer the “practicals” as being better visually, while not knowing what they’re even looking at.

I’ve seen interviews with actors even where they talk about how they look in a given shot or have done something, and not realize they’re not even really in the shot anymore.

People just have terrible eyes once you can convince them something is a certain way.

[Der_Einzige]

But films without CG are clearly superior and it’s not even in contention.

Lawrence of Arabia or Cleopatra alone have incredible fully live shot special effects which can not be easily replicated with CG and have aged like fine wine, unlike the trash early CG of the 80s and 90s which ruined otherwise great films like the last starfighter

[dagmx]

I’m sorry, but you make an absurd argument.

You’re taking the best films of an era and comparing them to an arbitrary list of movies you don’t like? Adding to that, you’re comparing it to films in the infancy of a technology?

This is peak confusion of causality and correlation. There are tons of great films in that time frame with CG. Unless you’re going to argue that Jurassic Park is bad.

[qingcharles]

The worst bit about working in CG, or film-making in general, is finding it harder to enjoy films because you are hypersensitized to bad work.

[dagmx]

Yeah, totally. It’s not even just bad work, but I’m constantly breaking down shots as I’m watching them.

Especially because I’ve done both on set and virtual production, it’s hard to suspend disbelief in a lot of films.

[circlefavshape]

> Still, most people cannot tell reality from fiction. If you just tell them it’s real, they’ll most likely believe it.

This goes for conversation too! My neighbour recently told me about a mutual neighbour who walks 200 miles per day working on his farm. When I explained that this is impossible he said "I'll have to disagree with you there"

[schoen]

Maybe not strictly impossible, just slightly better than an ultramarathon world record pace?

https://www.reddit.com/r/Ultramarathon/comments/xhbs4d/sorok...

https://en.wikipedia.org/wiki/Aleksandr_Sorokin

So, not very convenient for a non-world-champion runner to do (let alone while doing farm work) (let alone on more than one occasion).

[Cthulhu_]

That's a cultural issue that seems to have developed in the past years (decades? idk), where people take their own opinion (or what they think is their own opinion) as unchallengeable gospel.

In my opinion anyway, I'm gonna have to disagree with any counterpoints in advance.

[dagmx]

This is partially the result of being taught that every opinion is valid. What was taught as a nicety (don’t dismiss other people’s opinions was the intention) has evolved into all opinions are equal.

If all opinions are equal, and we’ve reinforced that you can find anything to strengthen an opinion, then facts don’t actually matter.

But I don’t think it’s actually all that recent. History is full of people saying that facts or logic don’t matter. The Americas were “discovered” by such a phenomenon.

[mandmandam]

What's weird is the projection you get when you challenge someone's opinion in any way. All of a sudden, you're the arrogant one who thinks they're always right, no matter how diplomatic (or undeniably correct) about the issue you are. Or is that just me?

[5040]

>Most people have terrible eyes for distinguishing content

A related phenomenon is not being able to hear the difference between 128kbps and 320kbps. I find the notion astonishing, and yet lots of people cannot tell the difference.

[lancesells]

> Most people have terrible eyes for distinguishing content.

But also in the case of the fluffy train there's nothing to compare it against. The reason CGI humans look the most fake is because we're trained from birth to read a human face. Someone that looks at trains on a regular basis will probably discern this as being fake quicker than most.

[krick]

Looks dope though. But what impressed me recently was some crypto-scam video, featuring "a clip" from Lex Fridman Podcast where Elon Musk "reveals" his new crypto or whatever (sadly, the one I saw is currently deleted). It didn't really look good, they were talking with weird pauses and intonations, and as awkward these 2 normally are, here they were even more unnatural. There was so much audacity to it I laughed out loud.

But what I was thinking while enjoying the show was: people wouldn't do that, if it didn't work.

This is the point. There is no such thing as "completely fools commenters". I mean, it didn't fool you, apparently. (But don't be sad, I bet you were fooled by something else: you just don't know it, obviously.) But some of it always fools somebody.

I really liked how Thiel mentioned on some podcast that ChatGPT successfully passed Turing test, which was implicitly assumed to be "the holy grail of AI", and nobody really noticed. This is completely true. We don't really think about ChatGPT, as something that passes Turing test, we think how fucking stupid useless thing mislead you with some mistake in calculations you decided to delegate to it. But realistically, if it doesn't it's only because it is specifically trained to try to avoid passing it.

[lelandfe]

I wish you were right that there is no way to completely fool viewers, but I know you are not. I was fooled! Note that I call out "AIGC." If that wasn't there (I only noticed it on repeat views), I would have simply had no way to tell. These are early, primitive AI generated videos, and I'm already unable to differentiate. Many in this thread talk about movie CG; there are countless movie scenes that fool all viewers.

[coffeebeqn]

If someone were to train a model on Joe Rogan podcasts whole run, I’m sure it would spit out extremely impressive fake results already

[vintermann]

> people wouldn't do that, if it didn't work.

You can't assume that with scams. Quite often, scams are themselves sold as a get-rich-quick scheme, which like all GRQ schemes, they wouldn't be if they worked well.

[peab]

Think about this: you very well may have already seen AI videos that fooled you - you wouldn't know if you did.

[coffeebeqn]

One of the clearest signs in the current gen is that the typography looks bad still.

[darkerside]

People are smart enough to know that what you see in movies isn't real. It will just take a little time for people to realize that now applies to all videos and images.

[nihil2501]

The frequency is so high, and I am getting so burned out on checking comments to gauge how much everything is changing, that I've nearly given up subconsciously. Pretty close to just ignoring all images I see.

[nurettin]

This is definitely something the Japanese would do, but it is not a real train unless a thousand salarymen are crammed into it.

[matwood]

The bigger problem is that people think something this ridiculous could happen.

[marci]

Weirder things have been created. I could definitely see one being made for a movie.

[espadrine]

> I'm quite nervous for the future.

Videos like these were already achievable through VFX.

The only difference here is a reduction in costs. That does mean that more people will produce misinformation, but the problem is one that we have had time to tackle, and which gave rise to Snopes and many others.

[ImaCake]

I mean the only real tell for me is how expensive this stunt would be. I personally think this is a really cool use of genAI. But the consequences will be far reaching.

[lelandfe]

Some of the comments were like, "come on guys, if this was real it would be way dirtier"

[starshadowx2]

The face of the girl on the left at the start in the first second should have been a giveaway.

[Perseids]

My intuition went for video compression artifact instead of AI modeling problem. There is even a moment directly before the cut that can be interpreted as the next key frame clearing up the face. To be honest, the whole video could have fooled me. There is definitely an aspect in discerning these videos that can be trained just by watching more of them with a critical eye, so try to be kind to those that did not concern themselves with generative AI as much as you have.

[yccs27]

Yeah, it's unfortunate that video compression already introduces artifacts into real videos, so minor genAI artifacts don't stand out.

It also took me a while to find any truly unambiguous signs of AI generation. For example, the reflection on the inside of the windows is wonky, but in real life warped glass can also produce weird reflections. I finally found a dark rectangle inside the door window, which at first stays fixed like a sign on the glass. However it then begins to move like part of the reflection, which really broke the illusion for me.

[booleandilemma]

No one is looking at her face though, they're looking at the giant hello kitty train. And you were only looking at her face because you were told it's an AI-generated video. I agree with superfrank that extreme skepticism of everything seen online is going to have to be the default, unfortunately.

[vlovich123]

Hard to not discount that as a compression artifact.

[magicalhippo]

Just like all the obvious signs[1] the moon landings were faked.

[1]: https://web.archive.org/web/20120829004513/http://stuffucanu...

[lelandfe]

Just wanted to say I really enjoyed this!

[Nition]

One thing that's not intuitive to spot but actually completely wrong, is that in the second clip we're apparently inside the train but the train is still rolling under us.

[lmm]

Or, y'know, the camera's moving smoothly backwards through the train? Would be bit of an odd choice (and high-effort to make it that smooth versus someone just carrying it) but not impossible by any means.

[tim333]

Also "HELLO KITTY" being backwards is odd - writting on trains doesn't normally come out like that eg https://www.groupe-sncf.com/medias-publics/styles/crop_1_1/p...

[slightwinder]

All the text is mirrored. It's not unusual doing this to avoid copyright-filters. This kinda adds to distracting suspicions.

[colordrops]

The whole video was probably mirrored before being posted. Doesn't seem to be related to being AI generated.

[solfox]

On the other hand, because these tools like this are being made available before output is perfected, you and many others are being trained in AI discernment; being able to detect fake things will be a helpful skill to have for some time: another form of critical thinking.

It would be FAR worse if a privately held advanced AI's outputs were unleashed without the population being at least somewhat cautious of everything. The real danger imho comes from private silos of advanced general intelligence that aren't shared and used to gain power, control, and money.

[underdeserver]

I think as these things will get bigger and better much faster than we can learn to discern.

[solfox]

With zero doubt. Faster than we expect. And yet, it's nice that we are learning to distrust what we see before the "real real" stuff comes out.

[echelon]

Open source has already caught up with SOTA:

https://www.reddit.com/r/StableDiffusion/comments/1hav4z3/op...

These are even unfair comparisons because they're leveraging text-to-video instead of the more powerful image-to-video. In the latter case, the results are indistinguishable.

Video generation is about to be everywhere, and we're about to have the "Stable Diffusion" moment for video.

Look at the comments: people are already fawning over open source being uncensored.

Cat's out of the bag.

[Jerrrry]

Very convenient for those who are waiting for the waters to get muddier.

[lancesells]

I'm wondering that as well but I also wonder if it's a bit like CGI where it's somewhat hit a limit on realness. I'm not saying CGI doesn't get better but is a 2024 Gollum that much more realistic than 2004 Gollum? Maybe I'm wrong but I wonder if that plastic feel to AI lessens but still sticks around.

[thinkingtoilet]

>you and many others are being trained in AI discernment

HN is a hyper specialized group of people. The average person can not do this and as we've seen devours up misinformation with no second thoughts.

[thereddaikon]

On one hand, I like to think that society is getting trained to recognize AI and distrust it. But at the same time my retired boomer parents are over for the holidays and I catch them watching youtube videos completely oblivious to the fact it's an AI voice and just reading an LLM generated script with B roll for eye candy. Often times it's just stolen auto generated captions from larger creators regurgitated by an AI voice. I'll point it out and they don't believe me that the voice is fake.

[wongarsu]

AI voices have gotten scarily good. They are easy to recognize because most creators use the same voices with the same intonations and don't care to cut out the mistakes. But if you don't recognize the voice it takes a couple sentences to discern that it's AI even with an ear trained on the difference.

But it is funny to see how much stuff gets uploaded with zero quality control and still gets traction. These models really don't deal will with "innocent" letter substitutions, Iike using I instead of l.

[thereddaikon]

I've heard enough slop using the ElevenLabs voices that I can recognize them almost immediately now. But you're right. Higher end models with less familiar voices are harder to notice. One consistent failing is that they are always too perfect. No mistakes or signs of cuts to edit out where a human VA would have made a mistake. Its all very smooth and perfect. As if they nailed it in the first shot. Once the cheap/free models manage to fix that then we are in real trouble. Also, some really lazy slop creators don't bother to fix issues with pronunciation. But that's not the fault of the model really.

[bumbledraven]

"More human than human" is our motto. https://youtu.be/ZbgmYhqFO-4?t=30

[solfox]

And yet, OP referred to a thread where the reality of the shorts were being questioned by "average" people. Imagine a world where OpenAI were the first out the gates with this and just started producing their own videos without telling anyone about their technology or letting creators play with it. They'd make loads of money, probably could topple governments... I'm glad these tools are being made generally available versus the alternative.

[quenix]

It saddens me. Innovations in AI 'art' generation (music, audio, photo) have been a net negative to society and are already actively harming the Internet and our media sphere.

Like I said in another comment, LLMs are cool and useful, but who in the hell asked for AI art? It's good enough to fool people and break the fragile trust relationship we had with online content, but is also extremely shit and carries no meaning or depth whatsoever.

[anxoo]

>who in the hell asked for AI art?

everyone who has ever used stock photography, custom illustrators, and image editing. as AI improves, it will come after all of those industries.

that said, it is not OpenAI's goal to beat shutterstock, nor is it the goal of anthropic or google or meta. their goal is to make god: https://ia.samaltman.com/ . visual perception (and generation) is the near-term step on that path. every discussion of AI that doesn't acknowlege this goal, what all of these billions of dollars are aiming for, is myopic and naive.

[rurp]

There was a recent discussion in another HN thread that I think summed it up well. Good art rewards a careful viewer; the more you look at and think about good art, the more you get out of it. AI art does the opposite and punishes thoughtful consumers. There's no logical underpinning to the various details, it's just stuff mashed together in a superficially nice looking way.

[mojuba]

I think AI "art" can be as useful as the text generators, i.e. only within certain limits of dull and stupid stuff that needs to exist but has little to no value.

For example, you need to generate a landing page for your boring company: text, images, videos and the overall design (as well as code!) can be and should be generated because... who cares about your boring company's landing page, right?

[whatevertrevor]

One could ask why the boring company landing page exists in the first place though. If it's not providing value to humans to warrant actual attention being paid to it...

[tomjen3]

The world is in need of soap. Not the fancy beautiful artistic kind, but the kind that comes in containers and you put in bathrooms. This objectively saves lives and is one of those boring things I can imagine.

[carlosjobim]

Then you don't understand the purpose of a landing page. If the boring company hires somebody to make the landing page who actually understands their job, the landing page will have great importance.

[yunwal]

> the landing page will have great importance.

Most companies don't need this. They need a page that has their contact info and some general information about services they provide so they can have a bare minimum internet presence and show up on google maps.

[carlosjobim]

Absolutely, if your company doesn't want to make sales or if you want to be bothered all the time by people calling and mailing only for them to find out your product isn't a fit for them. Or if you want third party sellers to take over most of your business like Booking.com, AirBnB DoorDash or Amazon.

Companies who understand the importance of a customer friendly and functional web presence get a great return on their investment. And it's much better for the customer.

[yunwal]

I have an ice cream shop by me that doesn't even have a website. They're mobbed every day, because good ice cream is fairly self explanatory, and doesn't need a web presence

[dale_glass]

> Like I said in another comment, LLMs are cool and useful, but who in the hell asked for AI art?

I did. I started messing around with computer graphics on DOS with QBASIC and consider AI art to be just an extension of that.

On the other hand I don't care all that much for LLMs most of the time. They're sometimes useful, but while I find AI art I enjoy very regularly, using a LLM for something is more a once every couple weeks event for me.

[computerex]

How do you know they are a net negative? What's your source?

[quenix]

My opinion ;-)

That's what HN is for

[CamperBob2]

It's quite well-supported on here, that's for sure.

Somewhere there's a site for "hackers" where it isn't, and I hope I stumble across that site at some point.

[Cthulhu_]

Do add "in my opinion" or prefix with "I think", because your definite wording implied you were stating a verifiable fact. Telling opinions like they are facts and then backtracking with "oh but it was just my opinion" is a big problem in (online?) society / discourse, and has led to a lot of misinformation and anti-scientific takes spreading.

"The earth is flat" - "Can you prove it?" - "Oh it's just my opinion". It's dishonest.

[randomlurking]

I agree with the first part. For me, AI art is the chance to have a somewhat creative outlet that I wouldn’t have otherwise, because I’m much worse at painting that I can stand. Drawing by prompts helps me be creative and work through some stuff - for that it’s also nice and interesting to see that the result differs from my mental image. I will tweak the prompt to some extent and to some extent go with some unintentioned elements of the drawing. I keep the drawing on my phone in the notes app with a title and the prompt.

To get back to the beginning: I really do agree that the societal impact on the whole appears to be negative. But there are some positives and I wanted to share my example of that.

[tomjen3]

That describes most art. At least ai art can be pretty and doesn’t have the same political message.

[Der_Einzige]

Go on civil.AI, it’s primarily used for hardcore waifu porn.

[jeffhuys]

You mean civitai.com? There's a lot more on there than just that...

[lmm]

Much of the time I don't want "meaning or depth", I just want a pretty picture of whatever it was. AI art is great, it's just that the people it most benefits are the people you don't see or hear much from (and, rude as this is to say, people who write less convincingly).

[computerex]

They should have kept this amazing tech under the wraps because you have a bad feeling about it? Hate to break it to you, but there have been fake videos on the internet ever since it has existed. There are more ways to fake videos than GenAI. If you haven't been consuming everything on the internet with a high alert bs sensor, then that's an issue of its own. You shouldn't trust things on the internet anyway unless there is overwhelming evidence.

[callc]

Amazing tech != socially good

Of course, as knowledgeable people in tech we can look at the last few years of AI improvements as technically remarkable. pen2l is talking about social impact.

I hope our trade can collectively become adults at the big table of Real Engineers. Consider the impact on humanity of your work. If you don’t care, then you are either recklessly irresponsible, don’t know any better, or are intentionally causing harm at scale.

[OskarS]

Very well put. There's always been this Silicon Valley instinct that all technological advance is always good for humanity, and it's just not that simple.

[callc]

Thanks OskarS

Tech is a very powerful tool that can automate the most mundane tasks and also automate harm like mass surveillance and erosion of ownership rights of your devices. The sheer ability to create new markets and replace inefficient non-automated markets leads to huge $$$ making opportunities which people may mistake as being good in itself (good for economy / GDP = good for humanity)

[arsenico]

Cannot even quality "It has always been shit, so no problem at it becoming even shittier" as a hot take.

[sergiogdr]

> If you haven't been consuming everything on the internet with a high alert bs sensor, then that's an issue of its own

"just be privileged as I was to get all the necessary education to be able to not be fooled by this tech". Yeah, very realistic and compassionate.

[cma]

With a heavy dose of "if masses of people are fooled by this, it can't affect me as long as I can see through it. No possible repercussions of mass people believing completely made up stuff that could affect laws, etc."

[JPKab]

This entire thread reeks of "I'm smart enough to know that videos can be faked, but Jethro in the trailer park isn't because he's just a plumber, and therefore this tech needs to be censored or else Jethro might believe stuff that makes him vote in a way I don't like" going on here.

While the average person overestimates their own intelligence, the average techy dramatically underestimates the intelligence of the average member of the public. The weirdos that latch onto every fake video and silly conspiracy theory are dramatically overrepresented in every online comments thread, but supposed geniuses in the tech/NGO/academic community forget this and assume a broad swath of the public believes in stuff like "Pizza gate" because nuanced thinking is a skill only the enlightened few possess.

[someothherguyy]

Some people aren't very skeptical at baseline, it doesn't mean that those concerned about the ability of others to recognize AI are disparaging people based on intelligence.

For example, some people can be very intelligent, yet not be discerning of information that resonates with prior biases. You see this in those who are devoutly religious, politically polarized, etc.

There is reason to believe that such biases will lend to ontological misinterpretations from algorithmically generated information.

You can see mistakes in interpretation on a day to day basis by the population at large. There are swaths of widely held beliefs that aren't based in truth. Pretty much anyone is likely to believe at least some stereotype, folklore, urban legend, or myth.

[sergiogdr]

It isn’t about being smart (you assumed this is what ‘education’ was pointing at). Most people aren’t even aware of what’s happening besides extremely superficial things that they get here and there on the news. Can’t you honestly see the real potential for massive damage coming out of all this?

[pixelsort]

With respect to the American public, the majority can and do utilize nuanced thinking as a survival skill. The problem of modern American era, is not that our public is low in average intelligence. Rather, that on average, we have been miseducated to seek the eradication of discomfort, uncertainty, inconveniences, and unknowns.

[cma]

That radio station in hotel Rawanda could be a bad thing for you and people you cared about even if you personally could discern the lies so it wasn't fooling you.

[krisboyz781]

Actually you overestimate the general public's ability to discern what's real or not. On top of that, most people don't even care if it's real. This is exactly why Trump won.

Example: if a gen ai vid of a politician doing some crazy crime came out. Even if it were proven fake, people would start questioning everything and still act as if the politician were guilty

[JPKab]

"This is exactly why Trump won"

See the part of my comment you are replying to where I specifically stated that the motivation for all of this is that "Jethro doesn't vote the way I want him to". You've proven my point.

The censorious attitudes on HN were non-existent before Trump won in 2016. I know this for a fact. I've had my account on here since 2012, after 2 years of being just a reader.

Meanwhile, you overestimate how immune to misinformation and lies the average HN techy is. Just a few years ago, the majority of people on here believed, with utter conviction, that the bat-borne coronavirus lab in Wuhan had absolutely no connection with the bat-borne coronavirus epidemic that started in Wuhan and that only bigots and ignoramuses could draw such a conclusion. I experienced this whenever I brought up the blatantly obvious, common sense connection in these same comment threads in late 2020 or into mid 2021. The absolutely absurd denial of common sense by otherwise intelligent people was reminiscent of trying to talk to a religious fundamentalist about evolution while pointing at dinosaur fossils and having them continue to deny what was staring them in the face.

[sekai]

> "just be privileged as I was to get all the necessary education to be able to not be fooled by this tech". Yeah, very realistic and compassionate.

This has nothing to do with privilege, a person in Indian slums on his 2005 PC with internet access can have better internet BS radar than an Ivy League student.

[sergiogdr]

I think that would be an exception rather than the rule, to be honest.

I think though, that if you are in the position of doing serious critical reflection about this stuff, which is in my opinion necessary for being in a position of discernment wrt this stuff, then you are privileged. This is the idea I wanted to convey.

[JPKab]

What education do you specifically think is necessary for people with average IQs all over the world to not be fooled by this, given that they are aware that videos can easily be faked in 2024? A high school degree? A bachelors?

[gambiting]

>>given that they are aware that videos can easily be faked in 2024?

That's a ridiculous assumption. In my experience no one outside of tech circles is even remotely aware that this kind of thing is possible already.

[IAmGraydon]

With all due respect, I think you may be out of touch.

[JPKab]

You think that the average member of the public isn't aware that videos can be faked with AI, or non-AI special effects, and your source of data for this is "your own experience"? Really?

My family is mostly working class in an economically depressed part of the Virginia/West Virginia coal country, and every single one of them is aware of this. None of them work in tech, obviously. None have college degrees.

I maintain that the attitude driving this paternalistic, censorious attitude is arrogance and condescension.

A prime example of how broadly aware the public all over the world is of AI faked videos was the reaction in the Arab world to the October 7th videos posted by Hamas. A shocking (and depressing) percentage of Arabs, as well as non-Arab Muslims all over the world, believed the videos and pictures were fakes produced with AI. I don't remember the exact number, but the polling I saw in November showed it was over 50% who believed they were fakes in countries as disparate as Egypt and Indonesia.

[gambiting]

>>isn't aware that videos can be faked with AI, or non-AI special effects

These two are very different things. My family believes all kinds of videos on the internet are fake. None of them have any idea what a tool like Sora can do. The gap between "oh this was probably special effects" to "you have to notice pixels shimmering around someone's hand to tell" is enormous.

>>My family is mostly working class in an economically depressed part of the Virginia/West Virginia coal country, and every single one of them is aware of this.

Your working class family has time to keep up with the advancements in generative AI for video? They have more free time than I do then. If we're sharing anecdotes about families then my family is from Polish coal country and their idea of AI is talking to your car and it responding poorly.

>>I maintain that the attitude driving this paternalistic, censorious attitude is arrogance and condescension.

I'm confused - who is displaying this "censorious" attitude here?

>> and your source of data for this is "your own experience"? Really?

Yes, really. I mean do you have anything else? You are also quoting things from your own experience.

[sergiogdr]

I’m not (exclusively) talking about formal education. There are lots of people (I would dare say the majority of the planet) that don’t have the ‘digital literacy’ required to handle what’s happening right now. Being from a developed country I am very much worried about this.

[8n4vidtmkvmk]

Fooled by what? Some of it looks real but is incredulous enough that it should set off your BS sensor. Other stuff is/will be more subtle and we will have no way of knowing.

[mrcwinn]

Too charitable indeed. Google was simply unprepared and has inferior alternatives.

My prediction is that next year they will catch up a bit and will not be shy about releasing new technology. They will remain behind in LLMs but at least will more deeply envelope their own existing products, thus creating a narrative of improved innovation and profit potential. They will publicly acknowledge perceived risks and say they have teams ensuring it will be okay.

[tziki]

>They will remain behind in LLMs

The latest Gemini version (1206) is at least tied for the best LLM, if not the best outright.

[pier25]

I wish Google would allow me to remove the AI stuff from search results.

99% of the times it's either useless or wrong.

[titzer]

Strong plus one here. Not only that, but it uses gobs of energy in total. Google has reneged on all of its carbon promises to stay in the running for AI domination and to head off disruption to search ads business. Since I've unconsciously trained my brain to not look at the top search results anymore because they long ago turned into impossible-to-distinguish ads, I've quickly learned to just ignore the stupid AI summary. So it's an absurd waste of computational power to generate something wrong that I don't even want to see, and I can't even tell them to stop when they're wasting their own money to do so.

[Tempat]

It’s often wrong anyway. Much like you, the thing that annoys me most about it though is all the power they must be using having it run on every single search by anyone.

[Lcchy]

I have been using Kagi for a year now and it's been liberating. Its an ad/seo-free search engine.

https://kagi.com/

Sorry for the name dropping, I have no affiliation and am just a very happy user, so I wanted to share it as it felt adequate.

[fraXis]

Add a -ai to the end of your Google search query. There are also browser extensions that stop the AI content from displaying. I use the one for Chrome called "Remove Google Search Generative AI".

[dangerwill]

Great tip! But it only remove's Google's terrible AI summary, not AI generated content from showing up in searches, which is what the OP wishes for. A combination of -ai and before:2022-01-01 is probably the closest we can get to that

[1kaizen]

This is vaporware/false advertising.

We don't have tech to correctly "detect ai" in 2024, which is why education has broken down over the last few years with serial cheating in every institution.

Every company so far that claimed to detect AI generated slug has failed.

[Slyfox33]

you can block is with Ublock Origin

https://www.reddit.com/r/uBlockOrigin/comments/1ct5mpt/heres...

[KeplerBoy]

Nobody has any clue what is AI stuff these days. Apart from the obvious ones, no one can tell a generative AI apart from 3D rendered stuff or low-res photos. Put image compression on top and it's definitely impossible.

[pier25]

I meant the bit of AI that Google adds on top the actual search results.

[KeplerBoy]

Ah sure, that stuff is just annyoing. I don't need a - probably wrong - summary of the top hit either.

[imiric]

I wonder what the outcome will be when new models are trained on AI-generated data. These companies are already running out of quality training data. So when most of the data on the internet is synthetic, will they find ways of separating the signal from the noise, or will all the noise lead to a convergence of performance across all models to something that is much inferior than what we have today?

This tech will make the internet even more unbearable to use, without mentioning its huge potential for abuse. This is far worse than whatever positives it might have, which are still unclear. What a shitshow.

[cobalt60]

Udm?

[tlrobinson]

This is all inevitable. At worst it's pulling the issues forward by a few months or years, and I don't think anyone will meaningfully address the problem until it's staring us in the face.

I believe the internet needs a distributed trust and reputation layer. I haven't fully thought through all the details, but:

- Some way to subscribe to fact checking providers of your choice.

- Some way to tie individuals' reputation to the things they post.

- Overlay those trust and reputation layers.

I want to see a score for every webpage, and be able to drill into what factored into that score, and any additional context people have provided (e.x. Community Notes).

There's a huge bootstrapping and incentive problem though. I think all the big players would need to work together to build this. Social media, legacy media companies, browsers, etc.

This also presupposes people actually care about the truth, which unfortunately doesn't always seem like the case.

[bko]

I don't think Google delayed or kept this under wraps for any noble reasons. I think they were just disorganized as evidenced by their recent scrambling to compete in this space.

[makestuff]

I don't even know if this will be possible, or how it would work, but it seems like the next iteration of social media will be based on some verification that the user is not using AI or is a bot. Currently they are all incentivized to not stop bot activity because it increases user counts, ad revenue, etc.

Maybe the model is you have to pay per account to use it, or maybe the model will be something else.

I doubt this will make everyone just go back to primarily communicating in person/via voice servers but that is a possibility.

[joaohaas]

Twitter Blue is paid and yet every single bot account has it in order to boost views.

[debugnik]

> Maybe the model is you have to pay per account to use it

Spammers can afford more money per bot for their operations than the average user can justify to spend on social media.

[mnau]

So Musk was right?

[CJefferson]

No, because Musk encourages AI slop if people are willing to pay.

What we probably need (this is going to sound crazy, but I don’t have a better suggestion), is some kind of networked trust system.

[jeffhuys]

Like Community Notes? It's actually a darn good system.

[lanthissa]

exactly one lab has passed the test of morals vs profit at this point, and thats deepmind, and they were thoroughly punished for it.

Every value oAI has claimed to have hasn't lasted a milisecond longer than there was profit motive to break it, and even anthropic is doing military tech now.

[dmix]

LLMs aren’t AGI

[kylehotchkiss]

> the image of those people on slot machines, mechanically and soullessly pulling levers because they are addicted. It's just so strange.

Worse, the audience is our parents and grandparents. They have little context to be able to sort out reality from this stuff

[soulofmischief]

Shorts are designed to trade your valuable attention for trite, low-effort content. Most decent shorts are just clips of longer-form content.

Do yourself a favor and avoid that kind of content, opting instead for long-form consumption. The discovery patterns are different, but you're less inclined to encounter fake content if you develop a trust network of good channels.

[jprete]

This is also my strategy. AI content makes me focus even harder on the source of the content instead of the apparent quality, because the current set of GenAI techniques are best at imitating surface-level quality features.

[soulofmischief]

What are some good channels you recommend?

[freehorse]

The way AI goes, it will actually raise the cost of valid services: cost of bullshit and spam is going down, which will raise the cost of valid, non-ai powered services to raise above the noise or be able to filter it out. There is only negative value to what "open"-ai is adding to the world right now. By playing the long-term AI safety card, of the hypothetical scenario some AI supposedly getting conscious in the future, they try to pass themselves clean and innocent in all the damage they cause to society.

I just hope the online, social media space gets enshitified to an such a degree that it stops playing a major role in society, though sadly that is not how things usually seem to work.

[DrScientist]

On the other hand by making public what technologies capabilities are - doesn't it stop the problem of people having this tech in secret and using it before anybody is aware it's even possible?

ie a company developing this tech, keeping under wraps and say only using for special government programmes....

[dyauspitr]

Pandora’s box is open, not releasing models and tools is just going to result in someone else doing it.

[whywhywhywhy]

They didn’t keep it under wraps, it’s just the team considered the paper shipping not the product. They still shipped the papers that decentralized the knowledge.

Could even argue shipping the product and not the paper would have done more for AI safety, least it would be controlled.

[ActionHank]

The best part is that eventually, over time, the AI slop will feed into training data more and more. I suspect it will be like the Kessler Syndrome of AI models.

[fullstackchris]

The ability to make strange videos as a consumer... it's not inherently good or bad, it'll just be... weird

[MrBuddyCasino]

It doesn't take AI to fool people. They have been propagandised and lied to on a major scale since mass media.

They also lie themselves: they cannot detect overt bias or reflect on themselves and be aware of their hidden motives, resentments and wishful thinking. Including me and you.

Most people hold important beliefs about the world that are comically inaccurate.

AI changes absolutely nothing how many true or false beliefs the average Joe holds.

[littlestymaar]

> And I can't help but be saddened about OpenAI's decisions to unload a lot of this before recognizing the results of unleashing this to humanity

Yeah, and it's especially hypocrite coming from them who said they'd refuse to disclose anything about GPT-3 because they said it was dangerous. And then a few years latter: “Hey remember about this thing we told you it was too dangerous before? Now we have a monetization strategy so we're giving access to everyone, today.”

[stronglikedan]

> there were enough AI'esque artifacts that one could confidently conclude it's fake.

And yet, you would not have known how to recognize those artifacts without "OpenAI's decisions to unload a lot of this before recognizing the results of unleashing this to humanity".

[serial_dev]

You could have said the same thing about photo shop... Some people will learn to spot BS and think critically even if they can't quite put their finger on it and the video is very good (What, Trump fought a T-Rex, AND WON?), some people could be fooled by anything, and there is a lot in between.

[raincole]

Considering google image search is polluted by AI-generated images at this moment, perhaps google is afraid of making the search even worse?