[equestria]

Eh, this is pretty unfair. That's a test of how good humans are at deceiving other humans, not a of how hard it is to distinguish run-of-the-mill AI art from run-of-the-mill human art in real life.

First, by their own admission, the author deliberately searched for generative images that don't exhibit any of the telltale defects or art choices associated with this tech. For example, they rejected the "cat on a throne" image, the baby portrait, and so on. They basically did a pre-screen to toss out anything the author recognized as AI, hugely biasing the results.

Then, they went through a similar elimination process for human images to zero in on fairly ambiguous artwork that could be confused with machine-generated. The "victorian megaship" one is a particularly good example of such chicanery. When discussing the "angel woman" image, they even express regret for not getting rid of that pic because of a single detail that pointed to human work.

Basically, the author did their best to design a quiz that humans should fail... and humans still did better than chance.

[bena]

Also impressionism is probably one of the most favorable art styles for AI. The lack of detail means there are fewer places for AI to fuck up.

A street with cafe chairs and lights, that's like an entire genre of impressionist paintings.

[bongodongobob]

I think it's fair. It's the same thing humans do with their own art. You don't release the piece until you like it. You revise until you think it's don't. If a human wants to make AI art, they aren't just going to drop the first thing they generated. They're going to iterate. I think it's just as unfair to include the worst generations, because people are going to release the highest quality they can come up with.

[equestria]

> I think it's fair. It's the same thing humans do with their own art.

No, hold on. The key part is that you have a quiz that purports to test the ability of an average human to tell AI artwork from human artwork.

So if you specifically select images for this quiz based on the fact that you, the author of the quiz, can't tell them apart, then your quiz is no longer testing what it's promised to. It's now a quiz of "are you incrementally better than the author at telling apart AI and non-AI images". Which is a lot less interesting, right?

I'm not saying the quiz has to include low-quality AI artwork. It also doesn't need to include preschoolers' doodles on the human side. But it's one thing to have some neutral quality bar, and another thing altogether to choose images specifically to subvert the stated goal of the test.

[bongodongobob]

I don't see why you wouldn't use the highest quality possible for both.

[npinsker]

But they didn't do this at all. They picked the most human-like AI images (usually high quality), and the most AI-like human images (usually mid).

The anime pictures are particularly poor and look much worse than commercial standard work (e.g. https://pbs.twimg.com/media/FwWPeNhXoAQZGW8?format=jpg&name=...) -- but of course those would be too easy to classify, wouldn't they? I wouldn't fault anyone for thinking the provided examples are AI.

[viraptor]

> They picked the most (…) the most AI-like human images

Why do you think so? I didn't see that explicitly claimed in the post (or did I miss it?)

[npinsker]

It's my opinion, but... him saying he "[took] prestigious works that had survived the test of time" isn't so believable, when he starts off with something from /r/ImaginaryWarhammer and immediately follows it up with a piece from "an unknown Italian Renaissance painter".

Part of it is he's handicapped by having to avoid famous pieces -- but you can still easily find work that outshines these examples. For digital fantasy, art for card games like Magic: the Gathering. For anime, the art for gachapon games is wonderful. For landscapes, he chose a relatively weak Hudson River School painting, and many have more striking composition and lighting that seem very hard to mistake for AI (e.g. https://collectionapi.metmuseum.org/api/collection/v1/iiif/1...).

[rurp]

Based on what I've empirically seen out in the world most people posting AI art are not using the same filtering as the author of this test. Plus the human choices used probably skew more towards what people think of as classic AI art than all human art as a whole.

The test was interesting to read about, but it didn't really change my mind about AI art in general. It's great for generating stock images and other low engagement works, but terrible as fine art that's meant to engage the user on any non-superficial level.

[majormajor]

> It's the same thing humans do with their own art.

How so? Humans distributed all those "I filtered them out because they were too obvious" AI ones that aren't in the test too. So they passed someone's "is this something that should get released" test.

What we aren't seeing is human-generated art that nobody would confuse with a famous work - which of course there is a lot of out there - but IMO it generally looks "not famous" in very different ways. More "total execution issues" vs detail issues.