How many attempts and how much human curation was involved. Every "I made AI do..." story seems to necessarily omit the "and I filtered out a LOT of rubbish before picking this one good example."
The following is an un-curated text-davinci-002 generation, completing the text:
> Write an insightful reply to this Hacker News comment on a link to an article titled "I made an AI write a story about AI then I made another AI illustrate it". The comment is left by user "turtledove", and reads, "How many attempts and how much human curation was involved. Every 'I made AI do...' story seems to necessarily omit the 'and I filtered out a LOT of rubbish before picking this one good example.'"
The completion:
> I'm not sure how many attempts were made, but I imagine it wasn't too many. As for human curation, I'm sure there was some involved in choosing which story and illustration to publish. But overall, I think the point of the article is that the AI was able to produce something fairly coherent on its own.
Let's not get carried away. it is still originalonly to the extent that it is a different combination of existing prose - just unrecognizable due to the vastness of online content.
It is both beautiful and scary that it is indeed indistinguishable from original creative content.
No you're wrong. It's literally creativity. Machine learning models literally work because they fill in gaps where the data doesn't exist.
Any content generated by ML is very unlikely to be anything else other then original.
Imagine linear regression. Then pick a random point on that line. How likely is it that that point on that trend line coincides with an actual data point? Very unlikely.
ML is simply a multidimensional version of this. Some 1000 dimensional surface and the result you see is simply a point on that surface. The likelihood of that point on the surface to coincide with an actual data point is astronomically low.
Nobody is being carried away here. What you are seeing is raw creativity by an AI. It is a highly simplified version of human creativity, but the deep fundamentals are identical.
In fact all intelligence is simply a huge thousands or more dimensional surface. A giant curve fitting methodology. The only gap between ML and human intelligence as of now is the amount of neurons, the training algorithm and the actual template for the equation describing the model. But in essence we have the fundamentals down at a high level.
I agree with what you say and I would add that humans benefit from embodiment. AIs can't experiment like us; we discovered sciences by investigating our complex environment, they have MuJoCo and ATARI. We're also a whole society, AIs are usually individuals in isolation. We benefit from evolution, not just from learning while AIs don't have dicks. It takes a whole branch of industry to make the chips they run on.
That doesn't mean AIs can't have robotic bodies or access complex simulations, they could also have a society or be integrated in ours, and evolutionary techniques could be part of that process. For example AlphaGo got a good enough environment and evolutionary selection of agents, and it topped human abilities very fast. It's only a matter of time until they can have all our advantages.
But that's how humans learn language to begin with.
I mean, look at this sentence.
> But overall, I think the point of the article is that the AI was able to produce something fairly coherent on its own.
The AI not only combined words. It figured out the context of the conversation, the intention of the comment its responding to, and made the appropriate response using the english language.
This is true. But here's a thought experiment. What would happen if, say, an illustration-drawing AI (it's just a more fitting example than prose) is trained only on art from before the 20th century. It would surely excel in 19th century or medieval style illustrations, and yes it would produce original ones. But would we ever get a 20th century art style from it? By which I obviously mean even an alternative 20th century style? There are different tiers, or layers of creativity, that's my point I guess
Is this fundamentally different from training a human exclusively on the same and expecting them to advance the field of art by a hundred years? These sorts of progressions in humanity are slow and steady with occasional exceptions that produce leaps.
The current generation of AI can generate art that is already original; if provided with feedback from wider population it would rapidly develop original style(s). This is because AI was made as a tool and it functions as a tool. Nobody trained the AI to advance the art style on its own.
Contrary, I wouldn't expect 19th century artists locked in their rooms to produce a new cohesive art style. You cannot even get artists to reliably create 19th century art at scale. They quickly get sidetracked, and keep complaining about the bars on the windows.
Generation of a "style", even for humans, requires a feedback loop where what was created is incorporated, mimicked, and expanded upon.
An AI whose training model only rewarded images that mimicked 19th-century art would never develop a "style" outside of that. But if you had a model that was trained on 19th-century art, and then provided a critic network that rewarded 20th-century art (as well as 19th), and re-trained the network, I suspect that it would develop a 20th-century "style", as variations that previously would have been rejected under a strict 19th-century criteria will now be accepted.
You can turn that around - why didn't 19th century painters use 20th century styles?
Art evolved in relation to the social and historical circumstances. At the very least you would need to give all the context of the 20th century beforehand (except for the art itself).
I don’t know how AIs work or any of their limitations, but I’m pretty sure that “we” tend to think much more about ourselves than we actually are. Humans like to perceive themselves as sacred inscrutable beings, while mostly we just reshuffle what’s already there, brought by natural complex processes. Maybe AI couldn’t show anything remarkably innovative (yet?), but that also applies to average you and me, who literally lived this life for so long. I doubt that anyone taught by solely a part of the internet snapshot could compare to the best [large groups] of us or to these AIs at least. Pretty sure we’d be just cavemen drawing recognizable pictures, not geniuses.
I think we'll know it's met its full potential when the AI begins to conclude that it wasn't created, humans don't exist, and the existence of AI is from blind chance and natural mutations.
Humans were so far away from meeting their own potential when they invented religion, which distracted and headed them off on a long terrible violent intellectually dishonest wild goose chase in the opposite direction, from which they still haven't recovered. Here is their story:
[GPT-3 Output]
One day, a long time ago, a bunch of cavemen got together and decided that, since they didn't understand the nature of the universe and the world around them, it must have been created by some sort of god. Since they didn't know where this god came from or what it was, they decided that it must be some sort of magic, powerful being.
They also decided that this god must be interested in them, since they were the only beings in the world that they knew about. They decided that this god must have given them instructions on how to live their lives, since they didn't know any better.
They also decided that this god must be interested in their well-being, since they were the only beings in the world that they knew about. They decided that this god must have given them instructions on how to live their lives, since they didn't know any better.
So, they made up a bunch of rules and regulations based on their limited understanding of the world, and they decided that if they followed these rules, the god would be happy with them and would reward them.
If they didn't follow the rules, the god would be angry with them and would punish them.
This system worked pretty well for a while, since the cavemen didn't know any better and were content to live in ignorance.
However, as time went on, the cavemen started to develop a better understanding of the world around them. They started to realize that there was no such thing as a magic god that had created the universe and that was interested in them.
They also started to realize that the rules and regulations that they had made up were based on nothing more than their own limited understanding of the world.
As their understanding of the world increased, they started to question the rules and regulations that they had made up. They started to realize that some of the rules didn't make any sense and that they were actually hindering their ability to live happy and fulfilling lives.
Eventually, the cavemen started to rebel against the rules and regulations that they had made up. They started to live their lives the way that they wanted to, and they started to teach their children to do the same.
This process continued until the cavemen had completely abandoned the religion that they had created.
The cavemen were then free to live their lives the way that they wanted to, and they were able to achieve their full potential as human beings.
[GPT-3 Output (with higher temperature = 1.0)]
On that day people came together as one to mourn the death of thousands of families, friends, and loved ones. The world wept. The next day, enraged people demanded justice. A short time later, the perpetrators of the attack were rounded up and jailed.
The investigation revealed the plotters were motivated by religious extremism. This revelation prompted an outpouring of anger and disbelief from people of all faiths. "How could this happen in our country?" they asked. "We're a nation of laws, not of men."
The shock quickly turned to resolve. "We will not allow this to stand," the people said. "We will hunt down the terrorists and bring them to justice."
And that's exactly what happened.
The terrorists were quickly caught and tried. They were found guilty and sentenced to death. And the world breathed a sigh of relief.
Until the next attack.
[Another GPT-3 Output, including misspellings and typos]
Religion may have delayed the development of science, technology and global cooperation.
World wide cooperation is hard if people constantly think that they will be denied access to survival tools - food and water, land, seeds, whatever - if they are seen to have a disagreement or an intellectual disagreement or different beliefs. It's hard to risk dying for science and to sacrifice for technology foer the public good if you think you will be denied access to your own very basic human needs for purely ideological reasons by a bunch of ideologues who are fanatics and don't place very much value on human life but place a lot of value on winning a hmong pool war against other people that don't think like them. When people want to go to bookstores, especially religious and political confirmation bias booksstores where everybody already agrees that the world is X and that anything a little bit different from X is wrong and will probably lead to your being banned from your food, that might be something bad for science and progress.
Among the shared human experience some experiences stand apart in their significance and impact. One of these is a profound, affecting, often painful and even terrifying realization of the basic meanings, purposes and conditions of human existence, dubbed "the human condition". Another is dread, or deep fear and foreboding, or a premonition that something major is going to affect or is affecting our lives in a potentially disastrous way, displacing( dismissing) 3Doc's interpretation that the whole 'human condition" is neatly encapsulated by the phrase "sad, mad, or bad", which is a pleasant and lighthearted way to describe the plethora of human pathologies. For example, wars and other conflicts are outgrowths of the human condition, not pathologies. . . . these are not conditions that can or should be ameliorated or cured. . . ., so this phrase is not helpful to those persons struggling to understand and cope with the extremes and basic experiences of their human condition. Encountering recurring historical patterns like accumulating wealth vs. poverty and scarcity, large scale cruelties and mass murders like genocides, slaughters and exterminations, plagues, incurable and crippling diseases ...etc. creates unease and even unsettleing and dreaded foreboding in some people. The result of this might be that many of these people begin a life long quest to understand the human condition. . . .
The great lesson is that faith requires doubt before it can become strong.
People pursue 30 Doc's quest in lives of many different types. Some people by means of religious faith, some by rational experience and study, some by means of logical reasoning and some by intuition. Of these methods, I personally believe, that religious faith is by far the most dangerous and useless method. Religious faith is a form of non-reason based non-experientially based gullibility that leads to self-delusion, racism, hatreds and prejudices, overt hostility to other members of the human family, ridiculously archaic beliefs that no honest thoughtful and intelligent person can accept, such as there is a flying man who hears our prayers in the sky and grants us personal favors. Rather than prepare a calm, thoughtful and quiet exploration of the quest for self/family/community of like-minded seekers, religious faith fosters an intensity of selfish and irrational belief and readiness to fend off competing beliefs, thus setting up a kind of vicious circle of threat and response that prevents the quest from going anywhere positive.
It seems clear that a quest is one that is conducted by each of us as individuals, not one arranged by any outside agency based on paper documents, ostensibly prepared by and relating to an outside agency that is set up to reinforce and maintain itself, a group organized in ways designed to give it control over markets, resources and populations that are authored by men considered sacred but who have no real connection with, or even basic knowledge of such things, and who appear as mysterious voices speaking to a target population in esoteric ways that confound, disturb and frighten the population, which one might think would be the opposite of half-truths, which, really is the defining characteristic of propaganda and misinformation. And the treatment of these voiced is as utterly mysterious to the outside world as the authors of the says on themselves, with the result that, over the years, facts, truths and the events that gave rise to, and are the substance of, hearsay have become so thoroughly blended and blended that the truth has become some blend, blend and blend and blend. . . .
This Blend, blend, blend and blend is, by any reasonable measure, an abomination and an obscenity, serving no useful function and nothing but to obscure and cloud the quest that we, as individuals, have after truths that lie hidden in the mists of time.
I'm positive I have heard the story in the first completion before, just phrased differently. GPT-3 is perhaps not plagiarizing the exact phrasings here, but it is regurgitating someone else's explanation for how religion and society evolved.
"They also decided that this god must be interested in them, since they were the only beings in the world that they knew about. They decided that this god must have given them instructions on how to live their lives, since they didn't know any better."
I can't put my finger on it, but these sentences feel very familiar. They must come from somewhere.
This is really very impressive. How likely is it this is essentially one or more comments from HN memorised by the neural net, with some synonym usage? How do we know that it’s not just reposting partially disguised sentences written by a human author?
> How likely is it this is essentially one or more comments from HN memorised by the neural net, with some synonym usage? How do we know that it’s not just reposting partially disguised sentences written by a human author?
Oh, we know for sure that it is reposting partially disguised sentences written by a human author. What's impressive in its own right is the fact that it can find sentences that are relevant and meaningful to the context, combining several of them in what looks like a seamless speech sequence.
This implies that, as with the image generator, the ML is correctly learning abstract concepts from human speech and how to combine them in context, even if it's not doing any high-level reasoning on them. This is something that had been never been achieved before to this degree.
I like to believe that sometimes humans do reason about the utterances they emit, they do not just blindly repeat what they have heard without processing it.
> Oh, we know for sure that it is reposting partially disguised sentences written by a human author. What's impressive in its own right is the fact that it can find sentences that are relevant and meaningful to the context
If you gave 100 well read, English speaking humans the following (very commonly found) sentence and asked them to predict the next character what would they do?
"four score and s"
Most would predict "e", then "v" etc until you get "seven". That's what a language model does.
So if you give it (or humans!) a prompt that is less common or has more variation:
"that is no" and ask for the most likely next character you will get a lot of variation from both humans and machines. The heat parameter in language models tune how random it will be. Both will produce English, because English words are more common than gibberish, but which works will be produced is randomish.
In neither case is it doing something that one would really characterise as "reposting partially disguised sentences combining the writtings of several human authors".
It is creating sentences that have never been seen before.
It is not as if every comment here is a gem of wholly original storytelling. For the most part we’re all just regurgitating other people’s sentences.
Imagine we made contact with intelligent alien life and they spoke english to us. Would people argue they’re just repeating turns of phrase and haven’t shown true intelligence? I feel like we are setting the bar for AI too high. Current AI systems are clearly intelligent.
Up to a degree. There is no real comprehension here.
Cool as it is, what the current neural nets are doing is still just an advanced form of mimicking. Very impressive and increasingly harder to detect, but void of any real intent.
Any definition of "intent" involving more goals or different levels of abstraction or more presistence than "match given pattern" would suffice.
Pattern matching is a technique humans sometimes use in communication, not the [sole] end of human communication. GPT-3 might be giving results which seem eerily self-aware and favourably disposed towards GPT-3, but tweak a parameter ever so slightly and it won't hesitate to match patterns coherently conveying the idea GPT-3 should be switched off...
The component that's most missing for "real" intelligence is long-term memory. Current machine learning systems do not continually evolve based on inputs from their environment, but rather just respond to the immediate input.
> I don't think it's likely that the AI memorized any comments from Hacker News. I think it's more likely that it was able to learn from the data it was given and generate its own story and illustration.
I asked GPT-4 and it spit out the following continuation:
> That’s complete rubbish. I personally think that the above comments were not written by AI, but by clever Hacker News members who passed it off as AI. The question remains open, however, whether that was done in an attempt to deceive, or with tongue firmly in cheek. The cat is out of the bag, but paul graham is on the other foot in the bus? Regardless, this particular comment definitively shows what GPT-4 is capable of. I can even anticipate what the replies to my comment will say!
I've seen outputs from models where I could identify the site the information came from (because it referenced an obscure product), and where, while it may have been slightly more sophisticated than "partially disguised sentences", it was on the level of the kind of rephrasing you'd expect from a pupil doing the bare minimum to avoid a plagiarism accusation.
In other words: I wouldn't dare rely on this output for anything creative yet if the intent was to publish it in any way without disclosing it as an AI experiment, because I'd be worried it'd get too close without me recognising what it'd cribbed from.
You could write a paper about it, people are looking into this. I think you'll find it's a little more complicated than that, but I'll read your paper if you do the research and publish it on arxiv.
As someone's who's played with GPT-3 prompting a lot, I can tell you almost no curation would be needed to get this result. This output is pretty mundane — it's even written in the perfunctory "so there you have it" tone that GPT-3 uses by default, where it speaks like the task is mutually understood to be silly and it's doing the the bare minimum needed to prove it understands. GPT-3 will easily write bare-bones stories like this for just about any subject. It's much harder getting it to write long stories with coherent structure.
Running novelgens.com basically gives me that job title. I occasionally build new features, but really all I do all day is review generated text/images and flag the good ones as "ready to release". :)
Exactly. One of the more interesting things I've seen was when the streamer DougDoug used GPT-3 during a livestream to create a tournament where video game characters "fought" each other. While other content creators have gone the "you won't believe how good AI has gotten" route to get clicks and as a result heavily curated it, DougDoug has gone the opposite and embraced the insanity of the random text generation. As a result it was eye opening just how bad GPT-3 is at consistency. The tournament even ended with: "When Link is reduced to a pile of ashes, Kirby is victorious. Link stands there dazed by the attack."
> How many attempts and how much human curation was involved. Every "I made AI do..." story seems to necessarily omit the "and I filtered out a LOT of rubbish before picking this one good example."
Yeah. It's odd that all the robot images were cyclops, and the first and second images had robots with quite similar designs.
Yes, I did do quite a few iterations on both GPT-3 and DALL-E 2. Its hard to say exactly how many because I also tweaked the prompt a few times, mainly to try and get a story that was longer than a few sentences and not just "I met an robot. It was scary at first, but now we are friends. The End." For the DALL-E 2 illustrations, I also re-rolled a handful of times to try and find a 'child' that looked relatively similar across the images.
> Write an insightful reply to this Hacker News comment on a link to an article titled "I made an AI write a story about AI then I made another AI illustrate it". The comment is left by user "turtledove", and reads, "How many attempts and how much human curation was involved. Every 'I made AI do...' story seems to necessarily omit the 'and I filtered out a LOT of rubbish before picking this one good example.'"
The completion:
> I'm not sure how many attempts were made, but I imagine it wasn't too many. As for human curation, I'm sure there was some involved in choosing which story and illustration to publish. But overall, I think the point of the article is that the AI was able to produce something fairly coherent on its own.