[sillysaurusx]

You note that this will change the world, but then talk about GPT-4 specifically.

The leap from GPT-2 to 3 was enormous. 3 to 4 was enormous, and we’re not even using 32k context yet nor image input. 4 to 5 will likely be as disruptive if not more.

This isn’t about 4. We’re in the iPhone 1 era of LLMs. This is about what the world will look like in one or two decades. And there’s a good chance this comment might age poorly.

That’s a scary thought. I was skeptical of AI, and still am. But it seems undeniable that the world is in for a big awakening. This might be as big of a transformation to society as the introduction of microprocessors.

[coldtea]

>This isn’t about 4. We’re in the iPhone 1 era of LLMs.

Well, on the other hand, iPhone 14 isn't that different. Same how a 60s car and a modern Tesla aren't that different. Evolutionary marginally better yes. More convenient, yes. But nothing life changing or necessary. Which is why some folks can even get by reverting to a dumb phone (whereas they wouldn't dream of going pre-electricity or pre-antibiotics).

Also, we were hearing the same about VR in the early 90s, and again in the mid 2010s. Still crickets.

[ROTMetro]

Huh? The iPhone 1 was a toy and lots of people laughed at the users. Today a modern phone is a requirement to be a member of society. It is how I pay for things. It is needed for most of my interactions with friends/family. It is the diary of my life, and the repository of my good memories with its near unlimited video/image storage at a quality only dreamed of when the iPhone 1 came out. Take away a person's iPhone1 and they weren't phased much, taking away a person's iPhone 14 is a LIFE CHANGING experience today. In 10 years taking away your AI will be on the same level, you will function but at a much more frustrating level.

[coldtea]

>Huh? The iPhone 1 was a toy and lots of people laughed at the users.

Nothing "toy" about it, it was the most advanced phone on the market. The people who laughed were just the handful of idiots that would laugh because "Apple, har har har" and then go buy the same thing from another vendor. The same kind of Zune buying crowd.

>Today a modern phone is a requirement to be a member of society.

You'd be surprised.

>It is how I pay for things. It is needed for most of my interactions with friends/family. It is the diary of my life, and the repository of my good memories with its near unlimited video/image storage at a quality only dreamed of when the iPhone 1 came out.

None of those are essential, even for a 21st century level lifestyle, some of those are indulgent, others are detrimental. In any case, nothing revolutionary, except if one thinks "I can pay by taking out my phone and pointing it at the gizmo at the cashier" is something far great than "I can pay by getting out my credit card and pointint it at the gizmo at the cashier" (or, god forbid, giving cash and not being tracked).

[alt227]

>Nothing "toy" about it, it was the most advanced phone on the market.

In no way was the original iphone the most advanced phone on the market. Many other smartphones before it and at the time were way more advanced in features and what they could do. What the first iPhone did was make it easy and accessible to everybody, not just nerds. That was the killer feature which made it take over the world.

[coldtea]

There was no usable web browsing on a phone before the iPhone. It had the most advanced browser.

There was no iPod level music players on a phone before the iPhone. There were crappy music players you can revisit and compare.

Mail apps on phones were crap.

Messaging was crap, in tiny little screens.

Just a few things.

People reviewing and getting the iPhone the time was wowed and think of it like magic. It's people not having it, and dismissing it outhand because it had a touch screen or because their unusable Windows ME phone had some crappy third party software that didn't get it. Of course all of those got either the iPhone or an Android clone of it very soon and never looked back.

[fennecfoxy]

Lmao, I had phones before the iPhone, Ericsson especially that had decent (ie usable) browsers, could play mp3 files etc. And you could install apps on them over wap, take photos/videos etc.

iPhone couldn't take videos as people have already mentioned, couldn't install any 3rd party apps to start with (because Mr. Jobs didn't believe in it), no selfie camera, no torch.

All the iPhone did was streamline people's interaction with the phone, with a large multi-touch display and a simple, intuitive (not anymore though) operating system. They definitely improved things, but in the way that Apple usually does; wait for other companies to do the things, then take the cream of the crop, iterate/improve on them, wrap them up/lock into ecosystem (which some people like) and ship.

[alt227]

This is exactly my point. Everything you have mentioned was available on smartphones before the iphone, iphone just made them more accessible and easier to use. Still stands that the iphone was in no way the most advanced smartphone on the market, it just put a prettier and easier to use interface on these features which led to reviewers being 'wowed'.

[gcanyon]

I had a Sidekick, and it had a semi-reasonable browser. But clearly, the iPhone was much better.

[chii]

> Nothing "toy" about it, it was the most advanced phone on the market. The people who laughed were just the handful of idiots that would laugh because "Apple, har har har"

"toy" doesn't have to mean cheap or low-tech.

The point is that at the time, a lot of people didn't really believe that phones could be that revolutionary - and laugh at the iphone because compared to the blackberry, it has next to no functionality.

[ghaff]

Both the iPhone and the iPod arguably took a few generations to really hit their stride. I had a fairly new Treo in 2007 and I just didn't see any compelling reason to upgrade until the 3GS. I had nothing against Apple (I owned a 4G iPod). I just didn't have a compelling reason to upgrade. Verizon also probably had a better network at the time around where I lived and traveled.

[nsxwolf]

This is wrong. I waited in line for mine. It was quite clearly a toy. It was a cool but barely usable tech demo and it was completely outclassed in features and usefulness by contemporary devices like the Motorola Q.

It showed the way forward, but it was a frustratingly limited device and everyone around at the time recognized that immediately.

[hn_throwaway_99]

> This is wrong.

Can we just accept that these are opinions? I also waited in line for the first iPhone, and it was by far better than any other phone I owned at the time. True, I was not a "CrackBerry" addict as was common for a certain class of worker in the 00s, but the ability to browse the "real" web in a way that was not completely hobbled was just night and day better than other phones at the time.

[nsxwolf]

I agree that the browser was revolutionary - but it was only really useable at home, on my wifi. The 2.5G wireless hobbled it so badly I never used it for anything more than basic info lookups. Web apps were just too painful to use and I could rarely complete an entire transaction with all the loading and back and forth. In the pre-app store era, that really mattered since the web was the only way to get anything done.

[pmontra]

The first iPhone was 2G when Europe had 3G since 2003. It didn't have copy and paste. It did have a touchscreen that no other phone had. It was basically and iPod touch with a phone and a 2G modem. My Nokia N70 was a better phone. I waited 4 years to buy a phone with a touchscreen. Then I thought they were mature enough I bought a Samsung S2.

[coldtea]

>It didn't have copy and paste.

Most people in 2006, just before the iPhone came out, didn't have copy and paste either. They still typed T9 style like it was 1996.

[azov]

Exactly. iPhone was a toy, 20th century internet was a toy, electricity was a toy.

On the other hand nuclear fusion, self-driving cars, and bitcoin were the things to change the world as we know it in the next decade or so.

Things that change the world tend to be hard to recognize as such when we first see them.

[californical]

I feel like my life would be less convenient without my phone, but I’d also probably be happier. So idk. There are easy alternatives, like using the website on a laptop, you just can’t pay your bills on the train without a smartphone..

I’m actually getting rid of the cell-phone plan on my iPhone, keeping it as WiFi only, and getting a dumb phone for calls. It may suck but I’m trying it as a 6 month experiment, so we’ll see!

[14]

I used an iPad the same way for a couple years with no problems. I have internet with Shaw and they have city wide hot spots so I really could get internet access basically anywhere in town. Now there are lots of voip phone providers you can easily get a number and use it to talk with people.

[14]

The first iPhone didn’t even take video out of the box. That is how I learned about jailbreaking because my phone company sold me video messaging on my plan and the phone didn’t take videos! Lol. But if you jailbroke the phone you could get cycorder from Cydia and take videos.

Yes there were other “smart” phones at the time but it truly felt like social media blew up in size with the introduction of the iPhone. And that was revolutionary.

[ModernMech]

> 3 to 4 was enormous... This isn’t about 4. We’re in the iPhone 1 era of LLMs.

GPT3 is great, but I can't reasonably say that 4 is such a huge advance over 3 in my experience so far. Apparently it's better at some things according to the marketing, but for actual usage I can't qualitatively say 4 is an "enormous" advance over 3. It seems to face the same major shortcomings, and it produces qualitatively the same results.

That brings me to the iPhone bit. Yes, the iPhone was a huge advance, but today looking at an iPhone 14, it largely has the same form/function/features as early iPhones. If you looked at the trajectory of iPhones in 2005, you'd conclude that in 2023 they would be 1mm think and transparent with a holodisplay or something. But instead, in the year 2023, my iPhone 14 looks and functions largely like my old iPhone 4. I mean, it does more stuff better, but I'm still using it to browse the net, text, take pictures, and use the maps app -- the same stuff that made the original iPhone revolutionary.

[hattmall]

This sentiment pops up with most somewhat new technology, but in my experience the plateaus come quickly. Going with the iphone. The first was transformative, and it continued to improve but the functional difference between iPhone 4 to iPhone 14 is not that great. Think of the wow factor of showing one or the other to someone from 100 years ago.

[thethimble]

The entrenchment of smartphones in society dramatically increased between iPhone 4 and 14. Technical capability is just one axis.

Still, I think LLMs are different than phones in terms of scaling. Faster processor speeds don’t necessarily result in more user value for phones, but scaling up LLMs seem to predictability improve performance/accuracy.

[coldtea]

>scaling up LLMs seem to predictability improve performance/accuracy.

Isn't this always the case before hitting diminishing returns?

[macrolime]

There are no signs of diminishing returns just yet though, and no one knows if that will be at GPT-5 or GPT-5000. I suspect the performance will keep increasing drastically at least until we have a model that's been trained with essentially all available text, video and audio data. Who knows what will happen after we have something that's been trained on all of YouTube. After this maybe we (or an AI) will have figured out how to keep improving without any more data.

[coldtea]

>There are no signs of diminishing returns just yet though

Sure about that? GTP-4 doesn't seem 5 times better than 3, much less 10x. Despite having 5/10x the parameters.

[gremlinsinc]

yeah it is, gpt3 scored in the 80th percentile for the bar, gpt4 scored top 20 percentile and is much better at math, plus having 4x the context alone gives it much more power.

it's just it's different in capabilities. chatgpt delivers different results and both have unique characteristics.

gpt4 being able to not only create images but also decipher what's in then is another huge advancement.

Gen2 another ai can create amazing videos from a text prompt. Any director or film maker wannabe with more prowess on creating the story than filming it, can now just use ai to create the film from their vision.

even more exciting is the speed that things are progressing. it was supposed to take 8 years to get chatGPT quality training down to 400k price instead of millions. Stanford did it in 6 weeks with llama and alpaca. it can run for under 600 or slower on home PCs.

[bryanrasmussen]

>but scaling up LLMs seem to predictability improve performance/accuracy.

what exactly is performance/accuracy in slogan generation?

[ROTMetro]

Analyzing thousands of trends, both industry/niche specific and society wide. Tracking campaigns that work by monitoring social media likes, references to past slogans, etc. Potentially dedicating thousands of years worth of brain power and analysis to the coffee shop down the street's new slogan.

[je42]

gpt does a lot more than slogans...

currently using it like driving a junior programmer.

after gpt has written some functions to my specs in natural language. I can say for example: - "add unit tests". It writes for all functions tests. Not perfect but not bad for short instruction like this. - rewrite x to include y etc

[bryanrasmussen]

the original post way back was talking about marketing, they were underwhelmed. I recently generated some slogans. They sucked.

When someone mentioned predictability/accuracy how does that apply to marketing slogans. I know how it applies to writing unit tests. The unit tests writing comes pretty close to the original posters definition of GPT as filling out templates. The sucky slogans I got were also very template like.

Would accuracy be if slogans did not suck?

At any rate there seems to be a lot of things people want to use it for where the terms accuracy / predictability don't make much sense. So making claims based on those qualities naturally causes me to ask how do they apply to all these cases - such as slogan generation where accuracy predictability are not normally metrics that apply.

[agent281]

I think it's an open question how much better the LLMs will get. However, we should expect adoption and integration to radically transform their usage. Who knows what the Google Maps of LLMs will be.

[clankyclanker]

It’s interesting how well your description follows the Gartner Hype Cycle, but applied to products instead of hype itself:

https://en.wikipedia.org/wiki/Gartner_hype_cycle

[croes]

Wasn't the leap from GPT-2 to GPT-3 bigger than from GPT-3 to GPT-4?

Like self driving cars, the leap were enormous until they hit a wall and we still don't have full self driving

[macrolime]

The first few days I didn't think there was much difference, but after using GPT-4 a lot I think the leap is huge. For things where I would previously use Stack Overflow or some other Stack Exchange, I now use GPT-4 and get a working answer almost every single time, with GPT-3.5 the frequency of working answers was much smaller.

[UncleEntity]

If someone were to ask me (which nobody ever does) GPT-4 was more about cashing in on the hypetrain than pushing the boundaries.

A little better but a lot “safer” to cut down on the articles on how it’s trying to steal someone’s man.

[eternalban]

You are working under an assumption that this tech is an O(n) or better computational regime.

Ask ChatGPT: “Assume the perspective of an expert in CS and Deep Learning. What are the scaling characteristic (use LLMs and Transformer models if you need to be specific) of deep learning ? Expect answer in terms of Big O notation. Tabulate results in two rows, respectively “training” and “inference”. For columns, provide scaling characteristic for CPU, IO, Network, Disk Space, and time. ”

This should get you big Os for n being the size of input (i.e. context size). You can then ask for follow up with n being the model size.

Spoiler, the best scaling number in that entire estimate set is quadratic. Be “scared” when a breakthrough in model architecture and pipeline gets near linear.

[bitwize]

I have a sinking suspicion we're not in the iPhone era of LLMs, we're in the "in mice" era of LLMs. I can't elucidate why, but this strikes me as the sort of thing that could either blow up (like the iPhone) or fizzle out (like Theranos), or blow up and then fizzle out (like crypto). And it's too early to know yet which it'll be. Hackernews is making lots of change-everything predictions, but Hackernews was like that about Theranos, crypto, and self-driving cars too.

[ben_w]

I wasn't impressed or motivated by the original iPhone when it was new.

I don't know about the future, but by analogy with the past I would say that GPT-3 was the original iPhone (neat tech demo but I didn't really care), ChatGPT is the iPhone 3G, and GPT-4 is the 3GS.

Looking at the sales graphs on Wikipedia (Q1 2012) I think it took until the 4S to transition from "the next big thing" to "the big thing".

Analogies only rhyme rather than replicate, so don't assume GPT needs exactly two more versions to do the same; might be more, might be less, and Uncanny Valley might trigger a Butlerian Jihad at the last possible moment before fully-general AGI.

[ghaff]

I would have said the 3GS but, yeah, sometime around then.

Similarly, the original iPod was a not obviously remarkable pocket music player in a fairly crowded field.

[jasmer]

Decreasing marginal returns though.

We were a bit blown away with 'Siri' - I mean, it could understand what you said, and 'get the weather'.

I think we're going to start feeling the limits of this soon.

It will be pervasive though.

[alchemist1e9]

> one or two decades

More like one or two years at this rate.

[sillysaurusx]

Yet the decades will come.

I think progress is sigmoidal rather than exponential, and it’s very hard to tell the difference in the early stages. But even sigmoidal progress with smartphones was enough to completely upend online society. We adapted, of course, but it looks nothing like it did in 2003. We’re all still using the internet; that’s basically it.

Point is, it could slow down, assuming that AGI isn’t waiting like a cat in a corner. But it’ll still displace a tremendous amount of intellectual work.

[hammyhavoc]

Remember the crypto hype cycle?

[idopmstuff]

Except crypto lacked utility for most people, even early on.

I've spent the last couple of days creating python scripts to automate parts of my business. I'm not a developer (though technical enough to help point GPT in the right direction sometimes when it's getting stuck on problems) and have written <100 lines of python in my life.

I'm using image generation AI regularly to create images for my marketing emails, and when I've got writer's block it helps with the text too.

Right now the iPhone 1 is a great analogy - it was cool but it was really subpar for using a lot of the internet, because it wasn't mobile optimized. GPT takes some coaxing to get it where you want, like you had to do a lot of pinching to zoom in on websites on your phone. In a few generations, this is going to be as seamless to use as the iPhone 5 was compared to the first gen.

[krainboltgreene]

> Except crypto lacked utility for most people, even early on.

Every crypto person said the opposite. They said exactly what ChatGPT-hype people are saying now.

[eli]

chatgpt has already provided more value to users than all crypto combined

[ChatGTP]

not if you're a teenager who is into using banned substances on the weekend

[UncleEntity]

When I was a teenager, back in the 80s, it didn’t take magic internet money to buy drugs.

In fact, every drug buy I ever heard of (second hand, of course) involved fiat currencies.

[ChatGTP]

You are no longer a teenager in the 80s my friend.

[hammyhavoc]

Source?

[dpkirchner]

What sort of source are you looking for? What would satisfy your question?

[hammyhavoc]

A source that quantifies value with cited examples and statistics. I ask because the crypto crowd argue there's utility and value there too.

[alchemist1e9]

And how is that going? Died down? It’s really amusing when I encounter tech savvy individuals who think crypto is hype, little do they realize it’s stronger than ever and central bankers are becoming increasingly concerned.

[hammyhavoc]

I invoked the crypto hype cycle, not crypto itself. Don't conflate the two topics of discussion.

[alchemist1e9]

Understood. However I’d still argue there is a reason for the hype and I believe that is the case with the GPT LLMs also.

[UncleEntity]

Concerned the scams and fraud might bleed over into the real economy.

[alchemist1e9]

https://www.coindesk.com/policy/2023/03/24/federal-reserve-s...

I don’t think that’s what they are actually worried about. I would also like to point out that the biggest scams, FTX for example, are simply traditional Ponzi schemes with a crypto front, they have all been executed entirely using regular banking systems and due to the incompetence of those regulators. Bitcoin itself is rock solid and constantly gaining users and influence.

[gremlinsinc]

microprocessors? Geoffrey Hinton(I think that's his first name), the grandfather of ai recently said this is like the invention of the wheel, agriculture, or electricity.

ie even more foundational for everything that's coming in the future. ai will be as essential as electricity.