Software expands to fill the available resources. If you want more efficient software, build it on less powerful hardware. AI training runs are no exception!
Can't agree more. Especially as I wait to download 1.3gb file to update some Windows driver like Realtek audio or LED display, and you realise a whole effing Debian operating system with Xfce & apps can fit in a lot less.
Whoever are downvoting this really don't understand that expanding a congested highway merely widens the congestion instead of alleviating it.
Software today require several dozen gigabytes of RAM and two dozen CPU cores just to render some simple fucking text because everyone has several dozen gigabytes of RAM and two dozen CPU cores.
The bloat is real, but it’s not all bloat. We ask for a lot more from our software today. Simple example: rendering text isn’t that simple when we expect full Unicode support.
It’s not altogether different from how everything seems to be much more expensive than it was in the 1950s. There are many real and concerning reasons for that, but one less concerning reason is that we have higher standards and expectations now, and that translates to higher cost.
I really would like to know what's good bloat and bad bloat like they say with Taliban.
To give a random example, Apache http server for windows is less than 12mb download. I used to download Oracle's http which I'm told is based on that, but that's like 1GB++ haven't checked in recent years.
This is one of the biggest reasons why China or India are threat to the West.
Because their Engineers are constantly in situations were they have to work with much less resources. And they come up with stuff that constantly surprises people for being much cheaper. The western reaction is to then go and buy them out. Its not going to work forever. They have already taken over large swathes of the tech landscape.
Even with AI for example, Real Time inference is not required in majority of cases.
But the western engineer and corp exec in big tech, are used to buying new data centers everyday (not because of specific Customer Demand, but because of intent to capture market/snuff out competition/build moats or what ever bs an environment of over abundance has trained them to do) and then they have zillions of machines idling which they use to provide real-time inference raising the cost for everyone.
Sooner or later someone in China or India working on non real time inference will offer large corps much cheaper solutions.
The Western model is where you forget how to cook a meal at home, and end up relying on McDonalds for food, because they are everywhere, thanks to their prime directive of survival via opening new stores everyday.
For how much we all bitch about beancounters, I'm kind of surprised that beancounters haven't managed to convince management to buy bottom-of-the-barrel Celeron laptops for software developers.
Make those fucking assholes use the hardware of the people instead of some monstrosity with a 256 cores CPU and 48 TB of RAM and 24 exabytes of enterprise SSD and an RTX Cinco Grande connected via optical fiber to the Amazonflare Cloudnet. We will see lean and mean software literally overnight.
> Make those fucking assholes use the hardware of the people
Are you sure it’s the engineers who are being complacent? I know quite a few coders who’d love nothing more than to spend months on optimizations eking out small performance gains. Their bosses (or the market) don’t let them.
Because shipping fast almost always beats speed/memory optimization. The only reason to even do performance optimization is if it is a constraint on the problem itself (e.g. you work in gaming or HFT).
If shipping fast means you spend an extra $1000 once per developer, guess what almost every company is going to do?
I have access to a couple of Macbook Pros and a Linux machine with very good processors, but I do my hobby development all in an old Macbook Air. That allows me to see when my code runs slowly much more clearly. On an M1, everything just runs so fast you don't notice when you have a serious performance issue which is painfully obvious on the Macbook Air! I recommend doing that, though it does require patience waiting for things that are instant on the better machines.
It should be possible to achieve the same end by forcing management to use the "hardware of the people", getting them to do nontrivial on-call duty frequently, and making them dogfood what they make.
The real fucking assholes are the ones that enable the fucking assholes you refer to.
I use "monster" desktops for my development. I appreciate luxury of language servers, proper syntax highlighting, smart complete etc. etc. Output results however are absolutely tiny (for what they do) single executable enterprise backends, firmware for microcontrollers etc. etc. Also produce desktop same style software. Runs fine on ancient computers.
So there is a use for fat development stations. Increases my productivity which is very important as I am an independent vendor and pay for all my tools.
One area where I'll be worried about China (not sure India as it's following USA in many ways) is they needn't spend time, resources on ensuring political compliance. Maybe yes in a different sense but not in the same sense as in returning "correct" answers for questions. We saw that with Google.
That should knock God knows how many GPU cycles, time and training of models.
I'm always baffled by this "forget how to cook at home" example, but I suspect this is more about having fast food within walking distance, while not having a grocery store within walking distance ??
Think it's not simple China or India issue. Even in those countries, new gen kids grow up with too much not too little, depending of course on who their dads are.
Even with latest hardware, there must be incentives to optimise on size, memory, CPU time etc. But given natural tendency, we just optimise on what's the rarest of them all - time to market. Get it shipping fast.
I was once in a training (as new consultant) for software, the instructor, an old fashioned guy said quit using the mouse learn the keyboard shortcuts - your customer is watching you. He was damn right cos years later, I realised so many customers did remark, how do you do that so fast? To this day, running some command in Excel (say) access Name manager, I use kB shortcut that hasn't changed in decades. I frankly even lost track of where to find them in the menu bar.
Thanks..Among many other issues, it is also a serious crippling factor for communities and countries that are low on budget/resources & a disaster for recycling, climate change etc etc.
I just shoved the whole .webvtt file in the header of a audio Response from the server so that I don't have to implement another API just for subtitles [1][2]
> Software expands to fill the available resources.
To make an analogy, that is why I think even with AI work expands to fill available resources (human+AI). I don't think jobless rate will be high, instead we will see demand expansion.
Based on the ones I have worked with and the news and commments I read on HN I do not consider "software engineers" to be professionals. I have personally seen how they waste enormous amounts of time.
Not to the volume needed to compete with the training infrastructure setups of Anthropic or OpenAI or other leading players
No ban is perfect, there is always some loopholes or illegal exports this is to be expected, but if it prevents large scale transaction then it it is achieved its goal.
The question is rather do they we need a lot of gpus to train or training with older gen gpus is not competitive is a different problem.
> if it prevents large scale transaction then it it is achieved its goal.
It doesn't prevent the transactions, it only makes them more expensive than they would have been otherwise. If the the US was able to covertly buy enough titanium from the USSR for the SR-71 program, China can buy the latest GPUs if it believes AI competence is in its national interests.
Oh look, there's sudden demand for H100s of by dozens of small companies in Brazil, Hong Kong, Vietnam, Indonesia, Singapore, Malaysia and Thailand. I better not look too closely at them or I won't get my sales bonus for this quarter.
Models like Llama 3 are trained on sixteen thousand GPUs, OpenAI probably 25k-100k GPUs. This is the kind of scale the sanctions make a lot harder to achieve.
> This is the kind of scale the sanctions make a lot harder to achieve
Harder and more expensive, but far from impossible. I doubt they pay more than doubles Nvidia's sticker price, all told. My comment was inspired by recent real-life events; Nvidia got into legal trouble in the last couple of months for turning a blind eye to questionable transactions - if you're curious about the mechanics of GPU sanctions-busting, read up on the governments accusations against Nvidia, and this was for low-hanging fruit.
The article has one analyst speculating they used tens of thousands of H100s (50,000 IIRC) instead of the 10,000 A100s the Deepseek CEO owns up to. They can afford to pay exorbitant markups for the logistical nightmare of importation through 3rd party countries at scale.
edit: AFAIK, the sanctions don't prevent Chinese AI labs from renting GPUs from any cloud provider. To simplify logistics, a shell company could avoid shipping the cards to the mainland by simply settings up a data center in not-China and give the parent company full access. I suppose the US government has to balance sanctions against Nvidia's share price, so they can't be too aggressive, there are just too many loopholes for demanded shock not to have been a consideration.
That really sounds plausible. They could open a training farm in Vietnam or Mongolia (or even Taiwan, or really anywhere the internet goes) and just use it, the GPUs don’t need to be located in the mainland. The only way to lock down the GPUs would be to just restrict completely to who could use them, and then prevent them from contracting with sanctioned entities.
When I was working in Beijing, we definitely had resources we couldn’t access locally but could easily access remotely so it didn’t really matter.
The Information claimed today that ByteDance is renting GPUs in the cloud, although ByteDance denies it (well, they call it "inaccurate" which is not exactly a strong rebuttal).
You are under-estimating the powers of the “black market”. North Korea and Cuba are shithole countries because their governments are shit and not solely because of sanctions.
If you follow the news, several people (bankers) will trade with Iran despite the repercussions (jail) of doing so. There is a premium but at the right price someone will execute.
I understand all that - my point is the price is nowhere close to being too high for what's (allegedly) at stake according to AGI zealots, not to mention where the Chinese government sees the role of China world affairs in the coming decades. Hell, I even ball-parked the all-in landed price to at most 2x Nvidia's MSRP.
Kai-Fu Lee describes the culture so well in AI Superpowers. The roots are well before GPU restriction. Absolute cut throat competition.
Imagine Sam Altman throwing a chair out a window in a meeting lol.
The message of AI Superpowers is that China will lag the US at first but once things stabilize this will happen because China has a lot more engineers and a lot more data.
Anyone who hasn't read AI Superpowers should really make it a point to read it in 2025. It is an incredible book.
I don't know, I've been hearing the story that China is about to upend the US as the leading global superpower ever since I was a kid. There's always a new vogue and novel twist put on the rationale and how it's gonna happen, but so far it's like fusion, always a few years away.
It's literally happening lol. When you were a kid China was making shoes and their GDP is 10% of the US. Now they're making drones / evs / high end electornics and it's 80%. This is why people's perception is so unreliable because it's impossible to notice things when they happen over a lifetime
And now they're at the point where the population pyramid is collapsing. It's hard to make any predictions about the future when they got here riding a baby boom and now their ratio of elderly to working age is about to go through the roof.
Japan, Germany, Italy, and numerous other countries are all much worse off. Assuming things don't change, China in 2050 will be in approximately the same position as Japan today.
Germany and Italy, to take but two examples of large Western economies, haven't had native above-replacement TFR since ~1970.
Even in the US, TFR is well below replacement right now, and in fact is basically comparable to China's TFR from 2010-2017.
> Assuming things don't change, China in 2050 will be in approximately the same position as Japan today.
What metric do you have in mind here? It looks like Japan’s population has dropped ~2% from peak in the last 25 years. China is projected to lose ~8% in the next 25.
Or if we look at percent of population over 65, Japan is at ~30% today. China is projected to jump from <20% to 40% by 2050.
Unalloyed good or not, it's way better than where China is at. And I'm not sure why "other countries are going to be even worse off soon" is an argument in favor of China being on the verge of surpassing as a superpower the one country on your list that is actually growing in population.
You’re conveniently leaving out that that China is still a middle-income country.
It can bear the burden of developing and producing things like solar panels or batteries or AI because of the sheer size of the population, but that doesn’t work if the population itself is the problem.
When I was a kid, China was a lot better integrated with the international community. Right now their relationships are far and few between, rarely featuring first-world nations.
If Russia couldn't beat NATO in a pitched fight against the rest of the world, neither can China.
> When I was a kid, China was a lot better integrated with the international community. Right now their relationships are far and few between, rarely featuring first-world nations.
Maybe in western media depictions it would seem so, but eg china invests more and more in europe over the years. Moreover, BRICS are roughly half of world's population. Perceptions of what "the world" or "international relationships" mean are sometimes distorted in the west.
If it's a democracy and reasonably friendly to the USA, it's not a big deal. In the 1990s everyone thought Japan was going to be the next superpower and that was just fine.
The question in a rational mind is, why would it even bother? US/China partnership is the most economically successful in world history, even more so than US/UK or AUKUS. But the downside of CCP government structure is that paranoia at the top ranks has a good probability of overruling rationality.
Albeit US cannot speak as US-centric paranoia/"exceptionalism" may do the same thing...and the electorate voted to self destruct the government despite US economy being the strongest in decades.
The vast majority of its people do not share in that success and have seen a declining standard of life relative to prior generations whereas in China, the opposite is quite demonstrably true, despite increasingly similar concentrations of wealth and political power.
>"If Russia couldn't beat NATO in a pitched fight against the rest of the world, neither can China."
The "rest of the world" is not limited by NATO. And China is not fighting "the rest of the world". It trades with it, invests and does all kinds of other things. But sure keeping one's head in a sand is a nice position.
China does have a current advantage on lithium battery and rare earth materials - dumb technologies that US and allies can replicate fairly quickly, less than a year. EUV and 3nm and below on the other hand, will take decades, since it involves a number of different and deep technologies controlled by dozens of companies. China has thrown $150B on it since 2014, and has only come up with low yield/unprofitable 7nm via existing DUV machines.
> 80% GDP
China's demographics will more than HALF to 500M by 2100, if not earlier, while US grows to close to 400M by then. Someone actually theorizes that China's population is already only 800M right now https://www.youtube.com/watch?v=fR5F_8dSjOw
Also, a lot of that GDP is debatable in 2024, when real estate prices have dropped by more than 50% in tier 2 and below cities, and deflation has raged on.
> when real estate prices have dropped by more than 50% in tier 2 and below cities, and deflation has raged on.
Can other economies copy that part? I know a bunch of people who'd like to be able to afford more houses & more groceries at the same time. I'd like that, I can't realistically afford a house in the city I live in without a 50% price drop.
I'm sure China has a lot of problems, but key goods getting cheaper is not one of them. What I'm guessing you meant to say is that retirees were led to put too much of their savings into the housing market and are discovering there is a glut. Which is tragic for them. But prices dropping is a good thing; the unachievable ideal is a utopia where everything is free, ie, 100% deflation.
China's real estate prices having dropped 50% or more has been accompanied by/caused by wages being slashed 50% or more, and increasing unemployment rate, such as 30%+ for youth.
Here are some good posts on why nobody wants deflation:
It's commonly referred to as a deflationary spiral because the falling prices lead to people (perhaps counterintuitively) holding off from large purchases, anticipating a continued drop in prices. Sort of a "buy the bottom" mentality.
The lack of spending then further contributes to falling prices, job cuts, businesses closing, etc. It's really not a situation any economy _wants_.
I think peak propaganda and manipulation was convincing people that inflation is good for the economy and therefore for them. Imagine prices constantly dropping and your money buys more than ever, uhm, like the deflation happening in tech. That would be very bad for the economy. The irony is that the best performing sector has been the one deflating the most.
So why aren't US and allies demonstrably replicating EVs (and other kinds of green technology) quickly? Tesla is still pretty much the only serious player. Why are CEOs of major western carmakers painting a very different picture than what you describe here? Where are the serious EU/US battery makers that are globally competitive? It looks to me like the EU has chosen the worst of all options: put up tarriff barriers while also not having serious domestic EV makers, and also not stimulating domestic EV development.
While the share of services in the US GDP is more than 3/4. What will you do with all these expensive NY lawyers when push comes to shove? Sue China's drones?
If you want to look at an objective numeric metric for this, why not foreign military bases? US has 128+, China had ~2. To project global military power China will need similar order of magnitude presence. I use that number as a check against sometimes breathless and sensational journalism about the topic.
It’s harder for me to come up with a simpler metric for “Belt and Road” / IMF style control-through-capital.
But, I think it will happen. After visiting China and seeing how much consistent progress both in infrastructure from the government and in daily life from the economy, my impression is US government makes 2 steps forward 1 step back in the same time it takes China to take 100 steps forward.
I was of the same viewpoint as you - just look at the militaries!
Except in today's world, being a military power is increasingly less relevant after a certain point, while economic supremacy is increasingly gaining prominence. While the West is content with self-platitudes for their "democracy", China has been building strong relationships with a number of countries looking to implement the "China-model", a capitalist but largely regressive nation that relies on surveillance and stringent media control. China is already licensing out their technology to a number of interested countries, some of which include Western countries looking to emulate Chinese autocracy themselves. On the other hand, countries are looking at the incoming US govt with pretty much strong uncertainty as to what their relationship with America will be like.
Not to mention, as automated warfare becomes increasingly more relevant, guess where these countries are buying their drones from? Hint hint, it's not the US with their overpriced toys.
> China has been building strong relationships with a number of countries
Number of irrelevant countries. US's allies are Europe, Japan, South Korea, Taiwan, Canada, Mexico, Australia, etc. 80% of the world's wealth. and 95% of the world's top technologies.
> guess where these countries are buying their drones from
Soon, not China. China Is Cutting Off Drone Supplies Critical to Ukraine War Effort [1]. China is reportedly making drones for Russia instead, according to multiple intelligence officials.
According to the World Economic Forum[1], the USA plus the whole EU make up 29% of the world's GDP-PPP, while the BRICS countries come up at 37.3%.
You also included Japan, Australia, South Korea, Canada and Taiwan, but I find it hard to believe they would make up a very significant part of the difference.
On the former point, all the wealthy countries are in team USA, although team USA seems intent on shooting itself on the foot multiple times.
Team China is not as tight as team USA, but has a number of strings it can pull with its members. These are countries that, while not prosperous themselves, provide the raw material for most of the West's industries. A lot of critical resources are often only found in these countries. Not to mention, they have often aligned against USA gang in most cases, at the UN, while also aligning with China. Look at how many countries have signed public declarations stating that the Uighur camps in China don't exist/don't repress Uighurs.
On the second point, the article only states that China is cutting Ukrainian drone supplies to supply Russia instead - that's exactly the danger the West should be worried about. China also supplies an increasing number of armaments to partner countries such as Saudi Arabia, Iran, Nigeria and Egypt. Needless to say, China is currently the world's largest exporter of military UAVs.
The US does not have 128 foreign military bases. It has ~50 nominal bases [1]. Most of them are just the US sharing an airfield with a friendly country; it's a refueling stop that would not be hard for China to replicate.
The US does have several large overseas bases but 90% of this list is are indefensible logistics hubs and not a meaningful projection of force.
Most of these bases are co-located with NATO or other allies for good reason, the US doesn't have to do everything itself wrt air defense, locating an airlift wing with a fighter wing.
But then it's a lower bar than people imagine, for China to buy similar friendship.
I think you have the right idea. China has yet to truly flex its muscle. They prefer to quietly grow stronger. Their response to Covid with the largely successful zero covid strategy gives a clue about the power of its government. Silly, you can’t become the champion without stepping into the ring.
China creates a superbug via GOF research. Accidently releases it from the lab. Shuts down its own economy. Puts the majority of it's citizens on house arrest, and that is "largely successful"? Please send me the AliExpress link to whatever it is you are smoking, it must be some good shit.
I think the real lesson here is that if you enough government power, there is no need to be competent. The feedback loop is destroyed so you can just do whatever random stupid thing you want until your country collapses like the USSR.
China has 900M people making less than $400/month, and 600M people making less than $100/month. relative GDP is a joke, go to China to see what most of them are eating (hint: it's unsafe food filled with chemicals, or its mostly carbs) and where these people are living (hint: it's shoddy constructed condos or run down farm houses)
Not sure why you have something against the flyover states. I'm sure there's more shoddily constructed condos in Florida/California/New York per capita than there is in the Midwest. Same goes for cheap high calorie food.
Of course, the same can probably be said about the large population centers in China too. More people concentrated in one area tends to mean more poverty in that area and all the things that come with it.
The thing is Bejing undercuts this completly by allowing local governments to perform rampant shakedown of investors and ceos through disappearances for bogus charges, even in other provinces.
Makes sense. When you restrict hardware, you have to spend all your energy on optimizing software that everyone else ignores
Imagine if they were forced to use IE7 as the only browser. The frontend frameworks would be blazing fast and we would never have bloatware like React or Angular or npm