Hacker News new | ask | show | jobs
by widdakay 1651 days ago
I think this also shows how inefficient modern website hosting is. The fact that this person was able to get a raspberry pi to host the #1 website on HN powered by a small 50 watt solar panel is very cool (meaning maybe 10w average power budget), but also shouldn't be as uncommon as it is today. To put this in perspective, a modern server uses 50-100 watts idle doing nothing, and many more under load. To handle the top of HN, the developer probably would use load balancing and other tech, multiplying the power usage accordingly. Edit: fixing typos.
20 comments

I hate to be that old foggy but, aren’t websites just getting worse and bloated with JS crap?

I’ve had a couple websites I use daily for work just get flashy new interfaces which causes 1/3 to 1/2 second delays in the interface which used to not exist, previously they just had normal page load delays.

For example, SalesForce Lightning, their UI overhaul. Old UI is mainly just flat HTML with some loading on fields. New UI doesn’t have as many page loads it seems but wherever you navigate to takes far longer to load because of api calls or just baaad JS.

Slow for the user, slow for the server. It's almost like the people who push website technology are the same one selling you servers. Hate it and want to go back.

I agree with this so so much.

The problem is too much reliances on frameworks and add on libraries.

Developers will import an entire framework that for the benefit of a single feature. It's mind blowing to look at the amount of js includes for seemingly simple sites.

Stackoverflow answers that direct you to import a library or framework should be banned in most cases.

I will often have to scroll past several answers that say to import a library before finding a simple and functional answer that uses only a few lines of code down near the bottom. Which in my eyes is the real answer. I often wonder if there's a behind the scenes effort on SO to promote certain includes.

The entire ecosystem of some languages / implementations relies on this far too heavily.

We are seeing some of the consequences other than just bloated systems from this style of coding with malicious node packages.

I am guilty of this, and I feel bad for it. I am not a front end developer, but I have built a few web sites for various projects here and there. I certainly don't NEED to use a front-end framework, but I don't want have to spend a ton of time crafting CSS rules and figuing out how many divs to nest. To get something done quickly, my choices pretty much boil down to plain, unstyled, pages, or a full blown framework like Vuetify. So far, I haven't found anything in between. I would love to find a CSS library that I can just import and be able to create simple, nicely styled pages, e.g. that look Material-esque, without jQuery, node, npm, gulp, grunt, sass, and all that jazz.
do you know the matrix movie quote "but there is no spoon" – maybe the framework you look for is vanilla CSS. Write sensible markup to hold your content (almost no divs), CSS it and be good. Sounds that feasible?
> ... vanilla CSS... Sounds that feasible?

Vanilla CSS is the other end of the spectrum, but the problem is there is apparently nothing between hand-crafted CSS and a full front-end framework. People, like me, who are not good at design will choose the convenience of the latter over going through the tedium of the former, even if we don't really want to.

You won't be a sculptor if you avoid the chisel. If in rome, do as the romans do. If you want to swim, you'll get wet.

There is no design in/for the web without html+css, is there?

Edit: by removing 3rd parties from your project you remove a lot of overhead and current and future risk. But be warned: Maybe your company sells exactly that for a good margin and you ruin the business model.

What's your opinion about bootstrap? Unfashionable I get it, but doesn't it serve the purpose?
It's been a long time since I looked at Bootstrap. I don't remember it being anywhere near as easy to create a nice-looking page with it as it is with Vuetify.
How about Tailwind?
Bootstrap is unfashionable?
In the same way that Corollas, Applebee's, and Walmart brand jeans are unfashionable, yes.
Bootstrap v2 had a LOT of unnecessary features while simultaneously lacking features because of Less limitations. By v5 it was significantly slimmed down if not for removing IE support and jQuery, but also a lot of rarely-used components.
aren’t websites just getting worse and bloated with JS crap

Maybe they are, but that bloat is just some static files that are sent to the user as far as the web server is concerned. They should have no practical impact on the battery life of the server.

There are JS sites that render on the server as well, but that's not the bloat you mean.

And, heck, there's a solid argument that server-side rendering is more environmentally efficient, since the work is done in a data-center, which can (1) utilize caching to avoid re-doing work and (2) be built in an optimal location for electricity generation.
Conversely, you are losing the distributed computing gained by rendering on the client, and therefore need a bigger server to scale when needed. And HTTP caching can and should be used for API responses as well.
If we're measuring by total resource consumption regardless of location, is distributed computing beneficial? Your server can be less powerful, but the client needs to be more powerful. I'd think the primary difference is who's paying for it.

Sure, most clients may already be adequately provisioned, but only because so many websites with bloated Javascript have forced their hand...

>Maybe they are, but that bloat is just some static files that are sent to the user as far as the web server is concerned. They should have no practical impact on the battery life of the server."

...and if those static files make 100 API calls as soon as they land?

I think he is talking about badly written JS code serving the APIs and the overhead of it.

Of course it's hard to debate whether JS, Java or PHP is most inefficient in that regard.

I strongly believe that the efficiency of an API is 1% down to the language it's coded in, and 99% down to who coded it.
Yes and no; while it's still true that you can write FORTRAN in any language, there are network effects that mean the effort required to write efficient code is different per language/community/framework.
I think that’s true if you take a no dependencies approach but as soon as you tap into the ecosystem the argument is lost.
Not really. It still depends on who coded it. It's just that there's more people involved now.
Corollary: The average API is far less efficient than the languages you like to complain about their efficiency.
it's gotten out of hand imo. page load times take longer than when I was browsing the web on dialup in many cases.
Website Obesity crisis going on and on: https://idlewords.com/talks/website_obesity.htm

Was here on HN several times, sadly still the case.

"I don't care about bloat because it's inefficient. I care about it because it makes the web inaccessible.

Keeping the Web simple keeps it awesome. "

I don't know, we're currently rewriting our UI from the classic "PHP renders everything with almost zero JS" to the more modern "single page application with a crap ton of JS" and the new UI feels much faster to me. The old way was to resend and rerender everything on each click, which is problematic for complex UIs with a lot of data.
On a modest 3 year old phone running Firefox, such websites are excruciating.
What a false equivalence. You are comparing static sites of the past to dynamic sites of today. Apples to oranges.
The vast majority of sites don't need to be dynamic though.
[citation needed]

Websites have become significantly more complex in the last two decades.

Let’s look at Twitter for a real-world example. The core concept of it hasn’t changed, it still just has to display a blurb of a few hundred characters at most. Back in the day this was achieved by server-side-rendered HTML and a simple form POST. I don’t have the numbers for the page back then but I’d estimate it at 100KB - nowadays it’s a multi-megabyte-sized pile of shit that often fails at its primary purpose of displaying a block of text with a stupid “something went wrong” message or endless spinner.

The “new” Reddit is also a good example. Even ignoring all the user-hostile functionality changes, the actual experience is still slower and less reliable.

Really ?

Take for instance news sites or blogs: When I read news article, what I want is mostly (there are some great interactive infographics, but those are tiny minority) text and few images, that are static.

And it's not like that as a consumer I get anything extra. The text and images are still static, for the most part. Except that now its makes 20 separate requests to load it all up. Hell, pages are usually even less dynamic, after comments fell out of favor.

What increased is number of extra stuff, that is mostly focused on selling me stuff, tracking every metrics possible, and trying to figure out somne clickbait I would click next for extra engagement.

But that's not content, content has largely stayed the same.

I don't want my browser to be loading entire JS frameworks and trackers and whatever other crap just to read a bunch of text. That's absolutely nonsensical
For the user's benefit, or for the dev's?
When you replace one with the other and notice a substantial change in time-required-for-task, I think you can make comparisons.
We’ve recently transitioned to Salesforce for a project. It’s remarkable how laggy the interface is. Removing a line from a quote takes three clicks and four seconds. The UI also doesn’t always refresh the items in a reasonable period of time, requiring a page reload.

Reloading the page is like 20MB as well. Great when you’re tethered to your phone.

I agree. There needs to be a substantial effort in web development to shed the bloat. Clean and small reduces issues with resources, security, and maintainability. The status quo is gross.
You're not alone in feeling this way

https://handmade.network/manifesto

>and bloated with JS crap?

CSS animations too , especially the ones that use infinite.

Nothing to do with site hosting. css animations don't eat the server's CPU; nor does JS bloat (other than bandwidth).
JS bloat can have significant server overhead when data is loaded dynamically. It’s generally more efficient to have one GET request that can be heavily optimized than a lot of tiny XMLHttpRequest that need to be parsed separately. That may flip around when someone spends a long time interacting with a SPA, but there is plenty of truly terrible code in the wild.
I've built embedded web interfaces serving up static pages that were precompressed with gzip and then used XHR to fill in dynamic content. I kept it under 100K for the uncached download (zero third party scripts). Everything worked well and was reasonably lightweight as long as you avoided framework bloat. Not having to compress anything on device helps a bit on energy usage although that wasn't a concern.
> It’s generally more efficient to have one GET request that can be heavily optimized than a lot of tiny XMLHttpRequest that need to be parsed separately.

Without context, this statement is misleading at best and downright false at worst. You’re right that splitting up a single request into multiple would incur a small performance penalty, but you also generally gain other advantages like more localized caching and the ability to conditionally skip requests. In the long run, those advantages may actually make your app significantly more efficient. But without discussing details like this, it’s pointless to make wild assumptions about performance.

The context was JS bloat, so we are specifically talking about the subset of poorly written sites. When it’s possible to shoot yourself in the foot many people will do so.

That said, if you ever actually profile things you will find the overhead per request is actually quite high. There is a clear tendency to request far to little data at a time.

yes, I wanted to add that css (especially infinite)animation also eats the client energy and CPU.
I like a good clean CSS animation! They can be very short and meaningful. Maybe not for daily driver UI but somethings I like them.

Infinite scrolling could be annoying with animations though, I grant you that.

My issue is with infinite animations, constantly moving/blinking stuff. They also do not have same effect on different system configurations so you might not notice any effect on your dev machine and on users it makes the page unusable, and some are super distracting).
The new lightning makes it such a pain to do my time cards. Try to open my sub projects to see how many hours are left and have a tab open with my time card and constantly errors out. I can’t be the only person who checks hours left on sub projects when entering my time.
Genuine question from a not-a-web-developer: Is this not mitigated by minimization tools like webpack?
Webpack is a front-end build toolchain, not a minifier. Usually Terser is used inside of Webpage to minify.
It is not a Raspberry Pi, but an Olimex Olinuxino A20 Lime 2 and a 30W solar panel: https://solar.lowtechmagazine.com/about.html#hardware

But yes, web hosting, especially for small/mid traffic websites has become very cheap (in power consumption and in money), especially for static websites where CDNs can be used to serve assets and static content from edge caches. A full x86 server or PC is often total overkill and a little SBC sufficient instead.

It is a dual core CPU btw, to bring the average CPU load into perspective. A very interesting project as a prove of concept, also for others to adopt in countries with unstable electricity supply and/or in relation high electricity costs :).

Did the author use a CDN? Because that's kindof cheating: you are just having another (free) service burning the electricity for you.

I assumed the solar server serves the sites directly, because of this. Maybe I was wrong.

True, in this case it is great, somehow mandatory, that it is fully self-contained :). However, dynamic content like the current power consumption and CPU load would still need be served by the origin, or cached at the CDN with short timeouts only.

Using CDNs was more an idea/suggestion for others who take this project as an inspiration to run their own website even with small hardware, unstable electricity supply and/or expensive/limited bandwidth, where a CDN can further reduce server load and traffic. Also when speaking about efficiency of the Internet in general, using small SBCs where sufficient, a CDN usually serves assets/content much more effective, given a network where a particular edge server is usually closer to the visitor than the origin server, and hardware that is specifically designed and run for that purpose and can be assumed to be highly loaded (less wasted power consumption). So as long as one trusts a CDN, or the content is not of any security or privacy concerns, it is usually a reasonable choice to make use of it :).

There is no CDN, thankfully.
> a modern server uses 50-100 watts idle doing nothing

I'm really tired of hearing this. "Serverless because otherwise server doing nothing", "very small virtual machine because otherwise server doing nothing".

The server is not doing "nothing" it's waiting for incoming requests. It's like if you told "this cashier is doing nothing because there is no customers in the store".

When a server is loaded at capacity minus some margin, latencies are going up, which may not always be acceptable. Also, not every web workload scales linearly nor is cacheable and traffic patterns may not be that predictable and some requests may generate higher loads.

Managing capacity is way more involved that just "this server is doing nothing".

Also, many of these technologies supposedly reducing "idle time" such as "serverless" are usually incredibly wasteful where handling a single request may start a completely new environment and may pull resources across the globe.

If there are 100 servers but only one is needed to handle the user traffic, then 99% of those servers are considered to be "doing nothing" even if they are powered on and running software. At the end of the day, running that software is meaningless to the business and to customers.
I think the point was that "ready and waiting" is valuable to the end customer, even if it only makes a different later when they are doing something. It's kind-of like how firemen are valuable even when they are not getting calls, because they are available for low latency response instead of busy doing something else. The idea that this is just wasted computation is therefore somewhat disingenuous.
Oh, but it could be improved. Linux can cold boot under 300ms (easier if you control the BIOS and can tune it for speed, like coreboot can), faster if resuming from RAM. That should allow you to perform load-balancing while powering off the extra capacity (using wake-on-lan).

If load becomes too important for the SBC or close to capacity, wake the server, and perform a handover once it's up. You can either hold the packets and use the SBC as a proxy, or change your router's config to point to the newly awakened server (alternatively, just exchange IP or MAC addresses). With a bit of magic to avoid closing existing connections (I believe home ISP routers should keep NAT connections open if a port forward is changed), it would work. Obviously it's even easier with a proper load balancer.

edit: actually even a router might be able to handle low loads

There seems to be surprisingly little interest in this (closest I found was https://github.com/kubernetes/kubernetes/issues/89271 ).

So yeah, it's still wasted power and computation in my opinion. "Ready and waiting" should not take 100W per server, but be closer to 0.1W (WoL), or lower if managing state from a central node. I guess it's not worth optimizing for most people, and big cloud probably does something similar already.

In a way, it's a bit like big.LITTLE with additional latency: small, power-efficient vs big, fast, inefficient for small loads.

Modern CPUs go to lower power states super quickly and draw almost nothing. The thing is, if the server is running many VMs, there's no way it's going to a low power state, eveb if some are doing nothing (others will be). You also have 10 jet engines blowing air at the front, which probably is more than the CPU uses when both are idle.
Totally agreed that 100w is a waste at idle, but I don’t think that’s what the parent was talking about. My read was that the parent comment was responding to capacity planning tending towards reducing the number of servers (without building out low latency cold boot infra) in the name of cost/energy savings, and that resulting in higher latency request latency. Anecdotally this seems plausible, but I don’t have metrics to back it up.
Still, if you switch 100 servers with 100 owners all waiting for connections for 1 server hosting 100 sites and 99 other in a low power mode waiting for traffic, you save a lot of power and doesn't lose much.

Anyway, it would be waste even if you couldn't save it at all. "Waste" is simply a name for things we consume but don't actually use. All industries use that term.

We waste so many resources customizing each response to time and observer and it’s just nuts. Most people aren’t going to notice if a calculation is being debounced, amortizing it over hundreds of seconds or requests. Instant gratification is the most expensive thing by far. And debouncing has such a profound effect similar to load shedding for traffic bursts, it really should be front and center in the literature.

When I was young I worked on a project that was so inefficient that I was professionally embarrassed to have my name associated with it. So I moved heaven and earth to fix it. Gave myself an RSI before I learned to better automate some transformations.

Today I’m also working on another, lesser embarrassment, but I’m not working weekends and holidays on it anymore. I’m not a hero surrounded by villains, I’m an observant person drowning in a sea of apathetic faces.

The amount of hardware we have per user request should have gotten someone fired. Most of the people responsible are gone, but one is still here complecting anything that isn’t nailed down, and few others know enough to realize that the reason they don’t feel confident in the code is because someone intentionally made it that way, and you should not be looking up to those people. They are literally making you dumber.

Fwiw, the top of HN isn’t all that stress. It mostly comes down to disk I/O and efficiency of the language.

I’ve been at the top of HN for extended hours a couple of times on just a Heroku hobby dyno with no caching at all, but I had Cloudflare out front absorbing all the traffic that would have come from serving static assets.

Not to be contrary, but if your site is largely static and you're fronting it with Cloudfare, then you're essentially saying Cloudfare can handle load.

Not much revelation there, right?

Cloudflare really doesn't make much of a difference for HN. The last front page traffic I saw (~a week ago?) was still at most a handful of QPS. Any nginx instance with default configuration serving static files from any modern computer should be able to handle that (given that your link is big enough).

Now if you reach the top of a large subreddit, or have a viral tweet with a link to you, that's a different order of magnitude. HN is just not that large.

Then why do people talk about the "HN hug of death"?
If you’ve got a bunch of images and your site is running uncached with a clunky CMS behind it, it will probably strain you.

The traffic is usually about 40k uniques over 24 hours. For reference, my uncached site is running on Elixir which is often better without caching.

It’s usually not images but rather the CMS making 100 read queries and 20 write queries per page load without any object caching that brings sites down. Even a slow uplink serving big image won’t bring an nginx server down, it’s all async.
That’s a fair point for GP’s Heroku + Cloudflare deployment. The OP solar site is a better example of efficient static hosting as it is run on a lightweight server [1] and not fronted with Cloudflare. The reading at the bottom of the website indicates 2.70 W power usage at the moment and over two weeks uptime.

[1]: https://solar.lowtechmagazine.com/about.html#hardware

Mine isn’t static fwiw. Just running Elixir w/ Postgres. Elixir usually performs well enough uncached that you need to justify any caching needs.
So, specifically, why are using Cloudfare? Kind of sounds like you're saying both that you do and don't need it.
It’s free, solid DNS product, gives me a CDN on top of it and stops a lot of pesky bots. Plus moving the domain registration there keeps the renewal at the lowest possible price.

I can’t see any good reason not to use it.

I see. I think I was confused by your statement that kicked off this sub-thread:

>...but I had Cloudflare out front absorbing all the traffic that would have come from serving static assets.

I interpreted that to mean Cloudflare contributed significantly load-wise, then you indicated that your site wasn't largely static or cacheable (I think?).

Anyway, in addition to curiosity, we're also considering Cloudflare to offload from more costly AWS instances. So, just trying to suss out whether Cloudflare was or wasn't instrumental for you per your comments. Still not 100% sure, but thanks for the discussion.

I know people who host a static website on a home dsl connection with 5Mbps upload, using Cloudflare. The CDN literally does all the work.
5Mbps when 56k would be more than enough.
The key here is that it's just a very simple website with very low computational requirements.
And yet it provides the same amount of information as other websites 10 or 100 times it's weight.
> I think this also shows how inefficient modern website hosting is.

I suspect it's the opposite. Hosting a static site like this on a service designed for it is going to use less power than using dedicated hardware. Single server can host hundred to thousands of static sites. The power use per site is going to be much lower.

Hey @dang, you wouldn't happen to know anything about how Hacker News is hosted, would you? Reading this has piqued my curiosity. If time / your position permits, of course.
From what I remember from previous posts by dang: It's living on a single dedicated server with some hosting company (I think you can look up the IP to figure out which) - at least a while ago the code also was single-core, not sure if that is still the case. (in the past it used cloudflare for caching, but hasn't in a few years)
It's M5 Hosting
Last I read it was hosted out of M5 Networks in San Diego. I used to like fairly close to their data center and had considered hosting there at some point, so when I read that Hacker News hosted there it stuck with me.
IIRC it's just stored on someone's Dell Inspiron that they've got laying around in an office. Might be outdated info, but it's really nothing special if memory serves.
It's a static page... not everything is a static page in the web world.
a lot of the web would probably be better off if it were
I think so, WordPress that isnt cached to a frozen state on the backend is kind of silly in my eyes, the only exception would be comments, but you could hack around that by using Disqus or something, voila.
> you could hack around that by using Disqus

But that's not actually solving the problem, it's just offloading it. Not to mention that you're selling your community to yet another tracking company, and jacking up user page load time.

>> by Disqus or something

depends on what you want, the low end is incoming comments as emails and putting them semi-manually into an iframe on the unchanged static article page. Doing it myself at https://blog.mro.name/2019/05/wp-to-hugo-making-of/ and sacrificed commenter speed.

Others may easily be more sophisticated than above brutalist solution. But still: comments in iframes align well with static sites IMO.

Edit: even better may be to phase in comments from HN or the fediverse or whatever you care about into an iframe. Be it copied or inline and re-styled.

I’m a technical lead for a SaaS community forum product and we handle billions of page views a month. Many of them don’t put any load on our servers though because guest pages are cached with a short duration and the cached page gets served up.

Today that’s cloudflare but in the past it was varnish.

Otherwise it is very dynamic. Different users have access to different content so we generally can’t cache full pages at the edge for authenticated users.

I like this approach, is there something specific you're using to automate this at least in part? I feel like it could be made into a simple service with very limited JS to make it less "slow", unless you decide to manually approve of a comment.
I agree, but comments could also be static. Have a service handle comment submission, regenerate the page a bit later. If displaying to the user is an issue, do it client side. Most websites use a moderation queue anyway.

One could even generate a dedicated HTML page for the comments, and include it in an iframe, although inlining them is probably more performant.

Forcing people to use disqus is a fantastic way to fuck over your userbase
Just POST to a PHP script that regenerates the cached HTML including comments.
Even comments come quite rarely in most cases, so that the complete page with comments can be cached.
You don’t need disqus, you just invalidate the cache and regenerate it for only the next request, serving stale copies until the regen is complete. nginx or varnish can both do that out of the box.
Comments are far from the only exception to static page caches! There are often dynamic changes via plugins or functions.php. There are shortcodes and a number of other examples too.
we had semi-static comments you are talking about. they were called guestbooks. but we gave up on them.
I see a fair amount of stories here where the endpoint appears to be a VPS, sometimes fronted by a CDN. It's hard to say exactly how efficient that is, since configurations vary, but it's likely pretty good. Sure, there's hungry servers under there, but the multi-tenancy spreads that out.
It's not all so simple. For one, this A20 is connected to a router which is connected to the grid. The connection used is a 100 Mb fiber which - thanks to small average page size and very little JS - is more than enough. The whole thing is in the owner's home. I have a similar setup, and I wouldn't say "This is a solar-powered website, which means it sometimes goes offline" but "This website is served from someone's home, which means it sometimes goes offline."
It's mostly static files, if it was a modern SPA with APIs and such, it would probably crash having to fetch the same data for what is quite literally a static site.
I think I've mentioned this before, but nothing about SPAs require that level of bloat. My personal site (https://chadnauseam.com/) uses React and SPA-type features like preloading internal pages so they load instantly when you click a link, but almost all of it works fine with js disabled. It used to get a perfect score on lighthouse too but it doesn't anymore :(
Wouldn't it be way more efficient to run it on some (virtualized) node in a datacenter that is optimized for it?
Yes but it only matters if you ignore all the constant factors and sunk costs that exist. For example, I already have an rpi and a solar panel. My crappy google home mini wastes more power than this doing absolutely nothing. It's kind of pointless to hyper-optimize efficiency of a little server like this given all the waste around it.
embracing outages, though reducing them, is revolutionary, I guess.

Just not serving everybody all the time.

I recall someone saying that being on HN front page resulted in a peak load of a few requests per second. I.e. absolutely nothing if you're just serving static content.
I mean, that's because a lot of websites these days are built on bloat on top of bloat. A periodically generated static HTML page is pretty easy to take HN load.
> this also shows how inefficient modern website hosting is

And this is in a world without distributed, locality-aware caching.

Though topical thought, the Uk NHS site is text and blue hyperlinks. Its still collapsing tonight because the prime minister just announced booster jabs available for everyone. (Guess what I'm spending my evening doing). So you can't always win.
It'd be interesting to know what you actually are doing!

Thoughts and Prayers, etc

Oh, actually, just browsing to kill time while watching the open browser window at the side refresh on the NHS booking site hoping it will work long enough to give me a booster jab appointment. It alternates between "you are in a queue ten minutes to go" and "our site is overloaded please try later" for about two hours now. Everyone else between the ages of 18-50 in the UK is basically doing the same thing, hammering the site. Its not quite as life and death exciting as I make it sound....
Ah, I thought you were working on the NHS IT systems!
https://twitter.com/AmandaPritchard/status/14703629354489856...

> Over 110,000 people booked their COVID-19 booster vaccine before 9am this morning.

I don't know if this is a huge amount of traffic, or just an unexpectedly large bump.

Good luck getting a booking!

Yeah not like the requirements have changed at all........
i wonder how much information is really needed on wires today