Hacker News new | ask | show | jobs
by kylemaxwell 28 days ago
I remember when Linux users were practically obsessive about uptime and restarting felt like a sign of failure. This was at a time when Windows seemingly needed to restart once or twice a day, at least.

These days I like to turn my work Mac off at the end of the week just so I feel a literal sense of closure. It's not really the applications minimizing and running in the background; it's ME.

17 comments

Yes, I remember feeling pride in the stability of my systems when I saw a large uptime. I had a server that had 1000 days of uptime, once. Now when I see a large uptime, I'm terrified of what security patches the kernel may be missing!
I still remember the days of servers as pets, rather than cattle, and I was harping about server uptime. A wizened server admin piped in and said he rebooted his servers once a week. Said, if you do it any less frequently, then the odds of catching an error causing change while the person who made said change (possibly himself) is still around and can remember what they did go down precipitously. So, to avoid headaches and potential downtime when it mattered, he would just take servers out of rotation and reboot them, and make sure they came back online.
So true. We have one older, rather large machine in a data center that's been up for.. (checks uptime): 963 days. It has IPMI but at some point something stopped working and now we have to physically go to the data center to restart it. And since we use it every day we can't really afford to lose access to it.
This is kind of like making sure your backups actually work.

You need to test when servers go down, and people who use them should know and understand what happens when the are off.

Live Kernel Patching has been around for about 20 years[-1] now.

Red Hat Enterprise Linux[1] and Oracle (Enterprise Linux) Unbreakable Linux[2] both use it as a selling point.

This feature is still a bit ad hoc because, in most setups, rebooting a system isn't a huge burden and is much simpler than using boutique commands to live-patch it.

[-1] https://en.wikipedia.org/wiki/Ksplice

[0] https://www.ksplice.com/

[1] https://www.redhat.com/en/topics/linux/what-is-linux-kernel-...

[2] https://docs.oracle.com/en/learn/ol-ksplice/

Live patching exists but it tends to cap out at around a year of updates for any particular kernel version. It's not getting you anywhere near 1000 days.

    8:59PM  up 1858 days, 22:51, 1 user, load averages: 1.69, 2.21, 1.60
    dblrabbit@cookie:~ $ uname -a
    FreeBSD cookie.server 12.2-BETA1 FreeBSD 12.2-BETA1 r365618 GENERIC amd64

    9:05PM  up 1859 days, 13 mins, 1 user, load averages: 1.19, 1.32, 1.39
    dblrabbit@mookie:~ $ uname -a
    FreeBSD mookie.server 12.2-BETA1 FreeBSD 12.2-BETA1 r365618 GENERIC amd64

    9:14PM  up 245 days,  8:46, 1 user, load averages: 1.26, 0.97, 0.91
    dblrabbit@dragoness:~ $ uname -a
    FreeBSD dragoness 16.0-CURRENT
Currently serving: vm's, dns, email, mx-relay, and multiple shoutcast radio relays 24/7 and some other miscellaneous stuff. Colocation is fun, do I win?

5years; I'm 37 now, I was 32. Life seemed easier then.

I dunno how much easier life was back in May 2021... a year into the pandemonium, heh.
Easier for some, harder for others. The economy was more unfucked because the government printed tons of free money (mostly to business owners).
I had a box set up as NAT (running amazon linux) when we moved from a local datacenter to AWS in 2012. Shut it down last year. It had not been rebooted. Should have grabbed a screen capture of the uptime. Part of me wanted to leave it to reach 5000 days....
I worked in a place with a lot of Solaris servers with years-long uptime. It would be my job to patch them. Having no idea what config changes that may have happened over the last 3 years which would take effect on boot was always terrifying.
Ksplice came out of MIT in 2008, which updates your kernel while it's running. No need to reboot! Supports Ubuntu.
Thankfully there's livepatching (e.g. https://ubuntu.com/security/livepatch )
6.19 added a new Live Update Orchestrator, which allows significantly more of the system to be retained while doing a kexec / Kernel Handover like transisiton to a new kernel too. https://www.phoronix.com/news/Linux-6.19-Live-Update-LUO https://lwn.net/Articles/1033364/

Systemd added support in recent 2.61. Theres also now ways to have user stores, that survive across switches. https://www.phoronix.com/news/systemd-261

There's also `systemctl soft-reboot` which initiates a userspace-only reboot, which quickly restarts the system without going through the full hardware and kernel initialization process.
But that doesn't give you kernel updates, being userspace only?
Real men use: killall -9
I'm glad to see this. Almost 18 years ago I implemented a similar kexec device+memory preservation for a storage vendor. It was done on a Linux kernel of that day, and it had had a memory reservation and handoff protocol between the two kernels to keep some specific PCI device alive, allowing for state restoration at the application side. I'm proud of the fact that the kernel replacement was just under 1 second in execution (after init process optimization) and the whole kernel+app was less than 10 seconds.
Was hoping this would be mentioned. This version of functionality will probably be more universally accessible to all configurations as compared to the soft reboot released in 254:

https://github.com/systemd/systemd/releases/tag/v254

Soft rebooting was even more exciting for me, but I wasn't able to get it to work on my bluebuild based system (customized universal blue). I haven't tried it in a while though.

I remember installing some new computers for a small shop around y2k, and their NT4 server was acting up a bit when I was adding the new users to the domain controller.

I opened Task Manager to see if any processes was running wild. Imagine my surprise when I saw it had well over 1100 days of uptime!

I remember when I was asked to replace a core router with a more powerful model. The uptime of the Cisco router was ten years - and it was ten years after the datacenter went into service.
I do still enjoy the odd >30 day uptime on my PC. Usually only reboot when a new kernel version is cut.

I used to reboot into every kernel patch but often I leave .0 running for a very long time now. They seem stable and the kernel moves fast enough nowadays there's often another .0 right around the corner. There might be exploits but they're not a valid threat model for my little desktop.

If something smaller like Mesa updates, I can reload everything simply by logging out/back in, no need for a full reboot/LUKS unlock.

In the mid-2000s I ran a. Fleet of RedHat servers that hosted millions of domains. I had boxes in that fleet that were up for over a year. Netcraft confirmed it!

Microsoft literally bought these 6 or 7 servers to migrate to IIS so they could “beat” Apache. It took more than double the servers, but after I did the initial work it was moved to a different team and I don’t know how the uptime compared.

> This was at a time when Windows seemingly needed to restart once or twice a day, at least.

Ah, the NT days… An IP address has changed, your computer needs to be rebooted for this to take effect. You have moved your chair, your computer needs to be rebooted for this to take effect. You sneezed, your computer needs to be rebooted…

My machine was rebooted this week due to a power outage. I don't recall the last time prior to that. It generally goes weeks if not months without a reboot.
> These days I like to turn my work Mac off at the end of the week just so I feel a literal sense of closure

It's also just nice to start Monday with a fresh boot.

If nothing else, it keeps me from getting to the point of 200 tabs open that I'm totally definitely going to need again "soon"

200 tabs? The children of summer are still among us, it seems (he says, glancing at the current tab count of slightly over 1800).
Just close them. You're never going to read them. If you really think there's something you need, export the browser state to an archive file, then delete in 10 years after you've never consulted it once.

(Disclaimer: I'm aware that there may be valid reasons for this workflow, but in most cases it's just digital hoarding and the above advice is sorely needed. If you really need 1800 tabs, you know who you are and you can safely ignore me.)

Brave has a function to bookmark all open tabs. I have used that from time to time. Or just made a temp file and written all the URLs to it, in case I want to open a few/dozen of them after rebooting.
I think all browsers have that now. I use it 3-4 times a year with Safari on both macOS and iOS. That way I feel comfortable closing them all without worrying about “losing” anything.

I’ve been doing this, or something similar, for at least 15 years now. Dozens of mass-bookmark “folders”. I’ve never once looked at the bookmarks I made, not a single time. I even have old bookmark files archived here and there, from machines and browsers for which the bookmarks weren’t auto-backed-up to “cloud” storage like my Safari stuff is. As soon as I can get a local llm running that’s up to the task, I’ll probably have it build some kind of table with categorization out of all of them, then have it edit out entire categories of crap until I have something I might actually scan over to recover a few interesting tidbits. Finally make some use of them, now that tools can take enough of the pain out to make that a less-daunting prospect.

The irony is that now, in the LLM age, it wouldn't be that hard to feed a few thousand browser tabs and/or bookmarks into a vector database or whatever and then if some fuzzy memory tickles your brain about something you saw a year ago, you can just query it.

It's the same logic as keeping RAW digital photos. Lightroom is already gaining "find the keepers" AI features. Sooner or later it'll be possible to feed a bunch of burst shots into an AI that just weren't worth the trouble to manually sort through. The AI can do the drudge-work of digging up any gems in the rough...

how do you see how many tabs you have open, an extension? i have tab session manager and it shows i have 80 tabs. about 60 of those are ephemeral, and the other 20 i'd have open on a new browser anyhow (email x3, goog cal, caldav cal, nextcloud files, router, local and remote proxmox, navidrome, the documentation server, etc) everything else is superfluous. although i'd probably be a bit sad if i lost all my tabs right now; hence tab session manager.
Sideberry
I think I have found my soulmate :)

Every crash cuts deep if it doesn't resume correctly.

2616. On an iPad pro. I am not updating the os because I am pretty sure that the current behavior is a bug. The hard limit has always been 500.
… Why…?
Think of it as a memory aide combined with a to-do list.
I can't. My mind explodes just having 10 tabs open. I need a clean browser, done with whatever, close tab. Reopen if I need that site again.
The implication here is that “done with whatever” happens, rather than “start another thing” while the previous thing is still in progress.
And an indictment of the favorites/bookmarks UX
At Javasoft there were Solaris test machines that had been up for 2+ years, and we had to reboot the windows test machines multiple times a day. It felt really good to leave a large queue of work at the end of the day on the Solaris/Sparc machines, knowing it would be done the next morning.
One of my servers is used as an Internet gateway, so it hosts many network services, e.g. e-mail, DNS, NTP, DHCP etc.

That one (which runs FreeBSD) is rebooted perhaps once per year or even more seldom, when I do a kernel update or a hardware upgrade. If I would need to restart it for other reasons, e.g. memory leaks, that would be a failure of the OS.

On the other hand, with my main desktop PC (which runs Linux), frequently I leave it running some overnight job, but when that is not needed I always shut it down for the night.

I have never understood the people who like hibernation, because my computer has always been optimized to power up in some 10 to 20 seconds at most, and shaving a few extra seconds per day from that seems meaningless.

If you aren’t rebooting a Linux box 15 to 30 times per year then you aren’t applying kernel security updates. I view uptime fanatics as walking security problems and assign an appropriately low trust value for their actions on computers in general.
Not necessarily, live patching is possible.

https://docs.redhat.com/en/documentation/red_hat_enterprise_...

Restarting windows twice a day meant a productive day, back in the days of windows 98 (which by the way lived well past windows 2000 and windows xp)
I've followed the same routine each Friday for at least the past 10 years.

- Install all updates

- Save tabs off to Obsidian (or Raindrop now)

- Reboot

Feels good coming in on Monday to a fresh session.

Ahem... yeah... "were"...

I do actually reboot occasionally these days, because the world is so serious now.

As a rule, if you don't reboot your servers while you are near them watching for problems, they will reboot by themselves at 3am in the first day you get sick or are traveling.
We had a vp of engineering at a retail place who came from windows shop and wanted to restart the servers every night of the holiday season. It took some doing but we finally convinced him “this is Linux we don’t have to that!”
It was a differentiator when distro updates where sparser, and in start comparison with Windows at the time which couldn't stay up for more than a couple of days without crashing (particularly the XP era).
I remember my good old PC with Windows around 2005. It wanted to reboot all the time and got stuck in an infinite updating cycle every time I did. It was a particularly lazy computer.
> I remember when Linux users were practically obsessive about uptime and restarting felt like a sign of failure

I remember it too, like it was yesterday. Wait - it was yesterday.

I see people rebooting Linux boxes to cargo cult trying to fix all kinds of issues and I’m like - rebooting is not a solution. This is not Microsoft Windows.

and I’m like - rebooting is not a solution. This is not Microsoft Windows.

Reminds me of the old joke:

"Thank you for calling Dell support. Have you tried turning it off and on?"

I mean linux boxes sometimes needed a lot of time to boot before SSDs became available. Today I reboot the system if I don't know the particular name of some systemd service I need to restart. So just restart everything!

To be honest, I would recommend doing that from time to time even on linux boxes. Not because the system becomes unstable, just to check if all your services come up as expected. For the case that you really do need to do a reboot at some point.

So you’re not using full disk encryption? Typing the encryption password is probably slower than systemctl | grep to find what you need to restart.
Feels like a distinction between server and workstation. I'd be ashamed to see only 7 days uptime on my servers D: