[TexanFeller]

I'm extremely happy after upgrading my network to 10gbit copper ethernet. It was much more expensive than I thought it should be, but worth it even if I only max it occasionally. Now I can easily fully saturate my 10gbit ethernet doing a first Time Machine backup or transferring files to my M.2 SSD NAS which saves me waiting rime and is satisfying to watch.

It's wild to me that 10gbit isn't the norm by now and tech people who should know better seem to think WiFi matches or even exceeds even 1gbit ethernet. My MBP connects to my WiFi7 setup(Ubiquiti E7) at a nominal 1.5-1.9gbit but Time Machine backups and file transfers are slower than plugging into 1gbit ethernet, probably in large part due to latency and retransmissions. Not to mention that ethernet works with near 100% reliability with dramatically less variation in speed and error rate.

[floathub]

It's wild to me Time Machine works on your network. Are you just doing "first backups" over and over again, or have you somehow achieved the very rare state where Time Machine can run for, say, a week at a time without falling over?

Sorry, this is snarky and off topic, but I'm nostalgic for the days when Time Machine "just worked".

[TexanFeller]

For a very long time I thought Time Machine had become flaky, and I'm sure it's partially to blame, but with my current setup I've literally never observed it corrupt a backup and have to start over.

Before I was using one of the common Synology consumer NAS boxes that are often recommended. The NAS didn't report any errors with the drives or its own hardware, but at least once a month TM would glitch on at least one of my home laptops.

My new setup is an Asus FLASHSTOR 12 Pro Gen2 FS6812X. For a year now it's been running without a single apparent TM glitch while backing up multiple personal laptops and my work laptop. Sometimes I'm plugged in and sometimes I'm backing up over WiFi, but it's always worked.

I tried various recommended settings for the Synology and nothing helped so I strongly suspect that the Synology network protocol(SMB, AFP, etc.) implementations were either buggy themselves or at least not compatible with quirks in Apple's implementations. Synology->Asus fixed all my TM problems instantly and seemingly permanantly!

[lostlogin]

Backup failed.

Again. I’m on a Synology, and your comment is interesting.

[lukasgraf]

I can't remember the exact phrasing, but are you talking about the error message that essentially says: "The Tardis is broken. Your backup has diverged into an entirely separate timeline, and I have no way of reconciling it. You may now sacrifice an entire weekend to do an initial backup again."?

I've been on a lucky streak for several years now, where I haven't gotten that one on any of my devices.

"Preparing backup..." taking an unreasonable amount of time is a regular occurrence, and some edge cases around adjusting TM backup size quotas aren't handled well. But other than that, TM has been working reasonably well for me to back up 10 TB over SMB to a Synology NAS.

My gripe is much more with Apple's abysmal support for SMB and NFS, especially after deprecating AFP. I've been back and forth between them over the years and over several OS versions, and their implementations for both are just terrible.

But over time SMB, for me, proved slightly more stable and performant, with the right tweaks in smb.conf, and authentication and permissions/ownership are easier to deal with than NFS, so I stuck with that.

I also yearn for the days where TM just worked, because somehow, the alternatives are even worse:

- Arq Backup does some things quite well, which is why I use it as part of my 3-2-1. But some of its bugs and implementation decisions just scream "hobby grade" to me.

- Kopia looks interesting, but it's not mature enough yet. Failed for me with absolutely cryptic error messages during repo init both times I tried it, with versions several months apart.

- Restic, Borg / Vorta: Not turnkey enough for me.

[TexanFeller]

> "Preparing backup..." taking an unreasonable amount of time is a regular occurrence,

TM heavily throttles disk I/O used for backing up in order to ensure that normal user activity isn't affected. That makes it appear that TM is dramatically slower than you would expect which greatly annoys me. This becomes obvious after you run this command which will make both the preparing and transferring phases go closer to the theoretical speed you'd expect:

sudo sysctl debug.lowpri_throttle_enabled=0

[lukasgraf]

> TM heavily throttles disk I/O used for backing up

That makes sense, and I usually quite like that behavior. I barely ever notice an impact when backups are running.

However, this is happening every time on one machine (Intel iMac), and semi-regularly on another one (M3 MBP), after a fresh restart, giving mds_stores some time to settle down, and the most recent backup just hours ago, with no significant changes on disk since.

In a situation like that, I would expect the "Preparing backup..." stage to just take a second to create an APFS snapshot, and maybe a minute to diff that snapshot against the remote state. But not 10+ minutes.

But thank you for the hint about that sysctl parameter! I will certainly give this a try.

[p_ing]

TM makes my M2 Air chug when it runs against a local Samsung SSD. It does no such I/O throttling.

[bombcar]

Time Machine to a network share via Samba has been pretty reliable for me - only once has it corrupted itself in the five+ years I’ve been using it.

Amusingly enough Time Machine to a local drive failed completely.

[ls612]

I've been using Time Machine for six months pointing to a network share on my TrueNAS box and it has worked fine. Sometimes a backup will fail when the Mac is taken off my home network (it doesn't play nice with Tailscale for whatever reason) but it will always work again if I tell it to retry the failed backup once I'm back on the local network.

[lostlogin]

> I'm nostalgic for the days when Time Machine "just worked".

I’ve used it for a long long time and never knew these glory days.

[kelnos]

> It's wild to me that 10gbit isn't the norm by now

I honestly just don't need it. Part of that is my ISP options top out around 1200Mbps, certainly. But I also just don't need that kind of speed inside my home. Streaming video needs at most 20Mbps or so, and I don't do much in the way of large file transfers. And when I do large file transfers, it's usually from or to the internet, and my home network is not the bottleneck there.

[Aurornis]

> It's wild to me that 10gbit isn't the norm by now

10G was too big of a step up from 1G. The expense and power required made it unattractive. Only recently have the interfaces for 10G over twisted pair become reasonably low power.

2.5G and even 5G are in a much better spot. It's where I recommend most people start as a default.

[rconti]

Yeah, 10GbE-over-copper switches are SO expensive. I bought a Ubiquiti enterprise switch for my home; 10Gb uplink and the copper ports are split between 2.5G and 1G. It's fine because only 1 or 2 clients can even talk 10G, and those are all across the house on Cat5 links anyway so they only negotiate to 2.5 even on a 10G port.

As much as I wanted to "future proof" by having a 24 port 10GbE switch... why? I'll just wait and buy one when I have a use for it.

[Aurornis]

> As much as I wanted to "future proof" by having a 24 port 10GbE switch... why? I'll just wait and buy one when I have a use for it.

As much as I enjoy looking at those wiring cabinets where every cable is cut to exactly the right length to reach a single port on the switch, this is why I prefer to leave an amount of slack in the wiring: It's good to be able to pull different wires to different switches depending on your needs.

One small high speed switch with enough ports for the couple of devices that can use it. One gigabit switch with a lot of ports to provide connectivity everywhere else.

[rubiquity]

This is what I ended up doing. A 2.5G switch for the few devices that can use it and a 1Gbit+PoE switch for all the other PoE and 1Gbit and less devices.

[lostlogin]

The new USM Beast looks interesting.

https://techspecs.ui.com/unifi/cloud-gateways/udm-beast?subc...

[rconti]

It does, actually. I dismissed it originally because I don't need most of the capacity, but considering I have no 10GbE copper ports available, and my home is being remodeled and I'm running some new ethernet to the other side of the house, I could take advantage of these ports now.

Also, I didn't realize my UDM Pro was kneecapped at 3-4Gbps IDS/IPS throughput. I think when I bought it I only had a 1 gig internet connection so it was overkill, but now I've got 10gig. It didn't even cross my mind that I had this bottleneck.

EDIT: Nevermind, 1500 american dollars??

[tehlike]

expensive but not prohibitively so? https://www.aliexpress.us/item/3256807859212052.html https://www.servethehome.com/finally-a-cheap-8-port-10gbase-...

You can also get enterprise C3850 with 10G ports for ~100$, but they will be loud and power hungry.

[justincormack]

5G barely exists and mostly is the same price as 10Gb. 2.6Gb is taking off and mostly replacing 1Gb, but try to find a 5Gb switch, there really arent any, and most appear to be 10Gb switches with 5Gb PHY in them.

[protocolture]

>It's wild to me that 10gbit isn't the norm by now and tech people who should know better seem to think WiFi matches or even exceeds even 1gbit ethernet.

2 things here. Upthread is the discussion about the old 10GBE modules that would constantly turn off due to overheating. Thats left a sour taste in a lot of peoples mouths.

I dont have anything in my home network that matters enough to have 10GBE anywhere. If I did, I would just get fibre. My wireless is fine for most purposes except some HD streaming, and plain old 1 gig works fine there.

[speleding]

Getting Time Machine to work reliably over a network is painful, even the old Apple-made Airport with built in TM stopped working twice a year.

However, I have multiple Macs where I simply have a USB-C laptop SSD attached for Time Machine and they have worked without issue for years. These laptop SSDs come in huge sizes nowadays, and you don't need an especially performant one, so they can be pretty cheap.

[ocdtrekkie]

Heh it's honestly wild to me anyone needs over a gig. My work has a one gig fiber line supporting hundreds of employees and usage generally remains below 10%.

The high expense of 10gig is, in part, because it isn't widely necessary and the people buying it are willing to pay extra.

[freetime2]

> The high expense of 10gig is, in part, because it isn't widely necessary and the people buying it are willing to pay extra.

I think the price has more to do with where you live and how the market is structured than how necessary it is. In Japan where there is competition between ISPs, I pay about $40/mo for 10Gbps.

Routers have also come down in price to where they are pretty affordable for consumers. I use a Ubiquiti Cloud Gateway Fiber [1] which has three 10Gbs ports (two SPF+, one 10GbE) for $279. A TP-Link router [2] with an upstream 10GbE port and 2.5GbE LAN ports and Wi-Fi 7 is about $140. 2.5 GbE NICs have become cheap and ubiquitous and could commonly be found on $150 mini pcs (before memory and SSD prices went crazy).

Yeah it's more than more than most people need, but I definitely appreciate having the increased speed when downloading 50GB games, uploading 200GB files to YouTube, or backing up files to the cloud. I've probably never maxed out the full 10Gbps, but exceeding 1Gbps is pretty easy in relatively common use cases.

[1] https://store.ui.com/us/en/category/cloud-gateways-compact/c...

[2] https://amzn.asia/d/03EKpC8E

[simoncion]

And if you don't need a router, you can get 10gbit ports for much cheaper than that. The Mikrotik CRS305-1G-4S gives you one copper 1gbit port, four 10gbit-capable SFP+ cages, and is ~150USD [0]. Their whole lineup of switches with SFP+ ports can seen here [1].

[0] <https://mikrotik.com/product/crs305_1g_4s_in>

[1] <https://mikrotik.com/products/group/switches?f[0]=s%3Ac&f[1]...>

[godzillabrennus]

I put 5Gbit internet into my home (fiber) to build my startup. I'm processing terabytes of data. I have over 100TB of storage in my basement. I can regularly saturate my internet connection. That said, I remember well when a 1Gbit connection provided enough bandwidth for a 500-person call center for daily workloads (back about 10 years ago).

[ocdtrekkie]

That puts you in an extreme minority, even amongst enterprise businesses. Many medium sized enterprises have storage that looks like "a couple dozen TB total" for hundreds of staff.

Having 100 TB of storage in your home basement is an even more extreme minority than that. ;)

A gigabit connection is more than enough for a 500-person call center today.

[bombcar]

Much works on 10/100 if you wanna know the truth about it - but it is really nice to hit full speeds when copying terabytes around.

[ocdtrekkie]

Agreed. We had IP phones with 100 Mbps switches in between most of our computers and the rest of the network for a long time and very few people noticed. It'd only really be when I was installing a system upgrade or something, and I'd be like "man, it'd be nice if this didn't take an extra two minutes". For normal web access, 100 Mbps and 1000 Gbps aren't really discernable, until you're downloading large files. A lot of 4K streaming videos though, you'll start to feel it quite a bit faster.

And then hilariously, once you go above a gig, the reality is most sites won't serve them to you any faster than that anyways.

[bombcar]

I found the nicest thing about fiber is I can hit over a gb/s uploading, which is often much more critical-path for whatever I’m doing than a download.

[godzillabrennus]

If that 500-person call center is a BPO, they'd have trouble with today's data-heavy workloads. I do agree, though. I'm in an extreme minority when it comes to this much bandwidth in my home.

[chromadon]

Off topic (sorry). Interested to know what your startup is?

[godzillabrennus]

I appreciate the interest, but it's not public yet.

[saltcured]

Depends a lot on your work type and your prior exposure. If you only work "locally" and upload/download rarely, you may be way less demanding of your network than if you actually do distributed work with remote storage, high-bandwidth communicating tasks, etc.

Over 20 years ago, I was used to having 1g LAN for basic workstations and laptops in an office setting and probably 10-20g uplink from the building (shared by hundreds of staff). I also used 1g at home for my very small LAN between laptop, desktop, and SAN functions. But, my home ISP links were often terrible, such as 128k ADSL or even just a tethered GPRS phone at some points.

You end up with entirely different work styles when you have these different resource constraints.

[TexanFeller]

1Gb Internet service seems low these days, much less 1Gb LAN. I have 3Gb Google Fiber service and actually get 2+ for individual downloads from some internet services like Steam. Even at 2Gb it's annoying to wait tens of minutes for 100+ GiB games to download. If I go on vacation I come home with 10s of GiBs of photos and videos on multiple devices that start syncing with cloud storage.

During the day I need to pull large data files from the work VPN so it's nice that that can happen at full speed even when Steam and movie streaming are also at full throttle. Combine that with backups and moving various files back and forth to my NAS and I'm very happy to have 10Gb local wiring.

[ocdtrekkie]

This is one of those things where I just have to express that a lot of the HN crowd is entirely divorced from the reality the rest of the world experiences. ;)

Nearly nobody has multigig anything in the home, a probably surprisingly large percentage of business networking is 1gig LAN or less. And most people would not notice the difference if they did.

I am glad it works for you, but everyone else most certainly doesn't need it. (Yet.)

Personally, I do try for mostly gigabit in my home, because I do selfhost, but I have a ~800 Mbps download service (200 Mbps upload, it's asymmetric) that was only 500 Mbps when I signed up. And to be honest most of my patch cables are CAT5e because I'm cheap. I do make sure to run CAT6 through walls though because I don't want to ever have to do it again.

Also, I used to have Astound, and I feel so much sympathy for Google Fiber customers, you have no idea what's coming. If you thought Google had a reputation for bad customer service... just wait!

[myrandomcomment]

I disagree. I pay $120 a month for 5Gbps symmetric connection. I could upgrade that to 10Gbps for 2x, but there is no reason at this point. Even the local max from the cable company is more than 1Gbps, 1.2Gbps down / ~300Mbps up for around $80. Everything is streaming now. I work from home, on video calls. My better half will be watching something the on AppleTV streaming, the kid will be doing the same. I have Backblaze running to do backups to the cloud. 3 different laptops that will run TimeMachine backups to the NAS. The AppleTVs also have the Infuse app on them to stream local video files from the NAS. The security cameras are a constant 60Mpbs 24/7/365 to the NVR. The laptops can push a gig wireless and 2.5Gbps when plugged into the Thunderbolt docks. It is not clear that I need 10Gbps everywhere, but it has its uses. The NAS is at 10G. The link from the main switch to the router is 10G. The 3 APs in the house are at 2.5G and the 2 outside are at 1G. There was a noticeable difference when after I right sized the shared links paths up from 1G. When I say noticeable it both perceived and measured. I used to work doing switch bring up and competitive testing so I have a pretty good idea how this all comes together. Given that a reasonably cheap set of APs can now handle clients at above 1G and internet speeds in some areas being above 1G, moving to at least 2.5G in places is useful and not divorced from reality. I am in tech, but I have help my not tech friends upgrade APs, et.al. for their normal everyday home use cases and they have all been quite happy with the change.

Not being divorced from reality is the only reason I have not dropped $5K on the new Dream Machine Beast that was just released and have not swapped out my Enterprise 48 PoE (1st gen.) for the newest version that has 12 10G-BaseT ports.

[jdprgm]

I have it more for the fast nas access and being able to treat nas disks as more or less the same performance as if they were directly sata in my machine. Significantly less so about the external network aspect.

[lostlogin]

It doesn’t have to be expensive.

You can hook up a few key components cheaply and 10gb pci cards are something like $20.

[nomel]

> It was much more expensive than I thought it should be

For 10G with copper, short runs will work fine on most anything (10m), with longer working ok on cat-5e cable (20 - 30m or so).

So, if you have existing wiring, especially in wall, just give it a try first, before gutting things. Most 10g ethernet adapters give you BER stats to see the error rate.

And, if there are issues, multi-gig may be an option, where it will drop down to 5 or 2.5 gbps, which is same as 10G, but with a reduced symbol table to handle the lower SNR.

[Havoc]

You can definitely get stable 1gig throughput through WiFi. Doesn’t even need WiFi 7/6E - possible on 6. Ran a WiFi bridge like that for years and - no packet loss, consistent gig through and maybe +1ms latency

The gotcha is both ends needs to be good radios. So a router to router bridge tends to work better than router to end user device. Also had near LoS which presumably helped a ton