10GbE and 100GbE on fiber is quite cheap and easy now - but 99% of consumers and people doing ordinary stuff have no capability or interest in doing fiber. You can terminate cat5e or cat6 with $25 in hand tools...
I think what's new is the prevalence now of 2.5 and 5GBaseT ethernet chips that are cheap enough companies are starting to build them into any $125+ ATX motherboard. At short lengths even old crappy cat5e has a good chance of working at 2.5 or 5.0 speeds.
Even 100GbE is hardly seen on company datacenters. Yes, it's cheaper than before, but still more expensive than 10G, and that's extra cost multiplied by all the devices that need to have the improved hardware to take advantage. Plus, most servers won't saturate a 10G link without tweaks on the setup. For 100G it's even worse, I think it will take a long time to see them on datacenters outside of core links or for companies with heavy bandwidth use (storage, video).
I think the common knowledge that most servers can't saturate a 10Gb Ethernet link is no longer true. In my experience even saturating 25Gb links is rather easy to do when using 9000 byte MTU on mid-tier server hardware.
100Gb links do take some thought and work to saturate, but that's improving at a good rate lately so I expect it'll become more common rather soon.
The main downside to 25Gb and 100Gb links still seems to be hardware pricing. At these speeds, PCIe network adapters and switches get rather expensive rather quick and will make you really evaluate if your situation really demands those speeds. 10Gb SFP+ and copper network adapters and switches are quite inexpensive now in 2022.
> In my experience even saturating 25Gb links is rather easy to do when using 9000 byte MTU on mid-tier server hardware.
But that's tweaking the setup already, it requires changes, testing and verification, and can cause problems in downstream equipment. And for a lot of applications, a 9K MTU will not be enough to saturate the link because they'll need NUMA awareness, or the NIC queues will need tweaking to avoid imbalances, or the application is not ready to send at that speed...
I'm not saying it can't be done, of course it can. But it isn't "plug a bigger card and it'll go faster".
> Plus, most servers won't saturate a 10G link without tweaks on the setup.
That doesn't seem right. When I got my first 10G server, it was running dual Xeon E5-2690 (either v1 or v2), and I don't recall needing to tweak much of anything. That was mostly a single large file downloaded over http, so not super hard to tweak anyway, but server chips are a lot better now than sandy/ivy bridge. It could only get 9gbps out with https, but the 2690v4 could do 2x10G with https because aes acceleration.
> That was mostly a single large file downloaded over http, so not super hard to tweak anyway
Well, my point is that most servers don't just download single large files over HTTP. Even if you only look at storage servers, going into multiple files and connections you can easily find issues and have downgraded performance if you don't prepare the system for the workload.
I'm seeing 2.5 and 5 popping up all over the place. My WiFi App has 2.5 with POE. The aggregate bandwidth of the AP exceeds 1G. Spectrum cable modems have 2.5G ports now and AT&T Fiber is shipping their garbage gateway with a 5G port.
Unfortunately, I'm finding switches with 2.5 to still be overpriced.
As I understand it 2.5 and 5G modes were originally primarily aimed at WiFi APs as real-world capacities started to scale past gigabit speeds but replacing existing wiring for an entire building worth of APs to support 10G or completely redesigning the power infrastructure to support fiber would have been impractical.
Instead we run 10G signaling at half or quarter clock rates and get something that works on the majority of existing wiring.
AFAIK the IEEE was initially resisting supporting this, but enough vendors were just doing it anyways that it was better to standardize.
I wanted to upgrade portions of my home network to multigig because Comcast is giving us 1.4Gbps and I wanted to use it. At least for me, 2.5 switches were way to expensive so I ended up with used Intel 10G cards connected with DACs to a cheap 5 port Mikrotik 10G switch. One 10GBase-T RJ45 SFP+ hooks into the modem.
It's generally around 35-40Mbps upstream and between 600 and 1.2Gbps observed downstream, depending on the time of day.
I would also very much rather have a symmetric connection than what I have now. We are within range of a fiber tap for Comcast's 3Gbps symmetric ftth service but construction costs are too high right now for them to do it. I keep asking every six months regardless. There's no other game in town as far as I can tell.
Comcast Business will run a 100Mbps symmetric fiber for ~$750/mo, maybe that would be a thing to do to get fiber in the door and then switch back to residential after the contract earns out.
... points out that 5GBASE-T needs 200 MHz channel bandwidth which is past what Cat 5e is specified for. So perhaps for runs approaching 100 meters or in noisy environments, Cat 5e won't be reliable for 5GBASE-T after all.
I think what's new is the prevalence now of 2.5 and 5GBaseT ethernet chips that are cheap enough companies are starting to build them into any $125+ ATX motherboard. At short lengths even old crappy cat5e has a good chance of working at 2.5 or 5.0 speeds.