[azonenberg]

The plan is 96 ports total for my own use, split across two 48-port 1U switches. And that's something that is now well within reach: I have the power boards assembled and ready to go, I have the FPGAs and PHYs and (not populated) line card boards in hand.

(EDIT: clarification, the plan had been 96 ports total since at least 2015, since that's what I had in existing Cisco switching. The major scope creep over that time was deciding that 2x 48 port switches with a bigger FPGA would be a more power efficient, easier to build, and generally superior design to 4x 24 port)

All that's left hardware wise is to design the main logic board with the FPGA, processor, and SFPs, plus the mechanical and thermal design of the 1U chassis. I've been focusing on firmware due to the tariff situation and wasn't planning on spinning the production-ish hardware for quite a while.

I don't plan any further scope changes now that I've e.g. already put in the production line card PCB order and mostly finalized the fabric architecture. I'm just holding off on the final logic board design until I'm sure about, for example, whether I'll want external packet buffer SRAM like LATENTPINK had or if I'll be fine with the KU5P's internal memory resourecs.

As far as the probes go, yes the antikernel.net site is way out of date and I've been too busy to refresh it.

* The AKL-PT1 did a kickstarter and I shipped probes to backers, but I stopped selling them because the fixed tip/ground spacing was too awkward and made it difficult to use.

* The AKL-PT2 (first gen solder in probe) was for sale for a while but extremely fragile, only 4 GHz bandwidth, and also had fixed signal/ground spacing.

* A few in-between designs were total flops and never reached the point of shipping a single unit

* The AKL-PT5 (second gen solder in probe) is >16 GHz bandwidth and has a flexible resistor based solder-in tip and a much nicer form factor. R&D is done but I ran into supply chain problems with the specialized resistor it needs (9 month lead time, poor yield, etc). I think I've mostly worked around those issues and I have a PVT run of a hundred units in the pipeline now. Fingers crossed they'll be hitting Digikey in the next 6-12 months.

[jmcguckin]

If i wanted to do a motherboard that held 16 or 24 raspberry-pi compute modules, could i connect them together using the onboard gig ethernet without using any magnetics? Just by using the gige output from the module going to a commercial switch chip.

[fxtentacle]

In general, that should work. Here's an example board with 4x CM4 modules and a RTL8367N switching chip: https://github.com/will127534/Miniature-CM4-Cluster

[azonenberg]

You're going to at least want to AC couple through some capacitors to avoid problems with common mode / RX bias issues between the different PHYs. But if it's on-mobo you don't need to have as much of a fault voltage rating compared to standard Ethernet which IIRC specs 1 kV isolation on the transformers.

[Bluecobra]

It might help to take a look at Arista/Metamako 10G L1 switches for some inspiration. These are 48x ports and use the FPGA for specific network applications (including switching). You might be able to find some of the older models on the cheap on eBay.

I’ve opened up a few and the board itself seems fairly simplistic. I do recall a giant copper heatsink though.

[azonenberg]

The board-level architecture is going to be super simple as big FPGA designs go:

* XCKU5P in the middle

* 12x GTYs routed to 4x Samtec ARF6 connector for the line cards

* 2x GTYs routed to 2x SFP28 uplinks

* RGMII to back panel management PHY

* Parallel SRAM bus to STM32H735 management processor

* A bunch of Murata MYMGK modules for power conversion off the 12V rail

* STM32L431 in QFN48 or more likely BGA100 depending on IO requirements as a PMIC and to manage reset sequencing etc

This will be fully FPGA based, no separate switch ASIC, and I want to do all of the hardware design. I'm not sure there is much I can learn from somebody else's FPGA switch design at the board level - it's basically just going to be a bunch of transceivers hooked up to SFPs and some power distribution. All the magic happens inside the FPGA.