[seanp2k2]

+1 for the ERLite; it's a MIPS62r2 Cavium Octeon with 512MB RAM[1]. With the latest firmware, it's running Debian Wheezy, and I've had no trouble with adding the normal Debian repos and adding things like Privoxy to it (though I suspect these would be overwritten in a FW update). With Privoxy loaded and being used as an HTTP proxy for my local net with the EasyList rules, it doesn't break about 5%CPU with 100mbit/sec of inbound traffic and some web browsing going through it (I'm running NAT as well).

Being honest, it's a bit hacky for consumers...you'd be good to know Vyatta (what it uses under the hood) to get the most out of it, since there are still some things the web UI can't do (L2TP VPN being one, or PPTP without a Radius server for auth). However, it's a heck of a lot cheaper, smaller, and more power efficient than my previous P4 box running pfSense with Intel Pro/1000GTs, so I'm pretty happy with it.

I do think it'd be super awesome if Ubiquiti released a pfSense or m0n0wall-based EdgeRouter with the same hardware acceleration...I'd gladly pay $200 or so for that, but the ERLite is damn hard to beat for $100.

1. http://wiki.gentoo.org/wiki/MIPS/ERLite-3

[donavanm]

Check out the Intel Atom Avoton and Rangley SOCs. Nice x86 cores, ECC, crypto acceleration, VT-x, passive TDP, and 4x 1/2.5gbe or 1x 10gbe depending on the serdes. I only wish they had VT-d to get sr iov. If you really need more connectivity going the trident + Intel + cumulus white box switch rate has crazy throughput per watt.

http://en.m.wikipedia.org/wiki/Silvermont

[newman314]

Can you shed more light/context on "trident + intel + cumulus". I'm familiar with cumulus but not trident.

[donavanm]

Sure. By "trident" I really mean any merchant silicon switching platform. The Broadcom Trident ASIC/chipset really kicked this market segment off in 2011/2012ish. I mentioned it specifically as products like the Juniper QFX3500 series really opened up the door for things like fat/high radix clos networks that we're seeing in production.

From memory the Trident boxes supported 640gbs of throughput on SFP+ or QSFP ports, about 10,000 prefixes/routes, a couple thousand ACL terms, 1 or 2u, and around 200watts. They cost maybe $20,000 at launch are down to $5-10,000 now depending on volume and vendor. That's great for a TOR or agg switch if you can manage the individual devices (as opposed to a switch chassis like a nexus 7K).

The other thing those really opened up is cheap as chips edge devices. 10,000 routes isnt a lot, but it works if you have limited peers or can do summarization off device like a route reflector. These chipsets, and trident in particular, also work great with things like OpenFlow as you move that expensive route computation off device to a specialized platform.

The trident platform is basically EOL'd, everyones moved on to Trident II for the most part. Trident II is like 100,000 prefixes, 50,000 ACLs, 1 or 2u, 200 to 400 watts, 1.2TBs of forwarding, and SFP+/QSFP ports. Price is $15-25,000 depending on volume and vendor etc. Pushing 640gbs of throughput for ~$20,000 is pretty crazy. It means I could build a single 10kVa server rack that pushes a legit 1tbs of traffic to the internets for about $200,000. Totally insane to think about compared to just a few years ago.

The next big change should be moving from 10/40 serdes to 25/100 in the next year or so. The Broadcom Tomahawk should be like 3tbs in 2u and a couple hundred watts for comparable prices. If you need to convert between 10/40 and 25/100 ("gearbox") cost and complexity will go up a bit.

http://etherealmind.com/merchant-silicon-vendor-software-ris... http://whiteboxswitch.com/collections/10-gigabit-ethernet-sw...

edit: and to clarify these platforms usually use Intel CPUs to run the OS/route engines. The OS/RE/HAL, like cumulus provides, is then responsible for pushing updates down to the switching asic.

[newman314]

Thanks, that was really helpful.

Any thoughts on the just announced Annapurna purchase? I know it's not apples to apples but would be interested to hear your thoughts.

[justincormack]

I have a vague memory that FreeBSD does ship with the binary blobs for the acceleration, so pfsense might be doable. I did install FreeBSD on mine for a while, but you need some external setup to build packages as there isnt enough storage for the ports tree and no mips binary packages.