[chrismsnz]

> Hardware like that makes me yearn for one with no wireless and a pair of Gigabit Ethernet ports to use as a firewall and server.

Unfortunately, general purpose hardware and operating system needs a fair amount of juice to route and inspect 1gbps of traffic.

Best performance per watt I've found is the Edgerouter Lite but that has dedicated routing acceleration hardware to achieve what it does with the little mips processor it has.

Closest you'll find is Soekris, ALIX or APU for a routing platform. A Beaglebone black makes a nice little box for lightweight serving on its own.

[seanp2k2]

+1 for the ERLite; it's a MIPS62r2 Cavium Octeon with 512MB RAM[1]. With the latest firmware, it's running Debian Wheezy, and I've had no trouble with adding the normal Debian repos and adding things like Privoxy to it (though I suspect these would be overwritten in a FW update). With Privoxy loaded and being used as an HTTP proxy for my local net with the EasyList rules, it doesn't break about 5%CPU with 100mbit/sec of inbound traffic and some web browsing going through it (I'm running NAT as well).

Being honest, it's a bit hacky for consumers...you'd be good to know Vyatta (what it uses under the hood) to get the most out of it, since there are still some things the web UI can't do (L2TP VPN being one, or PPTP without a Radius server for auth). However, it's a heck of a lot cheaper, smaller, and more power efficient than my previous P4 box running pfSense with Intel Pro/1000GTs, so I'm pretty happy with it.

I do think it'd be super awesome if Ubiquiti released a pfSense or m0n0wall-based EdgeRouter with the same hardware acceleration...I'd gladly pay $200 or so for that, but the ERLite is damn hard to beat for $100.

1. http://wiki.gentoo.org/wiki/MIPS/ERLite-3

[donavanm]

Check out the Intel Atom Avoton and Rangley SOCs. Nice x86 cores, ECC, crypto acceleration, VT-x, passive TDP, and 4x 1/2.5gbe or 1x 10gbe depending on the serdes. I only wish they had VT-d to get sr iov. If you really need more connectivity going the trident + Intel + cumulus white box switch rate has crazy throughput per watt.

http://en.m.wikipedia.org/wiki/Silvermont

[newman314]

Can you shed more light/context on "trident + intel + cumulus". I'm familiar with cumulus but not trident.

[donavanm]

Sure. By "trident" I really mean any merchant silicon switching platform. The Broadcom Trident ASIC/chipset really kicked this market segment off in 2011/2012ish. I mentioned it specifically as products like the Juniper QFX3500 series really opened up the door for things like fat/high radix clos networks that we're seeing in production.

From memory the Trident boxes supported 640gbs of throughput on SFP+ or QSFP ports, about 10,000 prefixes/routes, a couple thousand ACL terms, 1 or 2u, and around 200watts. They cost maybe $20,000 at launch are down to $5-10,000 now depending on volume and vendor. That's great for a TOR or agg switch if you can manage the individual devices (as opposed to a switch chassis like a nexus 7K).

The other thing those really opened up is cheap as chips edge devices. 10,000 routes isnt a lot, but it works if you have limited peers or can do summarization off device like a route reflector. These chipsets, and trident in particular, also work great with things like OpenFlow as you move that expensive route computation off device to a specialized platform.

The trident platform is basically EOL'd, everyones moved on to Trident II for the most part. Trident II is like 100,000 prefixes, 50,000 ACLs, 1 or 2u, 200 to 400 watts, 1.2TBs of forwarding, and SFP+/QSFP ports. Price is $15-25,000 depending on volume and vendor etc. Pushing 640gbs of throughput for ~$20,000 is pretty crazy. It means I could build a single 10kVa server rack that pushes a legit 1tbs of traffic to the internets for about $200,000. Totally insane to think about compared to just a few years ago.

The next big change should be moving from 10/40 serdes to 25/100 in the next year or so. The Broadcom Tomahawk should be like 3tbs in 2u and a couple hundred watts for comparable prices. If you need to convert between 10/40 and 25/100 ("gearbox") cost and complexity will go up a bit.

http://etherealmind.com/merchant-silicon-vendor-software-ris... http://whiteboxswitch.com/collections/10-gigabit-ethernet-sw...

edit: and to clarify these platforms usually use Intel CPUs to run the OS/route engines. The OS/RE/HAL, like cumulus provides, is then responsible for pushing updates down to the switching asic.

[newman314]

Thanks, that was really helpful.

Any thoughts on the just announced Annapurna purchase? I know it's not apples to apples but would be interested to hear your thoughts.

[justincormack]

I have a vague memory that FreeBSD does ship with the binary blobs for the acceleration, so pfsense might be doable. I did install FreeBSD on mine for a while, but you need some external setup to build packages as there isnt enough storage for the ports tree and no mips binary packages.

[jrockway]

There are ARM CPUs available that do network forwarding in hardware. Some even support iptables rules.

I would be very surprised if a quad-core current-gen Atom could not do that in software, though. I could route 300Mbps through OpenBSD's pf on an AMD Geode.

[wtallis]

There's a lot out there that can do NAT and some firewalling in software at large fractions of a gigabit/s for pretty cheap. But if you throw in QoS and queue management the CPU requirements get very high by the standards of MIPS and embedded x86. And unfortunately, none of the network acceleration hardware you'll find on any of those SoCs has anything like a hardware implementation of fq_codel or even RED.

The CeroWRT project has been searching for more than a year for a new generation of hardware to use as the platform for their development of better router software. There's nothing affordable that can keep up with the really fast DOCSIS connections available while doing anything intelligent on a per-packet basis.

[walterbell]

How about running the packet filter in a dual-NIC VM on a VT-d capable PC? Dell T20 has Xeon E3 for $500 with 1TB disk and 4GB RAM. Add a PCI NIC for firewall purposes and still have the rest of the PC for use to run other VMs. GPU can be passed through to another VM.

[wtallis]

Yeah, using desktop-class hardware works almost effortlessly, but it's not really a good substitute for a $120 router that gets by with passive cooling. This discussion is about whole computers that could hide inside the power supply for that server and run off its standby power rail.

If you're going to be running a server 24/7 anyways, it makes sense to equip it to also be your firewall and gateway. But that doesn't eliminate the huge gap between such a machine and off-the-shelf consumer networking equipment.

[walterbell]

It may become easier for consumers to buy a general-purpose PC once and change software as needed, rather than chasing the ever moving ceiling of low-end disposable hardware.

I've lost track of the number of cheap special-purpose appliances I've bought, which turned out to have limitations not present in a general-purpose PC. Consumer routers and NAS devices are already in this category, soon to be joined by compute sticks.

The problem is that buyers rarely know which part of the long tail they may need later. As Intel motherboards converge into a SoC and peripherals support USB3.1+, hopefully we end up with a future that looks like Google's Project Ara, i.e. small modules.

[rab_oof]

L7 SPI is expensive whereas simple routing of 1 Gbps usually demands only a single core and 64-128 MiB of RAM. Large production shops have such FE load balancer / router boxes in HA mode (ie CARP) that barely break a sweat.

[bduerst]

What about an Orange Pi? Comes with Gigabit and a dedicated ethernet chip.