Hacker News new | ask | show | jobs
by kragen 1819 days ago
Systems software research was dying 21 years ago when Rob gave that talk. (A better URL for the slides is http://doc.cat-v.org/bell_labs/utah2000/utah2000.html; the version you linked is pretty incomplete.)

Since then systems software research has been quite vital, in significant part due to Rob himself; in no particular order, relevant developments in systems software research since Rob's paper include Golang, MapReduce, HTML5 (including Web Workers, <canvas>, and WebSockets), Sawzall, Hadoop, Rust, wasm, Fuchsia (as you point out), V8, protobufs, Thrift, Docker, Xen, AWS, Azure, ZFS, btrfs, BitTorrent, Kafka, nearly all of Google's "warehouse-scale computing" stuff, memcached, OpenID, QEMU†, kvm, PyPy, SPARK, Julia and almost all the automatic differentiation stuff, Clojure, iOS, Swift, Factor, AMQP, RabbitMQ, ZeroMQ, Jupyter, the mainstream use of AJAX and Comet, QUIC and HTTP/2, Valgrind, LLVM, Kotlin, Bitcoin, Ethereum, OTR and Signal, Android, Dalvik, reproducible builds, seL4, Zig, Pony, CapnProto, Sandstorm.io, Fastly's fast-purging CDN, the Varnish cache it's based on, the fast SSDs that enabled it, TileDB as you mention (and Parquet), time-series databases like InfluxDB in general, Python 3, Racket, major new developments in ECMAScript, JSON, Z3, entity-component systems, general-purpose GPU computing, Intel ME (for better or worse, mostly worse, it's certainly relevant systems software research), OTR and Signal, Tor, Chrome, Firefox, Node.js, npm, LevelDB, record-replay for time-travel debugging, UBsan and Asan, stack canaries, epoll, io_uring, Qubes, Vulkan, CUDA, Wayland, Haskell's STM, XMPP, DTrace, the whole megillah around the shift to manycore, WPA for Wi-Fi, most of the work in making secure protocols resistant to timing and compression attacks, Git, SyncThing, ownCloud, rsync, zsync, GFS, BigTable, Cassandra, MQTT, and on and on and on. Oh yeah, and also eBPF.

Putting your filesystem on a ramdisk is a good idea but it's hardly innovative.

______

https://web.archive.org/web/20030601085257/http://fabrice.be...

2 comments

> Putting your filesystem on a ramdisk is a good idea but it's hardly innovative.

I am way late to the party, but here goes: In 2007 I started working for a company in Norway that had developed their own database, from hardware and up. It included parallell processing (a few thousand processors when I joined) and everything hosted in memory. It was fast. Boot up took a while though, since it had to load everything from disks to memory.

When I joined they were on the third iteration already, the first version went live back in 1992.

We have since retired the concept since off-the-shelf hardware caught up in terms of speed and lower cost, also it didn't scale very well.

It's even older than 02007! You could put your filesystem on a ramdisk in CP/M in 01979 if you bought a third-party utility, though this needed bank-switched RAM to be really useful in CP/M: https://en.wikipedia.org/wiki/Silicon_Disk_System

Dataram used to sell RAMdisks ("BULK CORE") for PDP-11s starting in 01976, but those were hardware devices, so maybe we shouldn't count them. See First also sold them. I'm not aware of earlier ramdisks, but at and prior to that time RAM would have been magnetic core, which changes the value proposition a bit: a core ramdisk is more like an SSD than anything else, because RAM contents weren't lost if power was lost.

And ramdisks were a standard feature of Atari DOS 2.5 by 01985: https://en.wikipedia.org/wiki/Atari_DOS

It was also common to use ramdisks in MS-DOS around that time. Bankswitching was part of the reason: starting in 01985 you could easily put several megs of bank-switched RAM on a LIM EMS board in even an original IBM PC, but most application programs couldn't use more than a meg at all (or more than 640K without difficulty), so a ramdisk was an easy way to get some use out of it.

And it was common to netboot SunOS in the 01980s; this usually involved making /tmp a ramdisk and NFS-mounting /, /usr, maybe /usr/share, and /var.

And the PalmPilot only had DRAM for its filesystem from 01996 until they added flash support in PalmOS 5.4 in 02004.

And most LiveCD systems copy the CD into RAM at boot time these days. And Linux usually boots from an "initramfs". I'm typing this on a Linux computer with "tmpfs" ramdisks (named after the SunOS facility, but not used for /tmp itself) mounted on /run, /dev/shm, /run/lock, /sys/fs/cgroup, and a directory in /run/user. Amazingly, there's a discussion about whether Fedora should default to swapping to a ramdisk now: https://rwmj.wordpress.com/2020/06/15/compressed-ram-disks/

The list goes on.

Genode, unikernels like MirageOS, TempleOS, Singularity OS / Sing#, compiler services like Roslyn and Kotlin, MILEPOST GCC, C++ 11+, Tensorflow / TPUs, GPT-3, all of the machine learning in compilers [1] and so much more. I truly think Deep Learning Compilers will be huge.

[1] https://github.com/zwang4/awesome-machine-learning-in-compil...

Oh, I didn't think to mention any of those, resulting in many significant omissions from my list. Some of them are things I didn't even know about! A few are kind of on the boundary: GPT-3 arguably isn't "systems software" (although we'll see, I guess) and TPUs are hardware. But certainly TPUs have big implications for systems software design if you're training ANNs.

The time since Pike's paper has been a golden age of systems software research, perhaps even more significant than the 01959-01980 period.