If you are looking to learn operating system concepts using RPi there is a new textbook by Prof. Vanderbauwhede, under ARM textbook initiative [1].
I think for educational purposes it will be very interesting and useful if someone develop from scratch a simple System V UNIX for RPi based on Bach's description in his seminal book on "The Design of the UNIX Operating System" [2]. The book also covers multi processors topic as supported by modern platforms such as RPi.
Is the ebook of the textbook a regular PDF/epub/mobi or is it only viewable in their viewer apps? The page doesn't say, and I don't want to spend money on it and then realize it's not a real download.
Might be a bit controversial but I found writing drivers for Linux on my raspberry pi helped me understand kernels, virtual memory etc. I used Linux Device Drivers (ldd3) which is free online. The book was written for the 2.6 kernel but someone on GitHub updated the code for the latest kernels.
I've never written a driver myself, because all my hardware is supported. But I've certainly added new PCI IDs to existing drivers, and done that kind of low-level learning.
I expect if I found a random piece of hardware that was close to an existing one that adding the driver, or updating things so that it worked on the new hardware would be possible. It's just not a situation I've come across.
What I found useful/educational was writing a simple linux security module.
1. LSMs are essentially isolated.
2. They don't care about hardware.
3. You can do lots of things with them. Even horrid evils things.
For example I put together the following module in which the kernel calls back to userspace every single time you execute a binary, to decide whether it should be permitted or not:
Not OP but my perspective is that much of software development, engineering, and practices become cult like, often without theoretical or empirical basis.
I'm rarely surprised anymore when I see criticism of one approach over another. Someone may think "the way" to learn Linux kernel development isn't what OP suggested and there's a cult of people who agree who will adamantly concur to a fault.
This doesn’t seem controversial. It’s how the operating systems class[1] was structured at University of Illinois, though the target was a VM instead of physical hardware. It was easier to reset things when your new scheduler module instantly crashes the OS.
Raspbian was renamed to "Raspberry Pi OS" on the 27th of May, 2020. That's so recent you can assume that there'll be a whole bunch of naming conflicts and/or confusion.
In terms of teaching fundamentals of operating system theory and systems programming, is there such a thing as an operating system kernel written in Python or some language which is meant to be easy to hack on for educational purposes? I wonder how useful that would be for a CS student, taking the complexity of C, C++, or Rust out of the picture and just running a kernel in a VM for educational purposes.
IMO this topic is entangled with hardware, the student needs to do a course in hardware and how CPUs work, do some simple assembly coding, after that C is a natural progression. If C is too complex for some students then system programming and OS development is not for them. For educational and getting started I think https://www.nand2tetris.org is a good source material, the complexity or real world stuff is missing there so you can focus on essentials.
An OS course taught in Python is really just a data structures and algorithms course. When you ignore the hardware aspect, topics like scheduling and virtual memory management are about selecting an algorithm with the appropriate trade-offs.
My OS course was hacking on x86 Linux in C. The C wasn't really the difficult part. Honestly, neither was the assembly, as most of was boilerplate: here's how you configure to registers for a system call, here's how you trigger the interrupt for the system call, here's how you get the return codes, etc.
I don't think the class would have been nearly as valuable if C were replaced with Python. At that point, so many of the interesting details would be hand-waved away that it wouldn't really be all that different from the other programming classes I took.
If the language is boilerplate and the algorithms/data-structures derive from the physical constraints, was the OS course you completed mostly about hardware ?
As to the inciting question, there is a Stonybrook OS course [0] designed around a java simulator that students implement modules for. Springer published M Kifer's book [1] about this OSP2 system. (The binaries are accessible via the link 'Students: Software Download'.) I'm of the impression that the course involves examining the trade-offs of each module. So the thread section spawns a bunch of request events per process for a module to deal with, whether with round robin or priority queues or whatever.
There are actually plenty of languages that can lead you down this path, including some designed for the purpose.
For real world learning, C is still what you want. It is one of the easier roads through.
However, here's a just few languages that can let you build a kernel up quickly:
+ Forth. It is well suited for the purpose. A tiny bit of assembly, and then just a bunch of definitions and you've built it all from the ground up. But if you think C is too complex for learning, then the idea of moving to a stack-based language could get in the way.
+ Lisp. The Symbolic Lisp machines might be in the past, but the various versions of Lisp are still flexible enough to let you build an OS, and lots of CS students do.
+ Lua. It is only slightly higher level than C, but with fun things like coroutines, hash tables, dynamic typing and GC. Due to its C99 nature and the desire to keep the language constrained, it runs on bare metal great. You don't have to modify Lua to run it baremetal on a Pi. [0]
C is a language which is closer to hardware than Python, so you'll find it to be the language of OS programming. In fact, Python is written in C. I think trying to do OS programming in Python would actually be more difficult than learning C.
If you are a solid programmer who already knows Python, learning C would only take 2-4 weeks. The benefit is worth the investment.
You could write a Java based virtual kernel designed to run within the JVM and manage child processes I suppose, but what is the real world utility of such a thing?
Almost everything to do with kernels is about directly writing to memory addresses and ports and these things can't be done with a managed language, at least not without a significant runtime doing all the grunt work, at which point you'll need to understand much of the runtime's internals and at that point, you may as well have started out with a small C/C++/Rust kernel, plenty of which exist.
Actually using C or C++ is the most straightforward painless way to learn kernel dev. C is not complex at all. What you need to learn first is something like what nand2tetris offers, at least the first part. Usually you will have to do some variation of nand2tetris in the first or second year of undergrad.
Depending on what, exactly, you want to hack on, building components on top of seL4 could be a way to go. There's a large collection of libraries built on top of it, that you can replace as you feel like - https://docs.sel4.systems/projects/available-user-components... - and there's a few different options for communication between them (seL4's IPC, shared memory, etc).
You could easily get a stripped-down interpreter or something running in that environment.
I want to learn about firmware development. I have been playing with rpi gpio and arduino i2c but have no idea how to start with firmware development. please throw at me references, i'm very fragile but i'll take it and learn from.
I think for educational purposes it will be very interesting and useful if someone develop from scratch a simple System V UNIX for RPi based on Bach's description in his seminal book on "The Design of the UNIX Operating System" [2]. The book also covers multi processors topic as supported by modern platforms such as RPi.
[1]https://www.arm.com/resources/education/books/operating-syst...
[2]https://github.com/suvratapte/Maurice-Bach-Notes/blob/master...