[6f666579]

This might be fundamentally irrelevant to this thread, but it got me wondering and I would like some assistance/guidance:

Whenever I read the mailing list, or threads related to it, I don't understand 99% of the stuff. Do I need to know C _very well_? Do I need to be familiar with kernel? If so, how do I go about doing that? At the moment I'm reading "Code: The Hidden Language of Computer Hardware and Software".

[isido]

Generally, you need to know the operating systems and hardware work together. And then depending on the details, you might need to know C and assembly. In this case, you need to have good understanding on how modern processors do performance optimizations.

There probably isn't any good single resource for learning all this, but perhaps getting some good textbook on computer hardware architecture might be good next step after reading the "Code". Unfortunately I don't have any immediate suggestions on the current crop - I remember learning from William Stallings' books some 15-20 years back - not sure if they are the best choice nowadays.

[ctw]

The course "Computer Systems Architecture" at Queen's University uses "Computer Organization and Design: The Hardware/Software Interface" by Patterson and Hennessy. It's a 400-level course in the computer and electrical engineering department. I can recommend it. Most chapters in it have a "Real Stuff" section where they look at a real world CPU and compare it with the contents of the book to see how the theory actually ends up in practice.

[SAI_Peregrinus]

I'd also recommend the text Structured Computer Organization by Tannenbaum[1].

[1] https://www.pearson.com/us/higher-education/program/Tanenbau...

[6f666579]

Thanks. A simple search returned that it is still being used by some universities. I should take a closer look after I'm done reading "Code".

[Symmetry]

At this level you need to start understanding details of the hardware and how actual execution differs from the model presented to the programmer. Here's the MIT course Computer System Architecture which gives an introduction to what you need to know.

https://ocw.mit.edu/courses/electrical-engineering-and-compu...

After that, I don't know, read some deep dives into a modern architecture and look up any terms you don't understand.

https://www.realworldtech.com/haswell-cpu/

[cesarb]

When you're working in userspace, the kernel shields you from much of the hardware complexity. When you're working in kernelspace, the whole hardware complexity is visible; it has to worry about details no userspace code has to care about. It doesn't help that the Linux kernel is highly optimized, using tricks that are uncommon in user space (like self-modifying code; the "alternatives" mechanism overwrites instructions with a variant depending on the hardware, at boot or hotplug time), or highly complex lock-free code (like the "read-copy-update" mechanism).

My personal recommendation is to read the Linux Weekly News (https://lwn.net/) every week (if you don't want to subscribe, the previous week editions are available for free). It always has great explanatory articles about the Linux kernel internals, written by Linux kernel developers. For instance, this explanatory article about the "read-copy-update" mechanism, from the author of the Linux kernel implementation of that mechanism: "What is RCU, Fundamentally?" (https://lwn.net/Articles/262464/).

[GuiA]

Well, you’ll only understand 100% of it if you’re a kernel developer (and a good one at that). What’s your target? Knowing C and fundamentals of computer architecture + OS design probably gets you to 20-30%. Having spent a bit of time around the Linux kernel itself (writing extensions, drivers, read random blog posts, etc) might get you 60-70%, and so on.

The book you’re reading is very good, your time probably isn’t wasted reading it.

[6f666579]

Thanks! My object is actually merely to understand it myself. I'll stick to reading the book, and then finding a book on OS design etc. as the other commentators suggested.

[rayiner]

A lot of the discussion where really isn’t about C. It’s about the mechanics of how the CPU works, particularly the functionality it exposes to the kernel. I recommend reading the Intel Architecture Reference Manual, Volume III.

[tarsinge]

Having a very good grasp on C (things like pointer arithmetic, bitwise operations, ...) is definitely a prerequisite

[buzzier]

1. Basic Computer Science: You need to be intimately familiar with hexadecimal and binary notation as well as boolean logic and similar fundamental knowledge, like data structures, their construction and manipulation, searching and sorting algorithms, abstract programming concepts, etc. etc...

2. Language and Vocabulary: You need to be able to read and write (technical) English at a competent level. Virtually all technical documentation is in English, and most of the resources you could find on the web (like this Wiki and the forum) are, also. Using incorrect terminology will make you look foolish and confuse the people willing to help you.

3. Language and Vocabulary, pt. 2: Most operating systems featured on this site, as well as most of the code snippets and examples, are written in C (or C++). Even if you choose to use another language (like FreeBASIC or Pascal), C is the lingua franca of programming, and you should be competent in making heads and tails of it.

4. Assembly: You should have knowledge about the low-level language Assembly. Read a book. Take a course at school. Write some user-space code to familiarize yourself with it. You will need it, even if you plan to write most of your operating system in a higher-level language.

5. Programming experience: Learning about programming with an OS project is considered a bad idea. Not only should you know the language in which you will be developing inside out, you should also be familiar with version control, debugging, etc. - in short, you should have written quite a few user-space programs in that language successfully before trying OS development.

6. Programming Practices: You should know how to write both code and user documentation, and be prepared to carefully document all aspects of your code and design, even if the project is purely for your personal use. Also, you should learn and use a suitable Code Management practices, including setting up and using an off-site repository (e.g., Savannah, GitHub, GitLab, Heroku). These two practices alone will - not may, but will - save you a great deal of trouble in the future, if you follow them.

7. UNIX experience: You will soon notice that many of the tools used in OS development are developed for Unix, and later ported over to Windows. The Linux kernel is often used as reference or example for how things can be done, and many of the hobby operating systems have some resemblance to Unix. Having experience with the Unix command line (preferably Bash or ksh) is a very important requirement. (Cygwin provides an easy-to-install Unix command line for Windows.). If you haven't, go ahead and use Linux or a BSD for a while - for Windows users, this can be accomplished quite easily by using a virtualizer (see below) to host a virtual system, without the need to re-partition your home system. For macOS users, you can simply use the Terminal - macOS is built on a Unix variant kernel (a combination of the Mach and BSD kernels), so as long as you have Xcode and an emulator or virtualizer installed, no further tools are necessary (though you will need to set up a cross-compiler); the default Terminal shell is the Bourne-Again Shell, but C shell and KornShell are available as well.

8. Toolchain: You must know the behavioral details of your compiler, assembler, linker, and make utility. You should know what do the emitted warnings and errors mean. You should have the documentation of the tools you use at hand, and refer to them before asking the community. Rest assured that any possible beginner's question about GCC, GNU as, NASM, GNU ld, Visual Studio and GRUB has been answered twice over. Search before you ask.

9. Emulators and Virtualizers: Familiarity with tools such as Bochs, VirtualBox, QEMU, or Virtual PC is key to having a reasonable turn-around in development, and provide a buffer between your real hardware and your test system. While these can be learned for the specific purpose of OS dev, you will certainly want to be aware of what they are and where to get them before beginning an OS project.

10. Executable Formats: Kernel space programming has many additional requirements unknown to application development. Being able to parse executable binaries is one of them (you do want your OS to load and execute applications, do you?) Make yourself familiar with Executable File Types, their internal structure, and how a linker generates them.

11. The Platform: You should have studied the manuals for the processor you will be programming for. They contain the information you need to design your kernel in the first place. Asking for information that can easily be obtained from these documents will only give cause to "Read The (...) Manual" responses, or simply RTFM.

12. The Concept: You should have an understanding of how existent operating systems function at the technical level (e.g. by having read some Books), what is good or bad about them, and how you want to go about your own OS. Having finished some tutorial and then asking "what now?" in one of the forums will just make you look silly.

https://wiki.osdev.org/Required_Knowledge

[gaius]

Read Tanenbaum, take it from there

[Beltiras]

To GP: Modern Operating Systems[1] is taught in almost every CS program in the world. It covers a lot of the topics needed. It might be a bit dated but has a solid foundation.

[1] https://www.goodreads.com/book/show/166195.Modern_Operating_...

[6f666579]

Thanks! I'll take a look at his book. You're talking about "MOS", right?

[gaius]

Yep, it might not be exactly how it works in the current Linux kernel but the underlying principles are solid.

[SAI_Peregrinus]

That and Structured Computer Organization. MOS is software-focused, SCO is hardware-focused. Both have some overlap, of course. Meltdown/Spectre are hardware bugs at their core, so having a more thorough understanding of hardware is helpful with them.

[ndesaulniers]

Check out /r/kernel for some links on learning more about kernel hacking.

[monocasa]

Inside the Machine by Jon Stokes will cover this kind of stuff.