[yjftsjthsd-h]

> Then run ./build.py. This will use the bootstrapped 30cc-compiler to compile 30cc itself. It then again uses the 30cc-compiled compiler to compile 30cc once again. The final compiler is then stored as ./30cc.

Why isn't that also done by the Makefile? The only catch I could see is that you'd need to have it build to different output names, but that seems fine for what it is?

---

Also, I'm curious - did you find yourself having to constrain your use of C in order to make sure that the compiler could compile itself? Or does it implement everything you would use naturally anyways?

(And of course, congrats/bravo; super impressive:])

[dbcurtis]

> Also, I'm curious - did you find yourself having to constrain your use of C in order to make sure that the compiler could compile itself? Or does it implement everything you would use naturally anyways?

That would be the "bootstrapping" process. Nearly a half-century ago I took a compiler lab class where we were given a working, but slightly lame, compiler, and were tasked with adding new, less lame, language features by bootstrapping. That is: 1) implement new feature without using the new feature, 2) using the compiler that results from step 1, re-implement the feature using the new feature, and compile again. 3) Repeat with more features until end of semester for best grade.

Oh, and to the OP, well done!

[abirch]

It seems that a programmer making their own compiler is like a Jedi making their own lightsaber

[williamdclt]

I guess the full bootstrapping process would start with writing a basic compiler in assembly?

[mbreese]

Assembly? Ha! In my day, we would have considered ourselves lucky to have had assembly, let alone a C compiler. No, we had to bootstrap our computers using switches and paper tape. You probably want a terminal too!

In all seriousness, the bootstrap process is fascinating to me. At some point, it did all start with switches and manually loading commands directly into memory. And over time, we’ve slowly kept this thing going and growing. Also… I’m old enough to have seen a Altair with toggle switches, but I’m not old enough to have had to toggle in a boot loader on one. :)

[tetha]

Back in university, we took the steps of:

- Breadboarding some CPU components

- Breadboarding a very, very minimal CPU. More like a simple Adding + Multiplication machine with 4 registers or so, with a few premade components.

- Eventually moving up to microcoding a simulated CPU

- Eventually writing binary code to control the microcoded CPU

- At that point I kinda cheated and wrote my own assembler because I got sick of checking so many bits.

- This project then stopped at an assembly level.

- But then we implemented our own ML-variant interpreter and later on ML-variant compiler in OCaml

- And later on we had that ML-Compiler compile itself and extended it from there.

Pick your poison where to start. Just be aware that breadboarding stuff becomes very... messy very quickly[1]. And I do include Hardware in this, because you needed simpler CPUs to design more complex CPUs.

In practice, you want to pick the best combination of a familiar language that is as high level as it can be. But back in the day, that would've been assembly, binary. In the case of C, it went from BCPL, in which a compiler for B was written, in which a compiler for New B (NB) was written, which then turned into C.

1: https://www.youtube.com/watch?v=l7rce6IQDWs&t=78s

[norir]

It depends on definition of full bootstrapping process. At some point, someone has to bootstrap in raw assembly but we don't need to do that in 2024. If I write a bootstrap compiler in c on a machine that can run c, I can write an alternate backend for any assembly language (that supports the necessary compilation features) and thus produce a compiler implemented in any assembly language without directly writing any assembly.

Taking this process to the extreme, no one could ever bootstrap anything because eventually you're bootstrapping mineral extraction.

[pabs3]

These folks say that you do need that in 2024 (commented machine code actually):

https://bootstrappable.org/ https://lwn.net/Articles/983340/

[atq2119]

The full bootstrapping process starts with entering instructions in binary (or hex or octal) to build an assembler in the first place :)

[swiftcoder]

The full bootstrapping process starts with a soldiering iron and a reel of copper wire.

[pabs3]

That is correct the path chosen by the Bootstrappable Builds project indeed:

https://bootstrappable.org/ https://lwn.net/Articles/983340/

[wqweto]

Usually it’s easier to just cross-compile for the new target. No need to start from ground zero.

[ijustlovemath]

Sounds a lot like the kernel class at UIUC! Fun stuff

[__alexander]

> Why isn't that also done by the Makefile?

My guess is that some people would rather write Python code than dig into Make’s documentation. That said, ChatGPT is excellent at creating valid make files.

[anyfoo]

Or just take a few moments to learn the basics about Makefiles.

The thing about Makefiles is that simples ones at least are really easy to write, and read. Much simpler and quicker than a cumbersome python script, that will most likely do less with much more boilerplate code (e.g. dependencies), and be much harder to read.

Of course, you may hit a point where you stretch your Makefile so much beyond its common capabilities that that no longer becomes true, but in my experience that point is pretty far away.

[unclad5968]

As someone that learned C# and python before C and C++, to this day I couldn't explain to you how make rules work. Make is so unlike build tools from other languages that it doesn't surprise me someone would rather use python to bootstrap their compiler.

[anyfoo]

It is so trivial, it takes you 10 minutes to learn.

Less even, here is an attempt:

    foo.o: foo.c
        clang -c -o foo.o foo.c

This builds a file called foo.o, if foo.c is newer than foo.o. Imagine that there is an identical bar.o as well, that builds it from bar.c.

    fooexec: foo.o bar.o
        clang -o fooexec foo.o bar.o

This links together foo.o and bar.o into a file called fooexec, if at least one of foo.o or bar.o is newer (which in turn means that any of its respective dependencies changed; e.g. change bar.c, and bar.o and then fooexec will rebuild).

Now type this into the shell:

    make fooexec

Congratulations! You have a fully built fooexec. Now change bar.c, type it again, and like magic, you have a rebuilt bar.o and fooexec. foo.o will stay put, it’s up to date!

But of course all the .c -> .o rules look the same, so instead of writing a new one for every file, you'd use simple wildcards, and now the whole, entire Makefile is just:

    %.o: %.c
        clang -c -o $@ $<

    fooexec: foo.o bar.o
        clang -o fooexec foo.o bar.o

Do $@ and $< look arcane at first glance? Yes. Is what they do simple, though? Also yes. (If your complaint is that you don't know what those "clang" lines do, then you don't know how to use the C compiler, and will have the same problem when writing a python script.)

It pains me if that is not immediately obvious. And it pains me even more that someone would want to write a boilerplate heavy, complex python program, that rebuilds everything all the time, instead of just learning this very simple concept.

[IgorPartola]

This is an excellent explanation of make! The only thing I will add is the slightly more abstract thought that make is meant to build files out of other files based on timestamps of the source files. And the target files can themselves become source files, which means that you can build dependency graphs. Lastly, it has “caching” built in: if bar.o is newer than bar.c, don’t rebuild it. If it doesn’t exist or is older, do rebuild it. So when you change one file, you don’t recompile the entire damn tree.

It also has “phony” targets that aren’t files. The canonical example of this is “clean” which basically just does something like rm -rf *.o fooexec

This is what the JavaScript tooling people have been chasing for the past 15 years and can’t seem to grasp. Someone actually did build a make-based JS build system I believe but it never got popular because it wasn’t written in JavaScript.

[fuzztester]

>Lastly, it has “caching” built in:

A nitpick, but I don't think that should really be called caching, although it kind of is, in a looser sense. It's just comparing the file timestamps of the .o and .c files, and building the .o from the .c, if the .c is newer, or if the .o doesn't exist.

[CJefferson]

Saying this is an 'incremental makefile' isn't really correct, as changes to header files aren't going to lead to rebuilding.

So, either you need to manually keep references to which .h files you include in your Makefiles up to date, or start worrying about M / MM / MG / MP, and of course you'd like those to be re-run when you change your files, and suddenly your Makefile is an awful lot less simple.

This is the main reason I stopped teaching Makefiles in intro to C courses -- students often lost entire afternoons to forgetting this stuff, and failing to realise their mistake. It really shouldn't be any person's job to keep this up to date, when the computer knows it.

[anyfoo]

It is correct to call it an incremental Makefile, it merely doesn’t name the dependencies for you. A Makefile has exactly the dependencies you specify. It’s that simple.

That is not necessarily the most useful. Especially as your project gets larger, you probably will not want to specify every header file as a manual dependency, or alternatively rebuild everything every time you change a header file. (Though for me personally, either approach often works for surprisingly long.)

Then you can do things like letting the C preprocessor create the dependencies that get included in your Makefile (yes, common C compilers can create Makefile fragments). Or do something more complicated. Or switch to another build system, if your project really outgrew, or really mismatches, make.

But at its core, Makefiles follow a simple concept, are entirely language agnostic, and even with just the simpler concepts they remain useful for a long time.

[fuhsnn]

Yes, this is quite important for C projects as it leads to subtle bugs. I also long for a way to let make detect a change of $CC (new build of the compiler itself) in the context of debugging a compiler against third party makefile projects.

[exe34]

You could have taught them `make clean` whenever there's any issues. I work with software engineers and I still have to remind them to try rm -rf ./build/* now and again, which always seems to solve the problem.

[peterfirefly]

This is an amazingly good 2-minute introduction to makefiles. It would have been better without the last two sentences (and most of your comments on the topic).

---

Something skipped over by your introduction (and most guides to makefiles) which very quickly either makes makefiles less short or much more complicated is header files.

C files often depend on one or more header files that aren't system headers (this C compiler certainly does).

They could be added to each individual rule by hand:

  codegen/codegen.o: codegen/codegen.c codegen/codegen.h linked_list.h libc.h
       $(CC) -c -o $@ $<

or they could be put into a variable:

  HEADERS = codegen/codegen.h parser/parser.h parser/goto.h linked_list.h libc.h 

  codegen/codegen.o: codegen/codegen.c $(HEADERS)
       $(CC) -c -o $@ $<

which causes too much code to be recompiled every time a header file is changed.

Or we could do something really fancy and get only the right header files added as dependencies for each compilation unit. That requires using a feature of the gcc/clang compilers (their preprocessors, really) that causes them to output a small makefile snippet that lists the exact header dependencies for a compilation unit (the '-MD' option). These snippets can then be included in the makefile. To make this look relatively clean requires some fancy macro use that is waaaay beyond what people can learn in ten minutes.

This is an extremely common use case and it is a bit of an embarrassment that GNU make doesn't handle it better.

[anyfoo]

I did mention that in other comments, including the C preprocessor helping. make is completely language agnostic, so it cannot “guess” header dependencies. In make, you get the dependencies you specify (whether manual or generated), that’s what I wanted to show.

[dzaima]

I've written a good amount of makefiles (and even one rather complex one.. before replacing it with a thing written in an actually sane language, bootstrapped by a stripped down version of the makefile), and I still mix up $@ $< $> or whatever else my brain comes up with. And this is coming from someone deep into array languages, which have entire sets of unicode glyphs.

For how often one typically writes/looks at build scripts, remembering the specifics about makefiles is quite likely to not be worth the effort.

Makefiles start becoming incredibly awful if you want to do anything outside of pure computations in the separate targets. Making the target directory requires either putting the mkdir call in each separate built thing (awfully redundant), introducing a whole another rule (and still adding it as a dependency on every rule that'll be outputting to said directory), doing some recursive make, or running it outside of rules (but putting it under a proper if statement is likely pain). That's a bunch of awful solutions to something that really really really should just be a single calls to mkdir in any sane system.

Another horrifically awful thing is if you want some reused data across rules (e.g. pkg-config results, current OS/arch). You can put them as globals, but that'll make them be calculated always, even if no build target needs them, unless you put them under an if, which is again non-trivial to get the condition correctly for. This becomes super bad if you do recursive make stuff.

More generally, makefiles start out simple, but doing anything that's not strictly simple is gonna be pain, at the very least requiring looking at docs of a thing you'll probably never use again, and debuggability is also very low (amusingly, '-p' and '--debug' break on recursive make, seemingly losing some variables; and '-n' requires copying out & running the recursive call to get to the actually important part; and afaict none of those help with figuring out why a thing didn't build).

And you're completely SOL if you wanted to do things like reorder compilations based on how long they previously took, or want a nice display of currently-compiling things.

[anyfoo]

Fair. But as said, in my experience, they remain useful and simple for a pretty long time. I usually always start out with a simple Makefile. Only sometimes do I start with or eventually switch to something else because of the projects’ needs, but for most stuff (even stuff where the project itself is complex, i.e. embedded kernel stuff where different units are pasted together and whatnot), relatively simple Makefiles do their job pretty well.

I also have a lot of phony targets. It really enjoy just being able to type:

   make restore kernel reset upload

To bring a device into a sane state, build the kernel, reset the device, and upload the built firmware.

[saghm]

> Do $@ and $< look arcane at first glance? Yes. Is what they do simple, though?

And yet, I still don't actually know what `$@` and `$<` actually mean outside of your example. I only can assume that somehow that substitutes in `foo.o` and `foo.c` because you literally put the non-magic versions right below it (and above it a few paragraphs above as well), but I'm honestly not even positive that I'd run `make foo.o` rather than `make foo` with your example, and I feel fairly confident in guessing that this would _not_ compile into a binary called `fooexec` like the target you defined above. I don't have an idea whatsoever what I'd need to put in the target to be able to substitute in something like that because the fact that the first `%` somehow becomes `$@` and the second becomes `$<` doesn't follow any obvious pattern I can use to figure out what the third symbol would be, or even if there is one.

The problem is that "simple" is not the same as "intuitive" or "discoverable". Yes, I could probably put in the effort to go and find out these things, but why would you expect someone to be motivated to do that by someone telling you that it's "trivial" and it "pains" them as they explain things in a way that doesn't actually clarify things in a way that helps me? If you actually want to try to make a strong case that this is something worth learning for people, repeatedly referring to it in ways like "very simple" and "so trivial" and "immediately obvious" is counterproductive.

As an aside, I think it's also a bit of leap to assume that they're "rebuilding everything all the time". It looks to me like the build.py script is only needed a single time to bootstrap from the original compiler to this one, and rebuilding every single C file in the new compiler rather than using the artifacts built with the original one is kind of the whole point. After that's done once, I don't see why the new compiler couldn't be used via the Makefile. If you're complaining about Python build scripts in general rather than this specific one, I don't know why you assume they can't be written to check the timestamps on the files to see if a rebuild is necessary before doing so. That's exactly what `make` does, and hey, it's a very simple concept!

[peterfirefly]

$@ is the name of the target -- the @looks a bit like an old-fashioned target for practice and competition shooting.

$< is the name of the input (the file the rule depends on). If the rule depends on more than one input, it is the first one. So you write your dependencies as xxx.c xxx.h yyy.h zzz.h with the C file first. Mnemotechnically, it is "the thing that goes into the build".

$^ is the names of all the files the rule depends on. Think of it as a large hat above all the input file names.

Many things in make are clumsy and GNU make is a behemoth that does so much more than the original make that it is really hard to learn -- but these shortcuts have really good names!

(They are GNU make extensions, btw.)

[anyfoo]

One might ask how you learned C (or python, or anything else) in the first place, if you can't be bothered to learn what the very simple $< and $@ mean, which would take you literally seconds to look up if my example wasn't actually enough already.

The rest is just... of course you can reimplement make, or a subset of it, in python, python is turing complete after all. But why? make isn't that arcane. I'm sorry it uses the weirdly looking $@ and $< for "target" and "dependency" respectively. Those two are extremely common and you have to type them in a lot of rules, so the authors chose a shorthand. One way or another your python make subset will have to bring in the same concepts they represent, too.

This is trivial in the sense that it's a part of any build system worth its salt. And I can almost guarantee you that OP for example, who wrote their own C compiler, understood it immediately.

[mturmon]

Since you’re asking, $@ is the “target” of the build, that is, what you are aiming to build (left of the colon in the rule). This is why it looks like a bullseye.

And $< is of course the thing the target depends on. It’s the input, like standard Unix notation for stdin.

Hopefully this helps?

[akira2501]

    output: dependencies
        <command(s) that use dependencies to create output>

If any "dependencies" are newer than "output" or if "output" does not exist the commands run. Otherwise they don't.

[yodsanklai]

> Make is so unlike build tools from other languages

Interesting, I thought that "make" was like multiplication tables, something you learn very early one as it's a super simple tool. Also I assumed it was part of any unix 101 class, together with bash and command line stuff. But probably it shows my age. Most people don't have any reason to learn these tools nowadays.

[unclad5968]

That is likely the case for anyone formally educated in C. If I needed to learn make I would, but I use cmake because I develop on multiple platforms.

[aleph_minus_one]

> The thing about Makefiles is that simples ones at least are really easy to write, and read. Much simpler and quicker than a cumbersome python script, that will most likely do less with much more boilerplate code (e.g. dependencies), and be much harder to read.

Whether this is true or not depends a lot on from which programming culture/background you come.

[anyfoo]

I honestly don't think so, and I think this is here is a prime example for proving that.

For example, a Makefile that does the same job as the build.py script in this project would be significantly smaller, simpler, and easier to read in several metrics that I'd reasonably call "objective" to a certain degree.

In fact, contrast the Makefile in that project: https://github.com/keyvank/30cc/blob/main/Makefile

With the build.py script: https://github.com/keyvank/30cc/blob/main/build.py

You need to know very little about Makefiles to make immediate sense of what that Makefile is doing, whereas you need to know much more about python to still not immediately see what the build.py script is doing. In fact, you will probably just "guess" that the python script is supposed to do a similar job only from its name before that.

And then the python script still does not do incremental builds at all!

Again, if it gets more complex that can change, but this is far away from that. It takes 10 or so minutes to learn enough about Makefiles to be productive with them, from scratch.

[Tor3]

A Makefile dependency:

  Left : right

Left depends on the right. I don't think the ability to learn that concept is related to programming culture/background. What follows below that is what happens.

  Left : right
         cp right Left
         echo and so on. Left is now updated.

The only tricky part of the above, and something I guess nobody has found any good reason for, is that the whitespace in front of the statements ('cp' in this case) has to be an actual tab, just spaces won't do.

When it comes to the more "advanced" concepts (wildcards etc) there are slight differences between Make versions. And to bother with that is to get into the mindset which created the (argh) Automake and Autoconf systems (to begin with), so back in the day our company simply decided that we'll use GNU Make on every single system (we supported lots of various UNIX systems) and not bother with any of that (no SYSV Make, no BSD Make or anything), because GNU Make was and is available on anything and everything. Made life very simple back then.

[pm215]

The story I have heard about why it must be a tab is that by the time the author of the original Make realised he didn't need to require a tab, he already had six users and didn't want to annoy them by breaking compatibility with the makefiles they'd written :-)

[robinsonb5]

> the (argh) Automake and Autoconf systems

The classic XKCD graph that plots life satisfaction against days-since-editing-xorg.conf could equally apply to days-since-thinking-about-autotools.

In fact one of the few times I've thought about autotools in the last decade was when a failing python-based build script inflicted similar frustration!

If I were forced to find one nice thing to say about autotools it would probably be that at least it doesn't assume an internet connection is always available, reliable and without cost.

[3836293648]

It's is absolutely objectively true. Whether you already know it or not depends on your background

[saghm]

On the other hand, I'd argue that the point where you've stretched it so far that it's more trouble than it's worth is basically impossible to see until you've gone beyond it.

[bluGill]

If make is too hard, ninja is simpler. Less powerful but in ways that rarely matter. I've written both by hand both are a little weird but not hard. Normally I use cmake these days (an aweful language but it works for hard problems)

[keyvank]

i think it just supports smth like 70% of C. I didn't bother supporting floating-point operators, or arrays (I mean, malloc and pointers are fine) or etc. I just wanted smth small and self-hosting!