[alok-g]

I would love to see a book that talks about going all the way to generate machine code, i.e., not stopping at generation of assembly.

Alternatively, I would like to learn about not just how to make a compiler, but also simultaneously a debugger, hot-reloading, etc.

[synack]

The debugger book is coming soon. https://nostarch.com/building-a-debugger

[alok-g]

Awesome! Thanks.

[hdbxbxndj]

Writing an simple assembler is trivial. Even macro assemblers are very easy.

However, it's also boring.

Nevertheless the contents of the book cover all the techniques required to write an assembler, if you'd really like to

[alok-g]

I understand that assembly file can be parsed in the same way. However, I want to learn about the machine instructions to the level of bits, and likewise the layouts of binary files. Unless I am able to go all the way to machine code loaded in memory, I would not know where in memory to add a breakpoint instruction when a developer wants the same on a line of code.

If there is some library that can help create machine code from assembly instructions on a line by line basis (at least as opposed to invoking a separate program that generates the entire binary collectively from the assembly code), that could also work.

In my case, I already know enough of the lexer, parser, etc., parts. What's missing is going all the way to making a debugger, profiler, etc.

[fuhsnn]

>If there is some library that can help create machine code from assembly instructions on a line by line basis

That's what JIT libraries do, for example asmjit: https://github.com/asmjit/asmjit/blob/master/test/asmjit_tes...

[dymk]

asmjit is great; I used it for building a primitive (but quite fast) query engine for in-memory graphs. It made it simple to go from query AST to native instructions, most of which was "call this other compiled function".

[alok-g]

Looks cool! Thanks.

[rramadass]

Checkout GNU Binutils and their usage of libbfd and libopcodes.

[jsnnsjxj]

Building a debugger and profiler is quite an advanced task compared to building an assembler though ^^

Also much of that work is heavily dependent on the used operating system.

Nevertheless, I'm wishing you all the best on your journey!

[peterfirefly]

Really? You can get quite far just with ptrace() on Linux... and maybe something like system("nm xxxx > file") for the symbols.

[peterfirefly]

There can be weird interactions unless there are strong enough limits on what kind of expressions the assembler allows. Especially if it supports conditional assembly and loops in the macros. One ugly way around it -- which causes its own headaches -- is to introduce pass-sensitive conditional assembly (as in "if in pass 1/2/...").

It's also "fun" if some instructions come in different sizes... and you may need stronger restrictions on allowed expressions in that case.