[reknih]

The LaTeX criticisms of the article really resonated with me. Long compile times and a narrow "happy path" are the things where I feel LaTeX makes me less productive.

This is a pity because, otherwise, it is a great tool with its focus on document structure and output quality. I'm currently working on a LaTeX successor which seeks to address these issues, but it is really hard to make the right design compromises here -- what can be programmed? What is accessible through dedicated syntax? How does the structure drive presentation?

Computer typesetting is a rabbit hole, but a fascinating one. And I'm sure the last word on it has not been spoken yet :)

[svat]

• As a rough rule of thumb, TeX can do about 1000–3000 pages a second on today's computers.[1] This is for a (presumably typical) book that was written in plain TeX.

• So if your LaTeX document is taking orders of magnitude more than about a millisecond a page, then clearly the slowdown must be from additional (macro) code you've actually inserted into your document.

• TeX is already heavily optimized, so the best way to make the compilation faster is to not run code you don't need.

• Helping users do that would be best served IMO not by writing a new typesetting engine, but by improving the debugging and profiling so that users understand what is actually going on: what's making it slow, and what they actually need to happen on every compile.

To put it another way: users include macros and packages because they really want the corresponding functionality (and everyone wants a different 10% of what's available in the (La)TeX ecosystem). It's easy to make a system that runs fast by not doing most of the things that users actually want[2], but if you want a system that gives users what they'd get from their humongous LaTeX macro packages and yet is fast, it would be most useful to help them cut down the fluff from their document-compilation IMO.

---

[1] Details: Try it out yourself: Take the file gentle.tex, the source code to the book "A Gentle Introduction to TeX" (https://ctan.org/pkg/gentle), and time how long it takes to typeset 8 copies of the file (with the `\bye` only at the end): on my laptop, the resulting 776 pages are typeset in: 0.3s by `tex`, 0.6s by `pdftex` and `xetex`, and 0.8s by `luatex`.

[2] For that matter, plain TeX is already such a system; Knuth knew a thing or two about programming and optimization!

[pdw]

I never got deep into TeX, but I browsed the code at one time and some of what I found seemed utterly insane to me. For example, it includes an IEEE floating point implementation, based entirely on TeX string expansion [1]. I don't know if it is widely used, but I'm not surprised by slow LaTeX compiles anymore.

You say "TeX is already heavily optimized", but that's only true for the layout engine. The input language is entirely based on macros and string expansion. That's fine if you're only going to use it for a bit of text substitution. But as a programming language it's inherently slow. (To be fair, I believe Knuth expected that large extensions, such as LaTeX, would be implemented in WEB.)

[1] https://github.com/latex3/latex3/blob/main/l3kernel/l3fp-bas...

[svat]

That's precisely my point: the slowness comes not from TeX but from the LaTeX macro packages that the user has included on top. And if you want to make things faster for the user, you don't have to replace TeX (which is already plenty fast); you have to replace the macro packages or provide faster alternatives.

If you're going to create a new system with a new way of doing things (i.e. not using the existing popular LaTeX macro packages), then you can already do that on top of TeX, by just not using those packages! (Use LuaTeX and program in Lua instead of via TeX macros, or do the programming outside and generate the TeX input, or whatever.)

What I'm proposing is that the hard/worthwhile problem is to take real users' real LaTeX documents and give them ways of profiling (what inefficiently written macro packages are making this so slow, because surely it's not the typesetting) and replacing the slow parts.

[nextos]

This is a topic I have been interested in for a while. Is it viable to compose fancy large documents in plain TeX without a lot of effort replicating functionality provided by LaTeX (if your requirements stay constant)?

I am a heavy user of the memoir class, and I have always suspected moving to plain TeX would not be that hard. However, the fraction of users doing this seems pretty slim so modern TeX workflows do not seem really well documented.

[grayclhn]

I’ve tried. Coauthors and journal requirements were my limiting factors, not anything inherent to the typesetting engines…

These days (in industry) I manage to use pandoc markdown to word for everything (for similar reasons), which is even more limiting than plain TeX. You learn to write around the limitations pretty quickly. :)

[ta988]

For a dissertation you often have diagrams and bibliography this is taking a while. Even my resume takes more than 1s a build because of that.

[nerdponx]

How does LaTeX compare to XeTeX, ConTeXt, and LuaTeX in compilation speed?

[reknih]

This is not really a meaningful comparison: You have your TeX compiler (pdfTeX, XeTeX, LuaTeX, and the older e-TeX) that takes a document in some format and produces a PDF or DVI. In my tests (that did not include e-TeX) pdfTeX tends to be the fastest here, but sometimes you need modern fonts, so you have no choice but to use the other two.

The TeX compiler then loads a format like plain TeX (which the above commenter uses), LaTeX, or ConTeXt. The format defines what macros are available. LaTeX adds a package system, as does ConTeXt (modules) so you can import even more macros on-demand. These TeX formats differ in scope and thus speed, LaTeX tends to be a bit heavier but what really weighs it down are the myriad of packages it is usually used with.

Many TeX distributions will define aliases like pdflatex in your path such that you can preload pdfTeX with the LaTeX format, but they are not really separate compilers.

[nerdponx]

But XeTeX and LuaTeX are still billed as "end-user-facing" systems, alternatives to LaTeX and ConTeXt. I appreciate the distinction between the actual typesetting engine and a macro library layered on top, but as someone who has only ever used LaTeX I would definitely be interested in a thorough comparison of their capabilities, speed, user experience, etc. One can of course include plain TeX itself in such a comparison!

[jonathanstrange]

I've become productive in LaTeX once I stopped doing any typesetting in it until there was a real need for it due to publisher requirements. LaTeX looks great out of the box, I just finished a book that I had to deliver camera-ready and the publisher (not a LaTeX shop) was very impressed with the quality. It was the standard Memoir book template with almost no changes. Ironically, many documentations for special typesetting packages in LaTeX look very bad. Generally, the less you change, the better.

LaTeX really fails at "register-true" typesetting, though. You have to allow it to extend pages here or there by a line or be willing to fix many orphans and widows by hand. AFAIK, this has to do with the text flow algorithms which are paragraph-based and cannot do some global optimizations. (Correct me if I'm wrong, I'm not an expert.)

Btw, I cannot confirm the compile-time criticisms. A whole book takes just a few seconds on my machine for one run. I wonder what people are doing who get slow compile times.

[bee_rider]

There are some packages that slow LaTeX down, I think... Tikz is one I think.

My masters thesis was written on an old netbook with an Atom processor, plenty of graphics, the compile times got pretty ugly. But I did different files for each section, and set it up so the latex process would automatically kick off and run in the background after writing to the file in vim. Working within constraints like that is sort of fun, it forces you to get the slow operations off the critical path.

Currently I use a script like:

  inotifywait -e close_write,moved_to,create -m . |
  while read -r directory events filename; do
    if [ "$filename" = "$1" ]; then
      latexmk -interaction=nonstopmode -f -pdf $1 2> $1.errlog
    fi  
  done

to just re-compile the .tex whenever it changes. I'm not really a bash programmer though so I guess this will probably be ripped apart by somebody here, haha (the top couple lines were probably taken from some post on the internet somewhere).

[farhaven]

FWIW, `latexmk` has a "Watch the sources for this document and rebuild if they change" mode builtin. It gets activated if you pass it the command line flag `-pvc`.

[bee_rider]

Suggesting things on this site is such an easy way to get better solutions. Thanks!

[andrepd]

Memoir has options like \sloppy bottom. But honestly, the reason it doesn't is that it's virtually impossible to have an algorithm that gives you the best layout 100% of the time. Sometimes it's physically impossible not to have orphans or awkward spacing with the text you've given. You can never remove the human from the equation.

[jonathanstrange]

Yes, that's my point. \sloppybottom can look fine but it violates the requirement of real register-true typesetting where each typeblock has the same size on every page and the lines on a double page match exactly. Some publishers have this requirement and it's hard to work around it. This could in principle be improved by some subtle tradeoffs between line breaks in previous paragraphs, paragraph breaks, and page breaks. It's a kind of global optimization that is not possible in TeX due to principal limitations of the engine. See Section 4 of [1].

[1] https://tug.org/TUGboat/tb11-3/tb29mitt.pdf

[pja]

Simon Cozens spent some time writing a new typesetter called SILE:

  https://sile-typesetter.org/

On of the design goals was to be able to achieve exactly that kind of line matching. IIRC it can ensure that lines on the front & back of a page line up exactly too; apparently this is important for bibles?

Worth taking a look at. It recently acquired TeX style mathematics typesetting ability & has a small but active developer group.

[reknih]

I figure I should also mention that my LaTeX alternative is called Typst. We do not have much public detail yet but there is a landing page [1] to sign up for more info and beta access as soon as it becomes available.

[1]: https://typst.app/

[chaoxu]

It looks really nice! I’ve sent this to a few friends to check it out.

To give some context, I'm a professor in theoretical computer science, so I write a lot of LaTeX documents and notes.

Some observations of my work flow.

  - Writing: I'm writing the source, and occasionally look at the output. So as long as the output time is reasonable, then it is sufficient. 
  - Editing: I'm reading the output, and then edit the source. So going from the output to the part of the source I have to edit should be as smooth as possible. 
  - Typesetting is the least of my concern. I only check if there are any glaring typesetting problems right before we publish. This takes at most 1% of the total time in preparing a document. 
  - Live editing almost never happen. But I see why it might be useful to incorporate it into the work flow. (A cursor on the rending of the live editing would be very nice)

There are some choices on how to present source and the rendering.

Typst went with the 2 panel design, with one side source, one side rendering. So I found something close to WYSIWYG is better for editing. However, full WYSIWYG is hard to get right and comes with its own problem. Currently I found there are a few common things people do with respect to source/rendering.

  - WYSIWYG editors, which renders everything (word, TeXmacs, Lyx). Editing is done in the rendering. It is smooth, but takes a long time to get used to.
  - The app Typora that renders everything except the part where you are editing (which shows as the source). This can be generalized to render all except the current line, or something similar. Editing is done in the source, but feels like I'm editing in the rendering. This is extremely smooth for my editing work, and is my preferred way. 
  - The app like Compositor https://compositorapp.com/ that renders everything, but can call out the selected part of the source. 
  - The source and render are in two different panels. Editing is done in the source. So usually one can click part of the rendering, and cursor jumps to the corresponding part of the source. This introduce some friction, as the eyes have to do a jump, and also a quick context switch.

[svat]

The screencast looks good! For parallel/prior work in this sort of "live update" of the typeset document (and to learn from their experiences), you may also want to look at:

• SwiftLaTeX (https://github.com/SwiftLaTeX/SwiftLaTeX / https://www.swiftlatex.com / https://doi.org/10.1145/3209280.3209522 — the cool demo that used to be on their site seems to be gone, but see HN discussion: https://news.ycombinator.com/item?id=21710105)

• Texpad https://www.texpad.com/

• BaKoMa TeX (http://www.bakoma-tex.com/) — its eponymous author Basil K. Malyshev passed away recently, but the product and page still exists for now

• VorTeX (see Pehong Chen's PhD thesis from 1988 https://www2.eecs.berkeley.edu/Pubs/TechRpts/1988/CSD-88-436... — it actually discusses the issues of quiescence, etc).

[teakettle42]

My main concern with rented cloud software for this space is that it seems like a great way to lose access to editing your own past a academic/technical work.

[reknih]

Very valid concern. We aim for an open core model so you can always take your projects and compile them locally

[GiovanniP]

If you are working on a LaTeX successor you could be interested in TeXmacs, which is a LaTeX successor which works very nicely in many ways, except apparently selling itself well :-) You could see there how it was designed and how the author answered the questions you are asking.

[atoav]

I wrote my MA thesis using markdown (with extended syntax). Structuring the document is easy: just use one hash for top level sections and more for subsections. Footnotes are easy, you just add a footnote[^reference] somwhere and then add the footnote text on a seperate line somewhere:

  [^reference]: some text

Inline math works by adding $x= \frac{y}{z}$ or in a seperate math block by adding two $ signs before and after.

The syntax of markdown is easier there, but LaTeX is arguably much more powerful, e.g. you can load tables from csv data, generate graphs, make it deal with your bibliography, draw circuit diagrams etc. And the layouts tend to look just good.

I ended up converting markdown to Indesign IDML and using this as a source in an Adobe Indesign layout where I could do all the basic typographic settings and styling once and update it on changes.

[andrepd]

Splitting a long document in chunks and using the `draft` option while writing speed up compilation times considerably. Otherwise you're producing a finalised typeset document of 100+ pages every time you hit F5, no wonder it takes ~10s to finish ;)

[reknih]

IMO draft seems like a crutch: Because TeX has to reprocess the whole document and the sty-files of each imported package every time, you do not have a huge budget for your document content. Instead, you are given the option to sacrifice image decoding, plot drawing, and output fidelity to keep TeX humming along.

Sure, that's better than nothing but I cannot help but wonder whether there could be an architecture where you cut down on repeating work and get faster recompiles that way!

[pingiun]

If you want something LaTeX like, but with a wider happy path you should try SILE

[AB1908]

Does pandoc help at all?

[singingfish]

I had a go at pandoc when I was writing my (social science) PhD, but I gave up and went back to just doing it all in LaTeX fairly quickly.

From memory the compile times didn't worry me at all. What did worry me was making it look like I'd put far more effort into documentation presentation than I actually had. Which worked well for me, especially big shout out to the hyperref package to get my far-too-many acronyms linked back to the acronym definition table for every mention), links to every citation, and proper links inside the document to each section /page reference.

Then on top of that my hacked together proofreading tools in emacs, and a torturous 10k word chapter 2 to sideline a difficult politicical problem, and I passed first time!

[reknih]

In use cases like the Markdown to PDF pipeline described in the article, sure! Documents there are also simple enough so that compile times aren't too much of a problem.

However, many of the documents we like to set in TeX are more complex than that: bibliographies, figure placement, special typographical flourishes.... And here is where the complexity of LaTeX and its macros adds to the inherent complexity of what we are trying to accomplish (and compile times quickly ballon again).

So, sometimes it helps...?