[s-km]

does it do anything to change/fix Emacs shitting itself on large buffers?

last time i gave it a try opening a large file would completely kill performance, and iirc in particular really long lines (ex. 1000+ chars) would make the thing chug even if the actual file wasn't super big or anything.

[BeetleB]

Large buffers are fine - long lines are not.

If you don't have long lines but you are experiencing slowdowns on large buffers, your mode is doing something. As an example, large XML files bring my Emacs to a crawl, but if I switch to a text mode it's fine.

For long lines, I believe the latest Emacs has a fix: https://www.reddit.com/r/emacs/comments/ccoksw/solong_mitiga...

[rml]

I've had good luck on big (multi-GB) SQL dumps, which often have very long lines, with

https://github.com/m00natic/vlfi

[yepguy]

I think I remember Emacs 27 will include some improvements in that regard, even without native code compilation.

[aasasd]

> in particular really long lines (ex. 1000+ chars) would make the thing chug

My uninformed guess is, this is bottlenecked by an inefficient algorithm that accesses the memory too much, i.e. it's nonlinear with respect to the number of characters. I doubt that tweaking the overall speed would fix it.

[morelisp]

My experience is it's generally either

- inefficient font-lock regexes, which are rarely benchmarked / optimized since most files don't have long lines and because font-lock behavior is quite complicated to begin with;

- inefficient thing-at-point implementations / use, e.g. an O(n*n) thing-at-point called at various points by an O(n) function;

- modes using font-lock regexps where they really just need "fixed" styled text, but other Emacs architecture makes it difficult, e.g. compilation-mode and derivatives.

[pulisse]

Emacs dev here. The primary cause is something else.

Emacs's inefficiency in handling files with long lines is due to two factors: (a) The primitive unit of work for the display engine (the code that determines how to combine text, font metrics, syntax highlighting, inline image display, etc.) is a line (a newline-delimited span of characters). (b) The redisplay routine is called very frequently—not just when the screen is repainted, but pretty much any time the screen location of a buffer element need to be calculated, and so, e.g., during navigation. So Emacs is constantly, under the hood, going back to the previous newline and re-calculating how the buffer contents from that point forward should be rendered.

[morelisp]

I’m not an Emacs dev so I’ll concede your expertise here - but if it is fundamental to the core navigation and redisplay, why does “stepping down” modes help so much? Fundamental and text mode certainly aren’t great with long lines, but are usually orders of magnitude better (eg a few seconds per command instead of 10s of seconds).

(I have not tried so-long-mode and am mostly on Emacs 26 with some 25.)

[pulisse]

What you're observing is caused by what I'm describing. The features of these other modes are expensive because they do things that implicitly invoke redisplay.

Consider an ostensibly simple operation like determining the column in which a particular character appears. The answer to that isn't the number of characters since the last occurrence of `\n`, because whatever modes are active can inject arbitrary spacing, or display particular strings as something shorter or longer (a trivial example is the mode that causes `lambda` to be displayed as `λ`), or cause some characters to be displayed in a non-roman font where a single glyph is more than one column wide, and so on. To determine the character's column accurately you need to take into account everything that could affect how you'd paint the screen at that position, i.e., run the whole redisplay loop for some portion of a buffer. The richer the set of active modes, the more expensive it is to do that and the more often it is done.

[foo_barington4]

Thanks for your work on Emacs :)

[na85]

I routinely edit C# files in the 5k to 15k LOC range, and emacs works well for me. Occasionally the "smooth scrolling" feature stutters on large files but otherwise it's perfectly usable.

[xxpor]

Are you opening it in fundamental mode? Without actually doing profiling, I'd guess it's syntax highlighting that's the real issue there. I've opened 100+ MB files without an issue, but that was on a box with an absolute ton of RAM.

[throwaway_pdp09]

Nah. IME it really is just line length.