[Xurinos]

The maximum size of buffers (and the largest fixnum) is doubled. On typical 32bit systems, buffers can now be up to 512MB.

This limited buffer space has long been a criticism of mine and an often-used point I make when I choose to play the Holy War game between vim and emacs. 512MB is still a weak and arbitrary limitation (vim just uses up your memory), but this is much better than it was.

[tl]

It's not arbitrary. It's simply the largest int pointer your emacs can represent. It's also a non-problem on 64-bit machines. Run the following code in your emacs buffer:

(let ((i 1024)) (while (> i 0) (setq i (* i 2))) (1- i))

The number it returns is your max file size. Mine is 1152921504606846975 bytes.

[vorador]

Can you give me a situation where you would need more space ?

[Xurinos]

- Viewing/modifying a fasta file containing human chromosome data

- Extremely large log files

Why not use less? Because it's ugly and does not allow me to edit or copy specific sections out. I like having the full power of a good text editor at my back.

I have brought this concern up before, and the excuse I heard was, "Well, you should not have to work with files that large." Seems someone in emacs-land disagrees; now you can play with files up to 512 MB on a 32-bit system! If there is a legitimate use case for, oh, 500 MB, we can imagine a use case for 2 GB. The world is moving on; we aren't playing with 64 KB systems any more, and data keeps growing.

In 2010, our text editors still have hardcoded memory limits? (I understand emacs's architectural reasoning for it -- and that is a choice.) IIRC, DOS 6.2's Edit could handle large files; it accomplished this with a partial-loading/paging trick, rather than actually loading the entire file in memory.

So yes, I would like to and have enjoyed opening files of large size. Admittedly, I have to wait a bit for the I/O to catch up! :) I also have to be conscious of my memory usage, since I am unaware of any paging mode in vim (or emacs). I expect, on average, to have the option to open a file of at least 3 GB size -- and that is without taking advantage of swap. Next year, I expect more.

I know it is not a common desire, so I do not blame anyone for imposing an arbitrary limit other than to say "nyah nyah; I am free".

[abrahamsen]

The "hardcoded memory limit" you see is Emacs stealing three bit of a 32 bit word as type tags for its dynamic type system.

It might be time for Emacs to switch to 64 bit words internally, should give plenty of bits left for buffer addresses after taking a few bits of for type tags.

The memory overhead when editing small files would be larger, but I haven't really heard people complain about Emacs' memory footprint lately.

[reneky]

> I understand emacs's architectural reasoning for it -- and that is a choice.

I think the choice they make is to spend their very limited time on more useful features. I'm sure many of the core Emacs developers would be very happy if someone would fix this limitation. It would be a very invasive change, a fair amount of work.

[jrockway]

Any change on 32-bit machines will require special CPU instructions, massive Lisp slowdowns, or massive integer math slowdowns.

Considering you can't even buy a new 32-bit machine anymore, however, I don't really see this as a major problem.

[koenigdavidmj]

>Considering you can't even buy a new 32-bit machine anymore, however

Netbooks.

[dagw]

Most netbooks that have come out in the past 6 month have a 64-bit CPU. Only the old atom N2xx series is 32bit, and they are rapidly being phased out.

[ghshephard]

I don't understand what you mean by "Viewing Large Files" - Is it the case the Emacs doesn't gracefully handle viewing greater than 512 MB Text Files? That isn't a particularly large log file (In Debug Mode, our Java APP gets there in about three-four hours), and we routinely work with files of 2 Gigabytes in size without considering them "large".

[imurray]

Having once managed to delete a file with hours of edits just before committing to version control, I opened my 60 GB partition in vim, found the text of the file and copied it out. While this is "stupid" — there will be better ways to do it — searching and recovering with vim worked immediately and easily.

Edit: it's possible I used "less", I can't remember.

[__mlm__]

Can you explain in just a bit more detail how you did this? This sounds really neat. I faced a problem like this a while ago, but luckily the file was still open in emacs (and in an emacs buffer), so I just re-wrote it back to disk.

[imurray]

This was a linux laptop. I immediately powered off and booted with a live CD. To be honest I'm not sure exactly what I did next. I might have just done "less -f /dev/sda5". I think what I actually did was make an image of the partition using dd to an external drive so I would be able to recover the file at leisure. Trying now, it seems that vim won't open a /dev block device directly, but an image is just a regular file.

[nkassis]

Hehe, cool trick, I'll have to try it out someday :)