Hacker News new | ask | show | jobs
by Xurinos 5892 days ago
- Viewing/modifying a fasta file containing human chromosome data

- Extremely large log files

Why not use less? Because it's ugly and does not allow me to edit or copy specific sections out. I like having the full power of a good text editor at my back.

I have brought this concern up before, and the excuse I heard was, "Well, you should not have to work with files that large." Seems someone in emacs-land disagrees; now you can play with files up to 512 MB on a 32-bit system! If there is a legitimate use case for, oh, 500 MB, we can imagine a use case for 2 GB. The world is moving on; we aren't playing with 64 KB systems any more, and data keeps growing.

In 2010, our text editors still have hardcoded memory limits? (I understand emacs's architectural reasoning for it -- and that is a choice.) IIRC, DOS 6.2's Edit could handle large files; it accomplished this with a partial-loading/paging trick, rather than actually loading the entire file in memory.

So yes, I would like to and have enjoyed opening files of large size. Admittedly, I have to wait a bit for the I/O to catch up! :) I also have to be conscious of my memory usage, since I am unaware of any paging mode in vim (or emacs). I expect, on average, to have the option to open a file of at least 3 GB size -- and that is without taking advantage of swap. Next year, I expect more.

I know it is not a common desire, so I do not blame anyone for imposing an arbitrary limit other than to say "nyah nyah; I am free".

3 comments

The "hardcoded memory limit" you see is Emacs stealing three bit of a 32 bit word as type tags for its dynamic type system.

It might be time for Emacs to switch to 64 bit words internally, should give plenty of bits left for buffer addresses after taking a few bits of for type tags.

The memory overhead when editing small files would be larger, but I haven't really heard people complain about Emacs' memory footprint lately.

> I understand emacs's architectural reasoning for it -- and that is a choice.

I think the choice they make is to spend their very limited time on more useful features. I'm sure many of the core Emacs developers would be very happy if someone would fix this limitation. It would be a very invasive change, a fair amount of work.

Any change on 32-bit machines will require special CPU instructions, massive Lisp slowdowns, or massive integer math slowdowns.

Considering you can't even buy a new 32-bit machine anymore, however, I don't really see this as a major problem.

>Considering you can't even buy a new 32-bit machine anymore, however

Netbooks.

Most netbooks that have come out in the past 6 month have a 64-bit CPU. Only the old atom N2xx series is 32bit, and they are rapidly being phased out.
I don't understand what you mean by "Viewing Large Files" - Is it the case the Emacs doesn't gracefully handle viewing greater than 512 MB Text Files? That isn't a particularly large log file (In Debug Mode, our Java APP gets there in about three-four hours), and we routinely work with files of 2 Gigabytes in size without considering them "large".