Hacker News new | ask | show | jobs
by oshanz 2049 days ago
These days I'm working on log files sized around 300mb. Vim is slow and some times stop responding. Geany edior works most of the time. Do you guys have any recommendations?
6 comments

If you're not editing them, then lnav is what you need: http://lnav.org/
For reference, a 435MB web access log with about 3 million lines takes:

* lnav about 4 seconds to index[1] and display

* vim about a second to display

* emacs about 3 seconds to display

This is on a late 2013 27-inch iMac with a 3.5Ghz i7 and the file cached.

[1] - Indexing in lnav involves parsing all log lines to extract their timestamp and log level (status codes in this case)

I've edited 1GB sql dump files in Vim with no problem. I had to turn off my vimrc for it to work, but 300MB should be no problem as long as you are using barebones vim.

For performant buffer-wide operations (:g, :%s), I had success with using vim like an improved sed by passing in commands directly via the command line. That was noticeably faster than opening vim up and running the commands before exiting e.g.

    # join together lines delimited by LF into one line, then convert all CRLF into LF
    vim -Es \
        -c 'g/[^^M]$/.,/^M$/join' \
        -c ':%s/^M$//g' \
        -c 'wq'
I haven't used this but watched a talk from the author about how it can load huge files really quickly and can even edit while saving. Sounds pretty god tier in terms of performance:

https://github.com/arximboldi/ewig

Check out the talks from Juan Pedro Bolivar Puente. They are truly next level imo.

Do you have a lot of plugins? I updated my config to disabled plugins when the file is above a certain size and mostly have no problems.
Years ago a friend had a problem like that, I suggested janky old programmers notepad 2. Which worked for him.
sed
This will likely be the most correct and most underrated answer.

I wish to give you enough money for a coffee and a fancy cookie.