[vidarh]

I think you're missing my point entirely. A 2GB file is exactly the kind of absurdly large file I wrote this about:

> The moment you're willing to set an arbitrary limit above which you're ok with dropping performance, you can go very simple the way you did.

and this:

> For my own part I'm fine with falling back to e.g. emacs the one time every few months I need do do something with an unusually large file.

Point being I have no interest in making my editor handle 2GB files, because I don't edit 2GB files. I may occasionally happen to want to look at a large log file or something, in which case I'm happy to use Emacs. Or grep.

To this you answered:

> You still can't avoid the performance penalty of a tree allocation for something like bracket pairing. Or AST analysis or any one of billion things people want from a code editor.

And at least for bracket pairing this is true if you want to apply it to 2GB files and want to be able to quickly find the closing token, but that was explicitly out of scope of what I'm talking about; I'd explicitly excluded large files from consideration in the comment you replied to.

For the types of files I'm interested in handling - up to tens of thousands of lines but beyond that I really don't care - just storing the parser state per line coupled with linearly scanning as needed is more than fast enough. Even in Ruby, which is what I'm using.

You'll note my objection was not to doing analysis, but to the need for keeping any more complex structure in sync over the data. My editor uses just a plain array of lines represented as strings.

It will break horribly on huge files.

That's perfectly fine.

The point of the comment you replied to was explicitly to argue that for some editors it'd be just fine to make the explicit design choice not to cater for extreme cases like that.

Of course I'm not arguing for all editors to do this. Then I wouldn't have anything to use on the rare cases I need it. I'm arguing for people to consider whether their editor need to able to handle all kinds of outliers; maybe it need to - Jetbrains probably do need their editor to for commercial reasons - maybe it doesn't; mine does not need to.

Instead I optimise for what makes it pleasant for me for my common case, and fall back to emacs the once every few months where I for some reason need to do something with an absurdly large file.

[Ygg2]

It's confusing you're replying to two different threads in this

> For my own part I'm fine with falling back > That's perfectly fine.

That's for you. I want to open a huge XML file and edit it with autocomplete.

I mean you could just say. I'm ok with Notepad. Ok. Sure. I'm not.

> My editor uses just a plain array of lines represented as strings.

Any naive approach is going to run into edge cases quickly[1]. Any more sophisticated approach is going to run into performance issues.

> It will break horribly on huge files.

I suspect it will break waaaay before that. Even using some language with greater complexity would cause issues.

[1] For example using regex to color syntax. Reasonably fast yes, but fails in any moderately complex scenario.

[vidarh]

> It's confusing you're replying to two different threads in this

I get that, which is why I specified what it was I had replied to in order to clear it up.

> That's for you. I want to open a huge XML file and edit it with autocomplete.

That's fine. My editor is for me, not for you. To me, if you have to do bracket matching on a 2GB XML file an editor is the wrong tool. I'd work with that in a REPL. But that's me. You're totally free to pick your editor based on the need to handle 2GB files, just as I'm free not care one iota about opening 2GB files.

Hence my point that a lot of people don't seem to think about whether or not they actually need this. I decided I don't need it, and it let me simplify things a lot. It won't work for everyone, and that's fine.

> I mean you could just say. I'm ok with Notepad. Ok. Sure. I'm not.

Well, if I'd been ok with notepad, I'd have used notepad. I wrote my own editor because I'm not ok with Notepad. Or Emacs. The tradeoffs I care about are very different.

> Any naive approach is going to run into edge cases quickly[1]. Any more sophisticated approach is going to run into performance issues.

For large files, yes. It's more than fast enough to run just fine - even in pure Ruby - on files of the sizes I pointed out are the sizes I've decided I care about.

> [1] For example using regex to color syntax. Reasonably fast yes, but fails in any moderately complex scenario.

Which is why I'm using Rouge for the syntax-highlighting for my editor. It uses Regex for low level lexing where people choose to, but it supports state machines and multi-layered lexing or your own custom code. It supports all the languages I care about and dozens more just fine. My approach of caching the lexer state works just fine with Rouge and the approach I've outlined to e.g. parse my Ruby code with a precise Ruby mode, switch to a Markdown mode inside comment blocks, and feed the output of that into a layer that applies further custom rules based on my preferences. All without needing to build an AST for any of those.

> I suspect it will break waaaay before that. Even using some language with greater complexity would cause issues.

It handles every language supported by Rouge [1] just fine, and that's far more than I have need for.

[1] https://github.com/rouge-ruby/rouge/blob/master/docs/Languag...

[Ygg2]

> That's fine. My editor is for me, not for you. To me, if you have to do bracket matching on a 2GB XML file an editor is the wrong tool. I'd work with that in a REPL. But that's me. You're totally free to pick your editor based on the need to handle 2GB files, just as I'm free not care one iota about opening 2GB files.

Ok. But tbh I hate REPLs and/or CLI. Text is just better served by a visual text editor.

Then you can't really compare something for general use case to something used for you.

> Which is why I'm using...

It was an example how naive approaches failed. Your personal editor doesn't particularly interests me.

[vidarh]

My point was exactly that it's not necessary for every editor to aim to support the general case, because not everyone need that, and aiming to support the general case adds a lot of complexity. I'm glad some editors do that, because some will need it.

For my part I love a repl. My editor has a repl too. Different needs.

> It was an example how naive approaches failed. Your personal editor doesn't particularly interests me.

You made pronouncements about how things would fail that you implied would affect a simplified model. I've pointed out why those assumptions don't hold under the design constraints I stated in my first comments. I get that you replied on the basis of the general case. That's fine - we simply talked past each other in that respect.

In that respect my editor serves as an example of viability, that's all. Nobody is asking you to be interested in it. If anything I'm explicitly warning people against expecting it to be usable for them.

That your needs from an editor are different is fine. I'm not questioning the validity of your needs for you.

[Ygg2]

> You made pronouncements about how things would fail that you implied would affect a simplified model.

And you said, they do fail? Can I load a several GB of data from weather or LHC?

I haven't really heard of Rouge or if something else that uses it. But some of the choices you mentioned (like external services doing code analysis) is some of the stuff I've seen as dead ends when making a code editor (see Lure of Modular software).

https://raphlinus.github.io/xi/2020/06/27/xi-retrospective.h...

------

Like ok, you made a toy editor that works in your case. Ok.

Good for you, but for me and probably 90% of people, if it can't open large file, can't automcomplete, can't reference XSD schema, can't reference, can't give good overall structure, can't refactor, can't recognize SQL in strings, can't deal with macros, source gens, debugging etc. It's not that useful.

[vidarh]

> And you said, they do fail? Can I load a several GB of data from weather or LHC?

It "fails" in the same way a car fails to fly. I set out my requirements. It meets my requirements. You're intent on arguing over how my car really should have wings and a jet engine.

> I haven't really heard of Rouge or if something else that uses it.

Gitlab is one of the most prominent examples.

> But some of the choices you mentioned (like external services doing code analysis) is some of the stuff I've seen as dead ends when making a code editor (see Lure of Modular software).

Every code editor that uses language servers makes that choice. Most modern code editors either have direct support or extensions providing client support for it, with a more coming on a regular basis. I was sceptical too, but writing my own editor very quickly demonstrated that the benefits far outweighed the downsides.

It's only hard to make this work if you make it hard on yourself. Xi made it hard on themselves by implementing their own RPC and using CRDTs instead of using "off the shelf" components and making functionality tradeoffs. That's a valid choice, but it makes their experiences with modularity relatively uninteresting when considering editors that makes entirely different choices.

To specifically address one point in the link you gave:

> I think it’s worth asking the question: is it useful to have the front-end continue after the core crashes, or the other way around? I think probably not; in the latter case it might be able to safely save the file, but you can also do that by frequently checkpointing.

E.g. to this I'd answer a resounding yes, because unlike their design 1) because of the use of Ruby and DrB, most bugs that could have taken down the server doesn't take down the server because the exceptions gets forwarded to the client. 2) every window is a client in my case, so a crashing frontend means one window goes. My - as of writing this - 16 other open editor windows do not. I can make changes to the editor itself and only affect the frontends I choose to restart. I've kept meaning to add ability for the clients to reconnect, but because of #1 it's exceptionally hard to make the server crash even if I make a severely buggy update, so it's stayed far down my priority list.

More importantly, this approach meant I did not have to build any GUI support at all to support multi-window editing. Splitting a pane is done by starting another client. It's part of a general decision to focus on minimalism.

Xi had different design goals, and Xi made tradeoffs that always sounded like a lot of bikeshedding ("hand-rolled our own xi-rpc layer" for example) to me. A lot of it was interesting, but at the same time it's pretty much the opposite of what I aimed for. Xi aimed for perfectionism where I aimed for MVPs. The Xi rope implementation alone has a higher LOC count than my entire editor... If that meets their goals, then that's fine.

My entire backend is ~500 lines. Including the buffer storage. It wouldn't be that small if it tried to cater for 2GB files, or tried to cater for concurrent editing of the same piece of data from multiple clients or cater for multiple frontends written by different people in different languages.

What my design tradeoffs got me was the ability to build an editor that was capable of being my main work tool within a few months.

My point is not that other people should use my editor. Absolutely not - I have no interest in supporting that. My point is that by building a special purpose tool and picking ready-made components where possible and making choices that focuses on modular design and optimises for simplicity over making something general, I now have an editor tailor-made for my own needs in a way that other editors I have used didn't.

> Like ok, you made a toy editor that works in your case. Ok.

The fact that it works in my case means it is not a toy editor, no matter how rude you try to be about it. It's my main work tool, and has been for several years, out of choice. I have plenty of editors installed. I switched slowly, over time, as I increasingly got the features in place that made me actually prefer to use it. That it's not a tool that suits you does not make it a toy.

This kind of attitude that something needs to be generalist to be worthwhile is something I've opposed my entire career. A whole lot of things would be a lot better if more people carefully considered whether to build yet another generalist tool or whether to build specialist tooling that does a smaller subset of things well.

I get paid a lot of money to write specialist tools that will never be available to most people, and would be of little use to most people.

> Good for you, but for me and probably 90% of people, if it can't open large file, can't automcomplete, can't reference XSD schema, can't reference, can't give good overall structure, can't refactor, can't recognize SQL in strings, can't deal with macros, source gens, debugging etc. It's not that useful.

That's fine. I didn't write my editor with the intent of 90% of people using it. I wrote it with the intent of using it myself. It has the features I prioritise, and not a single feature I don't use. That means alongside the software development I do to e.g. support the ability to read the front matter of the novels I write as a hobby and cross reference and highlight character and location names matters, while XSD doesn't matter to me one bit. Not for the last 15 years at least. If XSD had mattered to me, I'd have added support for it, as supporting it is trivial (been there, written schema validators and code generators, got the PTSD).

You seem to still struggle with the idea that not every editor need to be designed to cater for you or for the needs of everyone else. Which demonstrates that you're still struggling to deal with my original point:

That there are benefits to not aiming to build a general purpose tool if a general purpose tool isn't what you need.