Hacker News new | ask | show | jobs
by bluejekyll 2136 days ago
I might have missed your attention to this thread, but I'll ask and hopefully you'll be able to check later. I have to deal with a monolithic codebase that is of excessive size (10's of millions of lines of code) and has a very large number of developers committing against it.

I haven't looked at the internals of Git much, but performance becomes an issue with it in this context (we currently use a different vcs), and switching seems impractical at the moment. Issues from local performance (30 seconds or so for git status, git checkout, etc) and clone/fetch speeds.

I'm curious, given your experience on porting, have you noticed areas where we might be able to make improvements that might be easier based on Rust's safety improvements? Such as more threading or other areas that could improve Git for use with very large code bases?

(I plan to review the project as this would be an exciting area to work in to help with productivity in this space)

1 comments

That sounds like such a project should definitely see improvements when using GitOxide. Even in projects of the size of the Linux Kernel checkouts can take time as those only reach about 3000 files per second. You will see only 70% of a single core being busy and a lot of kernel overhead. On my machine, using all the 4 cores, this number can be 7500 when checking out in parallel, something that `gitoxide` is definitely going to do from day one. After all, there is no value in just being as fast as git.

The same applies when checking for changes on disk - right now git does not do that in parallel. These are low hanging fruit that I plan to pick for my own sake.

There is another contributor who is very interested in increasing pack performance, which will directly impact fetch and pull speeds. Judging from my experience with packs thus far, I believe a lot of options are still to be tapped in that field as well. Rust will open it up for contributions and experiments to a greater audience, so I would hope that this will go way beyond of what I can do.

Even though this project is not very contributor friendly right now, I will be working on improving this so more people can join in (see https://github.com/Byron/gitoxide/issues/8).

I’m excited. Thanks for the response. This is something I will try and find time to contribute to.