Hacker News new | ask | show | jobs
by stanhu 2674 days ago
There has been a steady march of performance issues that have been made over the last few months that may not have been mentioned in the release post:

* 11.8: https://gitlab.com/groups/gitlab-org/-/merge_requests?scope=...

* 11.7: https://gitlab.com/groups/gitlab-org/-/merge_requests?scope=...

* 11.6: https://gitlab.com/groups/gitlab-org/-/merge_requests?scope=...

With regard to memory usage, we've made some progress there as well. For example:

1. We cut 80-100 MB/per Unicorn process by removing a dependency: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21008

2. A change in our merge request processing in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22725 lowered gigabytes of runtime memory from our worst and most commonly-used Sidekiq background job.

3. In the second-worst performing Sidekiq job, switching to a faster XML processor in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23136 also dropped large spikes of runtime memory.

That being said, we still have a long way to go. I recently did an analysis to figure out, "Why does each Unicorn process take 400-500 MB RAM?" The note in https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4118#not... explains why. In short, there is truth that we're paying a price for Ruby overhead.

What can we do about it? I have several ideas:

1. Upgrade to Ruby 2.6 (significant memory improvements for free; see https://youtu.be/ZfgxvNUfQdU)

2. Replace many Unicorn processes with a multi-threaded Puma server

3. Continue profiling and optimizing inefficient code paths

4. Make the Ruby interpreter more efficient (e.g. with a compacting garbage collector, see https://gitlab.com/gitlab-org/gitlab-ce/issues/54555)

5. Make Rails more efficient (e.g. https://github.com/rails/rails/pull/34711).

Lastly, there may be significant parts of the code base that will need to be rewritten to improve memory and/or performance. Gitaly is a prime example of this: many Git-related calls have been abstracted into a Go process.

1 comments

Thanks for putting that post together - that is very informative.