| There has been a steady march of performance issues that have been made over the last few months that may not have been mentioned in the release post: * 11.8: https://gitlab.com/groups/gitlab-org/-/merge_requests?scope=... * 11.7: https://gitlab.com/groups/gitlab-org/-/merge_requests?scope=... * 11.6: https://gitlab.com/groups/gitlab-org/-/merge_requests?scope=... With regard to memory usage, we've made some progress there as well. For example: 1. We cut 80-100 MB/per Unicorn process by removing a dependency: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21008 2. A change in our merge request processing in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22725 lowered gigabytes of runtime memory from our worst and most commonly-used Sidekiq background job. 3. In the second-worst performing Sidekiq job, switching to a faster XML processor in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23136 also dropped large spikes of runtime memory. That being said, we still have a long way to go. I recently did an analysis to figure out, "Why does each Unicorn process take 400-500 MB RAM?" The note in https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4118#not... explains why. In short, there is truth that we're paying a price for Ruby overhead. What can we do about it? I have several ideas: 1. Upgrade to Ruby 2.6 (significant memory improvements for free; see https://youtu.be/ZfgxvNUfQdU) 2. Replace many Unicorn processes with a multi-threaded Puma server 3. Continue profiling and optimizing inefficient code paths 4. Make the Ruby interpreter more efficient (e.g. with a compacting garbage collector, see https://gitlab.com/gitlab-org/gitlab-ce/issues/54555) 5. Make Rails more efficient (e.g. https://github.com/rails/rails/pull/34711). Lastly, there may be significant parts of the code base that will need to be rewritten to improve memory and/or performance. Gitaly is a prime example of this: many Git-related calls have been abstracted into a Go process. |