Been using Code Climate for a long time; it's really fantastic. Bryan has done a great job finding the right things to point out that help improve our code quality.
The security stuff is pretty cool - it's really nice to have code-level audits that are continuous, ongoing, and automated. It won't replace the need for physical code review but it has already caught stuff our code reviews haven't.
I'm a huge fan of automation over manual labor and static code analysis is a great tool to help. For example, in our CI (CircleCI) we tie in different ruby gems (rails_best_practices, rcov, jshint, etc.) to help enforce code quality automagically. Violate one of our team-defined guidelines? The build will fail.
I often find it frustrating that teams introduce reams of documentation for code standards and then painstakingly hand-review code for adherence. Automate that and spend the rest of your time reviewing for things that can't be trivially automated (CodeClimate doesn't care about tabs v. spaces, of course)
"Automate that and spend the rest of your time reviewing for things that can't be trivially automated" - pretty much the mantra behind any successful security program. A false-positive/negative is usually fixed faster than any feature. <3
Absolutely. It's all static analysis -- so we're pulling all the information we can out of the codebase without running the code. Under the hood, it's based on the Brakeman open source tool: https://github.com/presidentbeef/brakeman
It looks for all the big issues (XSS, CSRF, SQLi). In many cases it means keeping track of where user input enters the system (e.g. params) and how it is eventually used. So if you interpolate a param into a string and then give that string to ActiveRecord, it will produce a warning with high confidence you're vulnerable to SQL injection.
We run it every 2-3 hours on your repo, then look for new things that come up and send out alerts.
Code Climate has been great for explaining tech debt and security to my clients. It shifts the conversation from begging for time to justifying a healthy dose of refactoring.
Having used static analysis security tools for other languages, no. Ignoring the fact that it is open source, it blows away every single other tool I have used in terms of speed, accuracy, and actionability.
The security stuff is pretty cool - it's really nice to have code-level audits that are continuous, ongoing, and automated. It won't replace the need for physical code review but it has already caught stuff our code reviews haven't.
I'm a huge fan of automation over manual labor and static code analysis is a great tool to help. For example, in our CI (CircleCI) we tie in different ruby gems (rails_best_practices, rcov, jshint, etc.) to help enforce code quality automagically. Violate one of our team-defined guidelines? The build will fail.
I often find it frustrating that teams introduce reams of documentation for code standards and then painstakingly hand-review code for adherence. Automate that and spend the rest of your time reviewing for things that can't be trivially automated (CodeClimate doesn't care about tabs v. spaces, of course)