This will get messy quickly if you have a PR that you are adding commits to and have a failure for each commit. This has the same problem as some code coverage solutions, they are spamming the PR with comments that are only important for a few moments, then they are just noise...
Speaking about finding test failures quickly though, Jenkins and many others (except ALL the cloud/SaaS solutions?) actually parse test results via junit or similar to show you failures easily. I'm wondering why more SaaS don't do this?
It's so much easier to find test failures now and doesn't involve leaving a comment on the PR. It's simply two clicks to find this view from a PR (click the status message, click on "tests")
CircleCI has been doing this for years. Not only does CircleCI read and store JUnit formatted test metadata but provides a dashboard called "Insights" to show this data overtime.
Thank you! I've often felt this way but never seen anyone else mention this. For many code coverage tools, you can integrate them at the GitHub point, which then gives you that "code coverage spam" or you can integrate it with a CI provider. This can simply fail a build if coverage drops. You can always have a status badge that shows coverage % in the readme as well.
Thanks for the feedback, we're working on methods to prevent TravisBuddy from commenting if the PR is being rapidly updated or if its comments are not relevant for some reason. One idea is that if the PR has a special label TravisBuddy won't comment on it. Another one is to be able to communicate with TravisBuddy using comments, so when you're sick of it you could just comment: "@TravisBuddy stop", and it'll stop immediately. (Maybe even "@TravisBuddy clear" to remove the previous comments)
Yeah, Jenkins' tests view was one of the inspirations for TravisBuddy :)
> This will get messy quickly if you have a PR that you are adding commits to and have a failure for each commit. This has the same problem as some code coverage solutions, they are spamming the PR with comments that are only important for a few moments, then they are just noise...
IIRC at least Codecov (https://codecov.io) has logic to go in and update the existing comment rather than spamming with new ones. It's nice!
Idea is great but this logo looks like evil douchebag kind of guy with this tagline "Not the bot you deserve, but the one you need" doesn't look great. You definitely need to work on your brand. :)
It also seems to be the avatar for the account that leaves the automated comments on the pull requests. I suspect it can cause a subliminal "this is a jerk" response in the author of the pull request who you'd probably prefer to feel receptive to this feedback.
Thanks for the feedback, the website and any part of the brand is open-source, so if you feel like contributing, you may start a discussion on GitHub or make a PR :)
I really like this idea. I think more tools should integrate into GitHub this way. The status API is nice, but you always have to click through to get the details.
I think you could do something similar with code analysis tools, so they make review comments in the code where issues occur.
I'm not a travis user, but with Jenkins there is an xunit plugin that opens up an api with test results for the specific build. since xunit formatting is standard/generic, we've implemented a parser that just gets the # of pass/failed tests and links to the build for further investigation.
We had a similar problem where posting all of the results was just more noise than useful, this is at least a good starting point for handling build result feedback.
Speaking about finding test failures quickly though, Jenkins and many others (except ALL the cloud/SaaS solutions?) actually parse test results via junit or similar to show you failures easily. I'm wondering why more SaaS don't do this?
See an example here (for our js-ipfs project): https://ci.ipfs.team/blue/organizations/jenkins/IPFS%2Fjs-ip...
It's so much easier to find test failures now and doesn't involve leaving a comment on the PR. It's simply two clicks to find this view from a PR (click the status message, click on "tests")