Hire them? Why? There’s nothing technically clever or novel here. Anyone can create a shell script to generate random commits and push them. I’d bet even GPT-3.5 could handle that. Why should GitHub hire them?
> There’s nothing technically clever or novel here.
Nothing technically novel. But evidently it was at least a somewhat novel stress test execution for GitHub’s live systems, otherwise surely it would have been dealt with sooner and messaged with less benefit of the doubt to the user.
Investigating the limitations of something doesn’t have to be novel to be interesting. It’s been a while (I think), but for example there’s been plenty of praise here for Netflix’s own stress tests of its live systems. The tests are often really mundane, eg just shutting some stuff off or triggering known error conditions. It’s interesting not because the nature of the fault is novel, but because systems are complex and it’s a way to learn about their failure modes.
I’m also not saying GitHub should hire them (and kinda I doubt they’d want a QA offer based on reading other blog posts on their site). Just that a hire would plausibly be similarly beneficial to a ban.
> Nothing technically novel. But evidently it was at least a somewhat novel stress test execution for GitHub’s live systems, otherwise surely it would have been dealt with sooner and messaged with less benefit of the doubt to the user.
Not really. This is boring stuff, and odds are they never bothered with it because a) it has no impact on operations, b) the blast radius of this doesn't go beyond the attacker's own repo, c) no moron with time to kill bothered attempting this stunt until now.
Probably now some low-level employee at GitHub needs to add a metric and an alarm to react to rate limits to prevent moron copycats from pulling this stunt for attention-seeking.
A more apt analogy, if we didn’t already know the general range where a given single comment thread degrades that thread (analogue to a local repo) and HN overall (analogue to the GH service), writing a bot to answer that specific question. It’s kind of wild that this yielded any new/not-widely-known information at all because it’s such an obvious thing to test. But apparently it raised at least some eyebrows on both fronts.
In load testing there's a difference between testing if something executes as specified versus testing where it breaks. I'm pretty sure that GitHub has tests to validate their performance specification. They may have tests even far in excess of that. They may not have tested where it breaks. They may have had a discussion like: what if someone tries such and such, and their answer may have been: we have good monitoring, we'll catch it before it gets out of hand, and lock out the user in question (which ultimately is what they did). In other words, spending engineering resources to determine the breaking point may not have been a priority. I'm not saying that I would agree with that in all circumstances, but it's their determination to make.
Sure, that all makes sense. It’s still true that someone stressing git and GH’s services in this particular way produced information that wasn’t especially redundant. Monitoring was good at catching it, but probably based more on service quality than on the actual thing under stress. Now there’s some data about the thing under stress, and if nothing else that allows some knob turns to calibrate monitoring. And if nothing else, that would more readily catch someone doing the same with nefarious purposes.
Nothing technically novel. But evidently it was at least a somewhat novel stress test execution for GitHub’s live systems, otherwise surely it would have been dealt with sooner and messaged with less benefit of the doubt to the user.
Investigating the limitations of something doesn’t have to be novel to be interesting. It’s been a while (I think), but for example there’s been plenty of praise here for Netflix’s own stress tests of its live systems. The tests are often really mundane, eg just shutting some stuff off or triggering known error conditions. It’s interesting not because the nature of the fault is novel, but because systems are complex and it’s a way to learn about their failure modes.
I’m also not saying GitHub should hire them (and kinda I doubt they’d want a QA offer based on reading other blog posts on their site). Just that a hire would plausibly be similarly beneficial to a ban.