[duncaen]

See page 9 of the already published paper:

https://raw.githubusercontent.com/QiushiWu/qiushiwu.github.i...

> We send the emails to the Linux communityand seek their feedback. The experiment is not to blame any maintainers but to reveal issues in the process. The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research. We obtained a formal IRB-exempt letter. The experiment will not collect any personal data, individual behaviors, or personal opinions. It is limited to studying the patching process OSS communities follow, instead of individuals.

[nkurz]

> The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research.

I'm not sure how it affects things, but I think it's important to clarify that they did not obtain the IRB-exempt letter in advance of doing the research, but after the ethically questionable actions had already been taken:

The IRB of UMN reviewed the study and determined that this is not human research (a formal IRB exempt letter was obtained). Throughout the study, we honestly did not think this is human research, so we did not apply for an IRB approval in the beginning. ... We would like to thank the people who suggested us to talk to IRB after seeing the paper abstract.

https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc....

[catgary]

I'm a bit shocked that the IRB gave an exemption letter - are they hoping that the kernel maintainers won't take the (very reasonable) step towards legal action?

[Arainach]

What "legal action" do you think applies here?

[eschaton]

Intentional misrepresentation that causes harm is commonly referred to as “fraud.”

[slenk]

I don't think any legal actions need to be taken. UMN can longer participate. Tough shit.

[madpata]

Is it illegal to intentionally create security risks?

[slenk]

I honestly do not know

[jasonwatkinspdx]

I'd guess they may not have understood what actually happened, or were leaning heavily on the IEEE reviewers having no issues with the paper, as at that point it'd already been excepted.

[volta83]

> We send the emails to the Linux communityand seek their feedback.

That's not really what they did.

They sent the patches, the patches where either merged or rejected.

And they never let anybody knew that they had introduced security vulnerabilities on the kernel on purpose until they got caught and people started reverting all the patches from their university and banned the whole university.

[duncaen]

This is not what happened according to them:

https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc....

> (4). Once any maintainer of the community responds to the email, indicating “looks good”, we immediately point out the introduced bug and request them to not go ahead to apply the patch. At the same time, we point out the correct fixing of the bug and provide our proper patch. In all the three cases, maintainers explicitly acknowledged and confirmed to not move forward with the incorrect patches. This way, we ensure that the incorrect patches will not be adopted or committed into the Git tree of Linux.

[tecleandor]

It'd be great if they pointed to those "please don't merge" messages on the mailing list or anywhere.

Seems like there are some patches already on stable trees [1], so they're either lying, or they didn't care if those "don't merge" messages made anybody react to them.

1 - https://lore.kernel.org/linux-nfs/CADVatmNgU7t-Co84tSS6VW=3N...

[treesknees]

The paper doesn't cite specific commits used. It's possible that any of the commits in stable are actually good commits and not part of the experiment. I support the ban/revert, I'm just pointing out there's a 3rd option you didn't touch on.

[coriny]

Patches with built-in bugs made it to stable: https://lore.kernel.org/linux-nfs/YIAta3cRl8mk%2FRkH@unreal/.

[bmcahren]

Here's the commit specifically identified by Leon Romanovsky as having a "built-in bug"

https://github.com/torvalds/linux/commit/8e949363f017

[corty]

Also, they are talking of three cases. However, the list of patches to be reverted by gregkh is far longer than three, more than a hundred. Most of the first batch look sufficiently similar that I would guess all of them are part of this "research". So the difference in numbers alone points to them most probably lying.

[finnthehuman]

I was more ambivalent about their "research" until I read that "clarification." It's weaselly bullshit.

>> The work taints the relationship between academia and industry

> We are very sorry to hear this concern. This is really not what we expected, and we strongly believe it is caused by misunderstandings

Yeah, misunderstandings by the university that anyone, ever, in any line of endeavor would be happy to be purposely fucked with as long as the perpetrator eventually claims it's for a good cause. In this case the cause isn't even good, they're proving the jaw-droppingly obvious.

[PeterisP]

The first step of an apology is admitting the misdeed. Here they are explicitly not acknowledging that what they did was wrong, they are still asserting that this was a misunderstanding.

[rualca]

Even their choice of wording ("We are very sorry to hear this concern.") is the blend of word fuckery that conveys the idea they care nothing about what they did or why it negatively affected others.

[HackOfAllTrades]

>We are very sorry to hear this concern.

..."Because if we're lucky tomorrow, we won't have to deal with questions like yours ever again." --Firesign Theater, "I Think We're All Bozos on the Bus"

[dwild]

> they're proving the jaw-droppingly obvious.

Yet we do nothing about it? I wouldn't call that jaw-droppingly obvious, if anything, without this, I'm pretty sure that anyone would argue that it would be caught way before making it way into stable.

[rurp]

I've literally never come across an open source project that was thought to have a bullet proof review process or had a lack of people making criticisms.

What they do almost universally lack is enough people making positive contributions (in time, money, or both).

This "research" falls squarely into the former category and burns resources that could have been spent on the latter.

[Apofis]

This is zero percent different from a bad actor and hopefully criminal. I think a lot of maintainers work for large corporations like Microsoft, Oracle, Ubuntu, Red Hat, etc... I think these guys really stepped in it.

[incrudible]

> And they never let anybody knew that they had introduced security vulnerabilities on the kernel on purpose...

Yes, that's the whole point! The real malicious actors aren't going to notify anyone that they're injecting vulnerabilities either. They may be plants at reputable companies, and they'll make it look like an "honest mistake".

Had this not been caught, it would've exposed a major flaw in the process.

> ...until they got caught and people started reverting all the patches from their university and banned the whole university.

Either these patches are valid fixes, in which case they should remain, or they are intentional vulnerabilities, in which case they should've already been reviewed and rejected.

Reverting and reviewing them "at a later date" just makes me question the process. If they haven't been reviewed properly yet, it's better to do it now instead of messing around with reverts.

[simiones]

This reminds me of that story about Go Daddy sending everyone "training phishing emails" announcing that they had received a company bonus - with the explanation that this is ok because it is a realistic pretext that real phishing may use.

While true, it's simply not acceptable to abuse trust in this way. It causes real emotional harm to real humans, and while it also may produce some benefits, those do not outweigh the harms. Just because malicious actors don't care about the harms shouldn't mean that ethical people shouldn't either.

[incrudible]

This isn't some employer-employee trust relationship. The whole point of the test is that you can't trust a patch just because it's from some university or some major company.

[simiones]

The vast majority of patches are not malicious. Sending a malicious patch (one that is known to introduce a vulnerability) is a malicious action. Sending a buggy patch that creates a vulnerability by accident is not a malicious action.

Given the completely unavoidable limitations of the review and bug testing process, a maintainer has to react very differently when they have determined that a patch is malicious - all previous patches past from that same source (person or even organization) have to be either re-reviewed at a much higher standard or reverted indiscriminately; and any future patches have to be rejected outright.

This puts a heavy burden on a maintainer, so intentionally creating this type of burden is a malicious action regardless of intent. Especially given that the intent was useless in the first place - everyone knows that patches can introduce vulnerabilities, either maliciously or by accident.

[incrudible]

> The vast majority of patches are not malicious.

The vast majority of drunk drivers never kill anyone.

> Sending a malicious patch (one that is known to introduce a vulnerability) is a malicious action.

I disagree that it's malicious in this context, but that's irrelevant really. If the patch gets through, then that proves one of the most critical pieces of software could relatively easily be infiltrated by a malicious actor, which means the review process is broken. That's what we're trying to figure out here, and there's no better way to do it than replicate the same conditions under which such patches would ordinarily be reviewed.

> Especially given that the intent was useless in the first place - everyone knows that patches can introduce vulnerabilities, either maliciously or by accident.

Yes, everyone knows that patches can introduce vulnerabilities if they are not found. We want to know whether they are found! If they are not found, we need to figure out how they slipped by and how to prevent that from happening in the future.

[bluesnowmonkey]

Open source runs on trust, of both individuals and institutions. There’s no alternative. Processes like code review can supplement but not replace it.

[nomel]

So my question is, what is a kernel? Is it a security project? Should security products rely on trust, or assume malicuous intent?

[powersnail]

> Yes, that's the whole point!

Well, in real life, you can't go punch someone in the face to teach them a "point". If you do so, you'll get punished.

> Reverting and reviewing them "at a later date" just makes me question the process.

I don't think anybody realistically thought that the kernel review process is rock solid against malicious anyway. What exactly does the paper expose?

[darkarmani]

> Yes, that's the whole point! The real malicious actors aren't going to notify anyone that they're injecting vulnerabilities either. They may be plants at reputable companies, and they'll make it look like an "honest mistake".

This just turns the researchers into black hats. They are just making it look like "a research paper."

[protomyth]

The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research.

How is this not human research? They experimented on the reactions of people in a non-controlled environment.

[hinkley]

Sounds like the IRB of UMn needs some scrutiny as well.

[dekhn]

This is exactly what I would have said: this sort of research isn't 'human subjects research' and therefore is not covered by an IRB (whose job it is to prevent the university from legal risk, not to identify ethically dubious studies).

It is likely the professor involved here will be fired if they are pre-tenure, or sanctioned if post-tensure.

[dfranke]

How in the world is conducting behavioral research on kernel maintainers to see how they respond to subtly-malicious patches not "human subject research"?

[alxlaz]

In the restricted sense of Title 45, Part 46, it's probably not quite human subject research (see https://www.hhs.gov/ohrp/regulations-and-policy/regulations/... ).

Of course, there are other ethical and legal requirements that you're bound to, not just this one. I'm not sure which requirements IRBs in the US look into though, it's a pretty murky situation.

[dlgeek]

How so?

It seems to qualify per §46.102(e)(1)(i) ("Human subject means a living individual about whom an investigator [..] conducting research: (i) Obtains information [...] through [...] interaction with the individual, and uses, studies, or analyzes the information [...]")

I don't think it'd qualify for any of the exemptions in 46.104(d): 1 requires an educational setting, 2 requires standard tests, 3 requires pre-consent and interactions must be "benign", 4 is only about the use of PII with no interactions, 5 is only about public programs, 6 is only about food, 7 is about storing PII and not applicable and 8 requires "broad" pre-consent and documentation of a waiver.

[dekhn]

rather than arguing about the technical details of the law, let me just clarify: IRBs would actively reject a request to review this. It's not in their (perceived) purview.

It's not worth arguing about this; if you care, you can try to change the law. In the meantime, IRBs will do what IRBs do.

[shkkmo]

If the law, as written, does actually classify this as human research, it seems like the correct response is to sue the University for damages under that law.

Since IRBs exist to minimize liability, it seems like that would be that fastest route towards change (assuming you have legal standing )

[Tobu]

That's still 10 thousand words you're linking to…

I had a look at section §46.104 https://www.hhs.gov/ohrp/regulations-and-policy/regulations/... since it mentioned exemptions, and at (d) (3) inside that. It still doesn't apply: there's no agreement to participate, it's not benign, it's not anonymous.

[dfranke]

If there's some deeply legalistic answer explaining how the IRB correctly interpreted their rules to arrive at the exemption decision, I believe it. It'll just go to show the rules are broken.

IRBs are like the TSA. Imposing annoyance and red tape on the honest vast-majority while failing to actually filter the 0.0001% of things they ostensibly exist to filter.

[dekhn]

are you expecting that science and institutions are rational? If I was on the IRB, I wouldn't have considered this since it's not a sociological experiment on kernel maintainers, it's an experiment to inject vulnerabilities in a source code. That's not what IRBs are qualified to evaluate.

[ebcode]

> it's an experiment to inject vulnerabilities in a source code

I'm guessing it passed for similar reasoning, along with the reviewers being unfamiliar with how "vulnerabilities are injected." To get the bad code in, the researcher needed to have the code reviewed by a human.

So if you rephrase "inject vulnerability" as "sneak my way past a human checkpoint", you might have a better idea of what they were actually doing, and might be better equipped to judge its ethical merit -- and if it qualifies as research on human subjects.

To my thinking, it is quite clearly human experimentation, even if the subject is the process rather than a human individual. Ultimately, the process must be performed by a human, and it doesn't make sense to me that you would distinguish between the two.

And the maintainers themselves express feeling that they were the subject of the research, so there's that.

[ClumsyPilot]

Testing airport security by putting dangerous goods in your luggage is not human experimentation. Testing a Bank's security is not human experimentation. Testing border securiry is not.

What makes people revieing linux kernel more 'human' than any of the above?

[salawat]

Tell that to the person on the hook if or when they get caught.

[CRConrad]

It's not an experiment in computer science; these guys aren't typing code into an editor and testing what the code does after they've compiled it. They're contributing their vulnerabilities to a community of developers and testing whether these people accept it. It is absolutely nothing else than a sociological experiment.

[ufmace]

This reminds me of a few passages in the SSC post on IRBs[0].

Main point is that IRBs were created in response to some highly unethical and harmful "studies" being carried out by institutions thought of as top-tier. Now they are considered to be a mandatory part of carrying out ethical research. But if you think about it, isn't outsourcing all sense of ethics to an organization external to the actual researchers kind of the opposite of what we want to do?

All institutions tend to be corruptible. Many tend to respond to their actual incentives rather than high-minded statements about what they're supposed to be about. Seems to me that promoting the attitude of "well an IRB approved it, so it must be all right, let's go!" is the exact opposite of what we really want.

All things considered, it's probably better to have something there than nothing. But you still have to be responsible for your own decisions. I bamboozled our lazy IRB into approving our study, so I'm not responsible for it being obviously a bad idea, just isn't good enough.

If you think about it, it's actually kind of meta to the code review process they were "studying". Just like IRBs, Code review is good, but no code review process will ever be good enough to stop every malicious actor every time. It will always be necessary to track the reputation of contributors and be able to mass-revert contributions from contributors later determined to be actively malicious.

[0] https://slatestarcodex.com/2017/08/29/my-irb-nightmare/

[dekhn]

I guess I have a different perspective. I know a fair number of world class scientists; like, the sort of people you end up reading about as having changed the textbook. One of these people, a well-known bacteriologist, brought his intended study to the IRB for his institution (UC Boulder), who said he couldn't do it because of various risks due to studying pathogenic bacteria. The bacteriologist, who knew far more about the science than the IRB, explained everything in extreme detail and batted away each attempt to shut him down.

Eventually, the IRB, unhappy at his behavior, said he couldn't do the experiment. He left for another institution (UC San Diego) immediately, having made a deal with the new dean to go through expedited review. It was a big loss for Boulder and TBH, the IRB's reasoning was not sound.

[karlmdavis]

Communities aren’t people? What in the actual fuck is going on with this university’s IRB?!

[tikiman163]

They weren't studying the community, they were studying the patching process used by that community, which a normal IRB would and should consider to be research on a process and therefore not human Research. That's how they presented it to the IRB so it got passed even if what they were claiming was clearly bullshit.

This research had the potential to cause harm to people despite not being human research and was therefore ethically questionable at best. Because they presented the research as not posing potential harm to real people that means they lied to the IRB, which is grounds for dismissal and potential discreditation of all participants (their post-graduate degrees could be revoked by their original school or simply treated as invalid by the educational community at large). Discreditation is unlikely, but loss of tenure for something like this is not out of the question, which would effectively end the professor's career anyway.

[d110af5ccf]

> This research had the potential to cause harm to people

I don't buy it, and you fail to back that claim up at all.

[liamwire]

At a minimum, is needlessly increasing the workload of an unwitting third party considered a harm? I ask, because I’d be pretty fucking mad if someone came along and added potentially hundreds of man-hours of work in the form of code review to my life.

[d110af5ccf]

Considering that the number of patches submitted was quite limited I don't think the original research paper would qualify as a DoS attack. The workload imposed by the original research appears to have been negligible compared to the kernel effort as a whole, no more than any drive by patch submission might result in. So no, I wouldn't personally view that as harmful.

As to the backdated review now being undertaken, as far as I'm concerned that decision is squarely on the maintainers. (Honestly it comes across as an emotional outburst to me.)

[drdeca]

If you steal only $5 from me, I am still harmed. If you break my computer monitor, then even if I can afford a replacement, I am still harmed.

[DetroitThrow]

In my experience in university research, the correct portrayal of the ethical impact is the burden of the researchers unfortunately, and the most plausible explanation in my view given their lack of documentation of the request for IRB exemption would be that they misconstrued the impact of the research.

It seems very possible to me that an IRB wouldn't have accepted their proposed methodology if they hadn't received an exemption.

[emeraldd]

> The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research. We obtained a formal IRB-exempt letter.

Is there anyone on hand who could explain how what looks very much like a social engineering attack is not "human research"?