[Cpoll]

A lot of people are talking about the ethical aspects, but could you talk about the security implications of this attack?

From a different thread: https://lore.kernel.org/linux-nfs/CADVatmNgU7t-Co84tSS6VW=3N... > A lot of these have already reached the stable trees.

Apologies in advance if my questions are off the mark, but what does this mean in practice?

1. If UNM hadn't brought any attention to these, would they have been caught, or would they have eventually wound up in distros? 'stable' is the "production" branch?

2. What are the implications of this? Is it possible that other malicious actors have done things like this without being caught?

3. Will there be a post-mortem for this attack/attempted attack?

[neoflame]

I don't think the attack described in the paper actually succeeded at all, and in fact the paper doesn't seem to claim that it did.

Specifically, I think the three malicious patches described in the paper are:

- UAF case 1, Fig. 11 => crypto: cavium/nitrox: add an error message to explain the failure of pci_request_mem_regions, https://lore.kernel.org/lkml/20200821031209.21279-1-acostag.... The day after this patch was merged into a driver tree, the author suggested calling dev_err() before pci_disable_device(), which presumably was their attempt at maintainer notification; however, the code as merged doesn't actually appear to constitute a vulnerability because pci_disable_device() doesn't appear to free the struct pci_dev.

- UAF case 2, Fig. 9 => tty/vt: fix a memory leak in con_insert_unipair, https://lore.kernel.org/lkml/20200809221453.10235-1-jameslou... This patch was not accepted.

- UAF case 3, Fig. 10 => rapidio: fix get device imbalance on error, https://lore.kernel.org/lkml/20200821034458.22472-1-acostag.... Same author as case 1. This patch was not accepted.

This is not to say that open-source security is not a concern, but IMO the paper is deliberately misleading in an attempt to overstate its contributions.

edit: wording tweak for clarity

[ununoctium87]

> the paper is deliberately misleading in an attempt to overstate its contributions.

Welcome to academia. Where a large number of students are doing it just for the credentials

[DSingularity]

What else do you expect? The incentive structure in academia pushes students to do this.

Immigrant graduate students with uncertain future if they fail? Check.

Vulnerable students whose livelihood is at mercy of their advisor? Check.

Advisor whose career depends on a large number of publication bullet points in their CV? Check.

Students who cheat their way through to publish? Duh.

[chalst]

The ethics in big-lab science are as dire as you say, but I've generally got the impression that the publication imperative has not been driving so much unethical behaviour in computer science. I regard this as particularly cynical behaviour by the standards of the field and I think the chances are good that the article will get retracted.

[chalst]

FWIW, Qiushi Wu's USENIX speaker page links to a presentation with Aditya Pakki (and Kangjie Lu), but has no talk with the same set of authors as the paper above.

https://www.usenix.org/conference/usenixsecurity19/speaker-o...

[partomniscient]

Can I cite your comment in exchange for a future citation?

[DSingularity]

Sure?

Edit: Oh now I get it you clever person you. Only took an hour ha.

[droopyEyelids]

Feigning surprise isn't helpful.

It's good to call out bad incentive structures, but by feigning surprise you're implying that we shouldn't imagine a world where people behave morally when faced with an incentive/temptation.

[Stratoscope]

I dislike feigned surprise as much as you do, but I don't see it in GP's comment. My read is that it was a slightly satirical checklist of how academic incentives can lead to immoral behavior and sometimes do.

I don't think it's fair to say "by feigning surprise you're implying..." That seems to be putting words in GP's mouth. Specifically, they didn't say that we shouldn't imagine a better world. They were only describing one unfortunate aspect of today's academic world.

Here is a personal example of feigned surprise. In November 2012 I spent a week at the Google DC office getting my election results map ready for the US general election. A few Google engineers wandered by to help fix last-minute bugs.

Google's coding standards for most languages including JavaScript (and even Python!) mandate two-space indents. This map was sponsored by Google and featured on their site, but it was my open source project and I followed my own standards.

One young engineer was not pleased when he found out about this. He took a long slow look at my name badge, sighed, and looked me in the eye: "Michael... Geary... ... You... use... TABS?"

That's feigned surprise.

(Coda: I told him I was grateful for his assistance, and to feel free to indent his code changes any way he wanted. We got along fine after that, and he ended up making some nice contributions.)

[thegrimmest]

Why should we imagine this world? We have no reason to believe it can exist. People are basically chimps, but just past a tipping point or two that enable civilization.

We'd also have to agree on what "behave morally" means, and this is impossible even at the most basic level.

[hoseja]

Usually "behave morally" means "behave in a way the system ruling over you deems best to indoctrinate into you so you perpetuate it". No, seriously, that's all there is to morality once you invent agriculture.

[Abishek_Muthian]

Thank you.

Question for legal experts,

Hypothetically if these patches were accepted and was exploited in the wild; If one could prove that they were exploited due to the vulnerabilities caused by these patches can the University/ Prof. be sued for damages and won in an U.S. court (or) Would they get away under Education/Research/Academia cover if any?

[patmorgan23]

Not an attorney but the kernal is likely shielded from liability by it's license. maybe the kernal could sue the contributers for damaging the project but I don't think the end user could.

[liamwestray]

Malicious intent or personal gain negate that sort of thing in civil torts.

Also US code 1030(a)5 A does not care about software license. Any intentional vulnerability added to code counts. Federal cybercrime laws are not known for being terribly understanding…

[Abishek_Muthian]

License is a great catch, thank you. Do the kernel get into separate contract with the contributors?

[gjack905]

I literally LOL'd at "James Louise Bond"

[cutemonster]

I wonder about this me too.

To me, seems to indicate that nation state supported evil hacker org (maybe posing as an individual) could place their own exploits in the kernel. Let's say they contribute 99.9% useful code, solve real problems, build trust over some years, and only rarely write an evil hard to notice exploit bug. And then, everyone thinks that obviously it was just an ordinary bug.

Maybe they can pose as 10 different people, in case some of them gets banned.

[Spooky23]

You're still in a better position with open source. The same thing happens in closed source companies.

See: https://www.reuters.com/article/us-usa-security-siliconvalle...

"As U.S. intelligence agencies accelerate efforts to acquire new technology and fund research on cybersecurity, they have invested in start-up companies, encouraged firms to put more military and intelligence veterans on company boards, and nurtured a broad network of personal relationships with top technology executives."

Foreign countries do the same thing. There are numerous public accounts of Chinese nationals or folks with vulnerable family in China engaging in espionage.

[varjag]

The principal researches appear to be alumni of mainland China schools.

[baby]

Read into the socat diffie-hellman backdoor, I found it fascinating at the time.

[throwaway2037]

Woah. I Googled that! Nice reference. This is a good explanation with more links: https://github.com/AllThing/socat_backdoor

[TheSpiceIsLife]

Isn't what you've described pretty much the very definition of advanced persistent threat?

It's difficult to protect against trusted parties whom you assume, with good reason, and good-faith actors.

[ethbr0]

The fundamental tension is between efficiency and security. Trust permits efficiency, at the cost of security (if that trust is found to be misplaced).

A perfectly security system is only realized by a perfectly inefficient development process.

We can get better at lessening the efficiency tax of a given security level (through tooling, tests, audits, etc), but for a given state of tooling, there's still a trade-off.

Different release trains seem the sanest solution to this problem.

If you want bleeding-edge, you're going to pull in less-tested (and also less-audited) code. If you want maximum security, you're going to have to deal with 4.4.

[mk89]

I have the same questions. So far we have focused on how bad these "guys" are. Sure, they could have done it differently, etc. However, they proved a big point: how "easy" it is to manipulate the most used piece of software on the planet.

How to solve this "issue" without putting too much process around it? That's the challenge.

[_d7dt]

What's next, will they prove how easy it is to break into kernel developers' houses and rob them? Or prove how easy it is to physically assault kernel developers by punching them in the face at conferences? Or prove how easy it is to manipulate kernel developers to lose their life savings investing in cryptocurrency? You can count me out of those...

Sarcasm aside, pentesting/redteaming is only ethical if the target consents to it! Please don't try to prove your point the way these researchers have.

[nsonha]

Just playing devil advocate here, the surprising factor does play into it. No bad actor will ever give you heads-up.

If the researcher has sent these patches under a different identity, that would be just like how malice contributions appear. The maintainers won't assume malice, waste a bunch of time communicating with the bad actor, and may NOT revert their previous potentially harmful contribution.

[newswasboring]

> the surprising factor does play into it. No bad actor will ever give you heads-up.

I too thought like this till yesterday. Then someone made me realize thats not how getting consent works in these situations. You take consent from higher up the chain, not the people doing the work. So Greg Kroah-Hartmancould could have been consulted, as he would not be personally reviewing this stuff. This would also give you a chance to understand how the release process works. You also have an advantage over the bad actors because they would be studying the process from outside.

[nsonha]

it's not simple like that, if Greg doesn't do the work of review then who gives him the authority to consent on behalf of others?

[newswasboring]

I see what you are saying. But he is also sort of the director to this whole thing. The research question itself is worthwhile and I don't think if it was done properly this much time would be wasted. All they have to prove is that it will pass few code reviews. That's a few man hours and I really don't think people will be mad about that. This whole fiasco is about the scale of man hours wasted both because they repeatedly made these "attacks" and because this thing slipped into stable code. Both would be avoided in this scheme.

But I would like to put in a disclaimer that before getting to that point they could have done so many other things. Review the publicly available review processes, see how security bugs get introduced by accident and see if that can be easily done by a bad actor, etc.

[krageon]

> No bad actor will ever give you heads-up.

Yes, and if you do it without a heads-up as well that makes you a bad actor. This university is a disgrace and that's what the problem is and should remain.

[simiones]

C'est la vie. There are many things that it would be interesting to know, but the ethics of it wouldn't play out. It would be interesting to see how well Greg Kroah-Hartman resists under torture, but that does not mean it is acceptable to torture him to see if he would commit malicious patches that way.

To take a more realistic example, we could quickly learn a lot more than today about language acquisition if we could separate a few children from any human contact to study how they learn from controlled stimuli. Still, we don't do this research and look for much more complicated and lossy, but more humane, methods to study the same.

[corty]

They proved nothing that wasn't already obvious. A malicious actor can get in vulnerabilities the same way a careless programmer can. Quick, call the press!

And as for the solutions, their contribution is nil. No suggestions that haven't been suggested, tried and done or rejected a thousand times over.

[dumpsterdiver]

Agreed. So many security vulnerabilities have been created not by malicious actors, but by people who just weren't up to task. Buggy software and exhausted maintainers is nothing new.

[dcow]

What this proves to me is that perhaps lightweight contributions to the kernel should be done in safe languages that prevent memory leaks and with tooling that actively highlights memory safety issues like use after free. Broader rust adoption in the kernel cant come soon enough.

I also consider Greg’s response just as much a test of UMN’s internal processes as the researcher’s attempt at testing kernel development processes. Hopefully there will be lessons learned on both sides and this benign incident makes the world better. Nobody was hurt here.

[dumpsterdiver]

I understand where you are coming from, and I agree that it's good that we are paying more attention to memory safety, but how would a memory safe language protect you from an intentionally malicious code commit? In order to enact their agenda they would need to have found a vulnerability in your logic (which isn't hard to do, usually). Memory safety does not prevent logic errors.

> Nobody was hurt here.

This is where you got me, because while it's clear to me that short-term damage has been done, in the long term I believe you are correct. I believe this event has made the world a safer place.

[chihuahua]

One could argue that when a safe language eliminates memory safety bugs (intentional or unintentional), it makes it easier for the reviewer to check for logic errors. Because you don't have to worry about memory safety, you can focus completely on logic errors.

[mk89]

This is for me unrelated and even a little bit minimizing the issue here.

The purpose of the research was probably to show how easy it is to manipulate the Linux kernel in bad faith. And they did it. What are they gonna do about it besides banning the university?

[dumpsterdiver]

I believe it comes down to having more eyes on the code.

If a corporation relies upon open sourced code that has historically been written by unpaid developers, if I was that corportion, I would start paying people to vet that code.

[mk89]

So you are just fine knowing that any random guy can sneak any code in the Linux kernel? Honestly, I was personally expecting a higher level of review and attention to such things, considering how used the product is. I don't want to look like the guy that doesn't appreciate what the OSS and FSF communities do everyday even unpaid. However this is unrelated. And probably this is what the researchers tried to prove (with unethical and wrong behavior).

[corty]

I'm not fine with it. But those researchers are not helping at all.

And also, if I had to pick between a somewhat inclusive mode of work where some rando can get code included at the slightly increased risk of including malicious code, and a tightly knit cabal of developers mistrusting all outsiders per default: I would pick the more open community.

If you want more paranoia, go with OpenBSD. But even there some rando can get code submitted at times.

[simiones]

If you've ever done code review on a complex product, it should be quite obvious that the options are either to accept that sometimes bugs will make it in, or to commit once per week or so (not per person, one commit per week to the Linux kernel overall), once every possible test has been run on that commit.

[mk89]

I am not sure if these are the only options we have here. Did you see the list of commits that this bunch of guys sneaked in? It's quite big, it's not just 1-2. A smart attacker could have done 1 commit per month and would have been totally fine. All they needed apparently was a "good" domain name in their email. This is what I think is the root of the problem.

[kaba0]

> So you are just fine knowing that any random guy can sneak any code in the Linux kernel?

I mean, it is no surprise. It is even worse with proprietary software, because you are much less likely to be aware of your own college/employee.

Hell, seeing that the actual impact is overblown in the paper, I think it is a really great percentage caught to be honest, assuming good faith from the contributor.

[rualca]

> However, they proved a big point: how "easy" it is to manipulate the most used piece of software on the planet.

What? Are you actually trying to argue that "researchers" proved that code reviews don't have a 100% success rate in picking up bugs and errors?

Specially when code is pushed in bad faith?

I mean, think about that for a minute. There are official competitive events to sneak malicious code that are already decades old and going strong[1]. Sneaking vulnerabilities through code reviews is a competitive sport. Are we supposed to feign surprise now?

[1] https://en.wikipedia.org/wiki/Underhanded_C_Contest

[mk89]

Bug bounties are a different beast. Here we are talking about a bunch of guys who deliberately put stuff into your next kernel release because they come from an important university, or whatever other reason. One of the reviewers in the thread admitted that they need to pay more attention to code reviews. That sounds to me like a good first step towards solving this issue. Is that enough, though? It's an unsolvable problem, but is the current solution enough?

[rualca]

> Bug bounties are a different beast.

Bug bounties are more than a different beast: they are a strawman.

Sneaking vulnerabilities through a code review is even a competitive sport, and it has zero to do with bug bounties.

[mk89]

Sorry I think I didn't understand/read correctly what it was about.

It's just f** brilliant! :)

[jnxx]

What would be the security implications of these things:

* a black hat writes malware that proves to be capable of taking out a nation's electrical grid. We know that such malware is feasible.

* a group of teenagers is observed to drop heavy stones from a bridge onto a motorway.

* another teenager pointing a relatively powerful laser at the cockpit of a passenger jet which is about to land at night.

* an organic chemist is demonstrating that you can poison 100,000 people by throwing certain chemicals into a drinking water reservoir.

* a secret service subverting software of a big industrial automation company in order to destroy uranium enrichment plants in another country.

* somebody hacking a car's control software in order to kill its driver

What are the security implications of this? That more money should be spent on security? That we should stop to drive on motorways? That we should spent more money on war gear? Are you aware how vulnerable all modern infrastructure is?

And would demonstrating that any of these can practically be done be worth an academic paper? Aren't several of these really a kind of military research?

The Linux kernel community does spend a lot of effort on security and correctness of the kernel. They have a policy of maximum transparency which is good, and known to enhance security. But their project is neither a lab in order to experiment with humans, nor a computer war game. I guess if companies want to have even more security, for running things like nuclear power plants or trains on Linux, they should pay for the (legally required) audits by experts.

[WmyEE0UsWAwC2i]

I agree with the sentiment. For a project of this magnitude maybe it comes to develop some kind of static analysis along with refactoring the code to make the former possible.

As per the attack surface described in the paper (section IV). Because (III, the acceptance process) is a manpower issue.

[spullara]

Ironically, one of their attempts were submitting changes that were allegedly recommended by a static analysis tool.

[rjmunro]

It's possible that they are developing a static analysis tool that is designed to find places where vulnerabilities can be inserted without looking suspicious. That's kind of scary.

Have they submitted patches to any projects other than the kernel?

[spullara]

Guess we have to wait for their next paper to find out.