Wow, I normally am all about how google handles security issues (they get dragged through the mud for some Project Zero stuff), but this def did not get handled well.
Super unclear communication, starting with "you're just using it wrong", more than six month turn around, and even then at the end no clear explanation of what went wrong with someone who was collaborating with you? That's amateur hour security.
I'd pay to see Bryan Cantrill's reaction [0][1] to this: A seemingly mysterious firmware bug of a secure element / trusted-execution-environment but there's no knowing if there are more bugs (or, shudder backdoors).
Since the source code isn't available for scrutiny (though Google has promised firmware transparency [2]), it is kind of difficult to tell what really went wrong in the current reported case and what else possibly could go wrong given the use-cases for it are far-reaching and sensitive: Google has advocated StrongBox as a trustable companion that could be used to attest user actions on medical devices [3], for instance; or for use as an Identity verificafion for documents such as Driving Licenses and Passports.
My reaction might not be worth the price of admission, because it's exactly as you'd expect: this is great, diligent, responsible work -- and if the Titan M firmware had been open, it seems likely that the author would have been able to get to root cause (or much closer to it). This in turn would have in turned tightened Google's response time, led to a better fix faster, etc.
That said, there are several reasons for optimism.
1. OpenTitan.[1] On the one hand, this is not about opening up extant Titan implementations so much as developing next-gen Titan in the open -- but it is nonetheless a laudable and important effort and it is increasingly reasonable to expect that the hardware roots-of-trust of the future will be entirely open.
2. Open firmware more generally. The Open Source Firmware Conference[2] this past fall was truly inspiring in terms of the broad interest from the industry: while there is much work to be done, there is more reason than ever to believe that it's attainable.
3. Rust. It's hard to speculate without knowing what the root cause of this issue actually was, but to the degree that memory corruption was at root here, the emergence of Rust for firmware is an incredibly important development. Speaking personally, if there was any doubt in my own mind about the appropriateness of Rust at this lowest level of software, it has been erased by our own experiences at Oxide over the past few months: Rust is unequivocally the right language for firmware, and it will yield higher quality artifacts.[3]
More troubling to me than the closed source firmware is that the bug in TFA seems like something that the most basic of a test suite should be catching.
It’s reminiscent of Apple’s “goto fail” lack of certificate checking - another easily testable case that simply wasn’t.
The test authors don’t even need to be on the same team/manager. They can just write black box tests to the spec, like the author of this post did.
I’m not even some big TDD guy. It just seems to me that in these core security-critical libraries/functions that should be pretty side-effect-free that you should have some basic “receive x, produce y” functional tests to make sure the API is doing what it claims to do on the tin.
the most basic of a test suite should be catching.
A most basic test suite is not likely to wait some arbitrary amount of time (2 seconds, as the author found by trial and error) between calls to the HSM.
The images in the 'Digging deeper' section suggest otherwise. They appear to show a successful run followed by one that fails because the 'encrypted' value is garbage. Where am I missing the instantly reproducible failure?
In what way? 'Temporal' fuzzing to an eon-like range of two seconds seems, naively at least, entirely impractical.
Edit: a somewhat different way of putting this concretely - what is a practical stochastic testing regime that can reasonably be expected to find this bug?
I think Google's handling of security issues in Android has been badly broken for quite a while now. Here's a timeline from another security bug in the same (security critical!) subsystem:
- 2014-02-24 AM - Discovery.
- 2014-02-24 PM - Vendor notification.
- 2014-02-24 PM - Vendor acknowledgement and confirmation.
- 2014-02-26 - Attempt to setup coordinated disclosure (no response).
- 2014-04-07 - Public disclosure.
- 2014-10-17 - Response from Android security team offering line in Android security - acknowledgements.
- 2014-11-03 - Verified fixed in Android Lollipop.
(The impact of this bug was keystore material leakage between apps. No CVE was assigned!)
That was a long time ago, and of course two anecdotes aren't data. But it makes it all the more interesting to read Project Zero's frustration with poor disclosure practices by others.
"Comments on a number of StackOverflow questions have pointed out that a fault address of deadd00d indicates a deliberate VM abort."
That ending up in the ciphertext multiple times seems to point to some memory corruption issue. It's also a good argument for using magic numbers that stand out.
In what way did they violate their own rules? Google didn't prevent the researcher from disclosing and the researcher could have disclosed - the timeline describes requests, not demands. For reference, Project Zero's disclosure FAQ:
There are several cases in which deadlines were extended way beyond 90 days. And in the post itself, the researcher points out they could (and, in hindsight, feel they should) have imposed a hard 90 day deadline.
Yep I fully agree, you can't hold others to a standard you're not willing to hold yourself to. I specifically remember Microsoft begging for more time on a bug.
Also if I understood it correctly, it seems as though some devices may require a factory reset to apply the new firmware? If so, for a lot of devices this still isn't fixed.
Author here. I should have made this clear in the blog post, but I'd be interested in seeing boot logs from Pixel 3 (or newer) devices. If the firmware update failed on more devices than just mine, it would be good to know about that. If you'd like to help, first make sure you're at least on the December 2019 security update. To capture the log plug your phone into a computer, run "adb reboot ; sleep 1 ; adb logcat | grep -i citadel" and turn the phone back on. Wait for it to boot, unlock the SIM card and unlock the screen. This should yield the version information of the firmware of Titan M.
The most important takeaway is to stick to the 90-day disclosure policy. The deadline is only credible when it's enforced, and on a number of occasions Google have stated they believe so themselves. [1]
The fact they still haven't gone around to open-sourcing the Titan M firmware is, honestly, what worries me the most about this. It's probably going through some heavy internal auditing now, which is not the best approach, all things considered.
If I were an Android user: I'd feel good with how Google managed this one.
Regarding NSA vs Google (seems like after I commented on a corona virus thread a few times, I was rate limited - editing existing comments still works though):
@baybal2:
"NSA infiltrates links to Yahoo, Google data centers worldwide"
PRISM doesn't give the government additional access to anything. It simply ingests data that the FBI has already collected by wiretapping individuals under investigation.
Edit (responding too fast):
> It by default gives government access without anyone at Google or anywhere else granting that access at time of use.
Where did you get that idea? All the documents show that it ingests data the FBI already has, for individuals the companies already manually approved after potentially fighting about it with a judge. You simply made up an illegal system out of whole cloth that wouldn't last a minute in court if anybody challenged it, unlike the phone metadata program, which went through two courts to conclude its illegality.
Edit 2:
> Page five lists the companies and page six lists the per company agreement date. Unless you're trying to argue that Google didn't respond to wiretapping requests from the FBI at all before 2009.
The FBI has to set up a system for canonicalizing and routing data from each different company. Those dates list when the FBI did that for each company. Since almost nobody (including suspected terrorists, apparently) uses Apple's email service, their system was the lowest priority to support.
This is well documented, both in Snowden's documents and in documents the government later declassified. Once again, if PRISM were as you described it, it would be flagrantly illegal and shut down long before the phone metadata program.
Edit 3:
iMessage was launched near the end of 2011, and FBI's DITU handles content collection via wiretaps. When are you going to address the fact that the program from your fever dreams is insanely illegal and that it doesn't match any of the documents? If you would like me to respond normally, upvote my comments, so I don't get rate-limited.
It's automated systems for those requests. It by default gives government access without anyone at Google or anywhere else granting that access at time of use.
It does record the request though which is why NSA tried to exceed the bounds of that with MUSCULAR.
Edit to respond to your edit: Page five lists the companies and page six lists the per company agreement date. Unless you're trying to argue that Google didn't respond to wiretapping requests from the FBI at all before 2009.
Edit 2 since apparently this is how we're doing this:
> Since almost nobody (including suspected terrorists, apparently) uses Apple's email service, their system was the lowest priority to support.
There's a fuck ton of metadata that iMessage reports back up; PRISM isn't just about email. And yes, iPhones are the most common smartphone in the world. I guarantee you that Apple isn't last because they were a low priority, that's absolutely absurd.
Edit 3:
Your argument that "it would be illegal and shutdown like the other illegal programs documented here if it were actually illegal" has to be one of the hottest takes I've heard.
And the PRISM collection was part of what the Supreme Court dismissed not because it isn't illegal, but because you can't prove that affected the claimant personally without a breach to national security, so they can't prove they have standing, so the case had to be dismissed. https://www.aclu.org/files/assets/amnesty_v_clapper_scotus_o...
The NSA's slides disagree with you. Here is the slide explaining how the data flows. https://imgur.com/setOJIm
PRISM is simply a data integration system that gets data from the FBI's Data Intercept Technology Unit, which is the group that handles Internet communication wiretaps on specific individuals under investigation.
@alexbakker: could you please make code in the snippets wrap at least when viewed on a phone? I gain nothing from them being non-wrapping, while having to scroll back and forth to read. In each editor I've used, code soft-wraps at screen edge, so I don't even understand where this trend came from.
PRISM involved automated systems created by the various companies to comply with FISA requests that could originate from the NSA but would be served by the FBI.
No. The FBI issues Section 702 data requests for individual users to the companies, whose lawyers manually review the requests and may dispute them before a FISC judge. Only after the company approves the request do they start sending data to the FBI's DITU. PRISM consumes the data from DITU's servers.
The system that sends new communications with the monitored individual to the FBI is definitely automated, but configuring an account to be surveilled is a manual process controlled by the company, not the FBI, and certainly not the NSA. The reason you cannot provide documents that say otherwise is that they don't exist. The reason those documents don't exist is that the program that you've described is a conspiracy theory fiction.
Yes, Qwest (the telecom, formerly US West) resisted and fought tooth and nail to not cooperate.
Their CEO was prosecuted to hell and back for daring to do this, and the company was forced to sell to a competitor. Nobody even remembers his name anymore, few people even remember Qwest.
His name is Joseph Nacchio. If this story is true (and it has been around for many years), he is a hero for standing up for transparent governance, and the privacy of Qwest's customers.
I don't know anything about it, but I think there may still be related litigation going on.
Edit: maybe not. Wikipedia says he was convicted in 2007.
However, I was given an NDA regarding something that sounded related at some point after that, so...
You skipped the part where he acquired a competitor under false accounting, contributing to monopolization, as dumping his stock with irregular sales while to profit from value NSA contracts before the public knew they were canceled. Should a CEO make a fortune selling stock while the company loses 90% of its market cap?
Yeah it's funny how he was the only corrupt CEO in all of corporate America during that time. So weird how nobody else besides him and Martha Stewart ever got prosecuted for insider trading despite it happening ALL THE TIME EVERY DAY EVERYWHERE.
> Former Qwest CEO Joseph Nacchio, alleged in appeal documents that the NSA requested that Qwest participate in its wiretapping program more than six months before September 11, 2001. Nacchio recalls the meeting as occurring on February 27, 2001. Nacchio further claims that the NSA cancelled a lucrative contract with Qwest as a result of Qwest's refusal to participate in the wiretapping program. Nacchio surrendered April 14, 2009 to a federal prison camp in Schuylkill, Pennsylvania to begin serving a six-year sentence for an insider trading conviction. The United States Supreme Court denied bail pending appeal the same day.
He was happy to get paid hundreds of millions of dollars for spying on his customers, but didn't want to spy on his customers? What did he think those contracts were for?
I don't know what the lesson is after reading his wikipedia page. His refusal to cooperate with the NSA seems like a non-sequitur as a response to the whole insider trading and fraud thing. I mean, if the prosecution was revenge, well, ok, but I can't connect the dots to how that makes him not guilty.
Super unclear communication, starting with "you're just using it wrong", more than six month turn around, and even then at the end no clear explanation of what went wrong with someone who was collaborating with you? That's amateur hour security.