Hacker News new | ask | show | jobs
by gsnedders 3464 days ago
Who's going to make sure LineageOS users get security updates in a timely manner? Is anyone going to be paid to work on it?

Any large OSS distribution is going to have a fairly continuous stream of security fixes to ship to their users, and that takes a fair amount of time, and I'm always concerned about whether any new project (okay—it's not quite new, but they have a fraction of the number of developers they did twelve months ago!) has the resources to ship them in a timely way.

5 comments

Not directly an answer, but one of the big issues with security patches for custom ROMs is the amount of patches they don't (read can't) ship. The proprietary blobs are very often not patched when the device is vendor-supported, and once it reaches end of life from the vendor (but the community ROMs give devices significantly extended longevity), there's no more patches to these blobs.

Blobs incorporate the modem, baseband firmware, bootloaders, and many (most?) of the hardware drivers and imaging drivers.

51% of Android kernel vulnerabilities in vendor drivers are a result of missing or incorrect bounds checks, and over the whole Android kernel, 44% of all vulnerabilities were missing bounds checks, and 12% for null pointer dereference.

Looking across the whole kernel, from Jan 2014 to April 2016, 85% of kernel bugs are born in vendor drivers, with the remainder in the core kernel.

Vendors therefore are shown to write bad code. It's fairly safe to assume this is reflective of the quality of their blobs too - there's certainly a load of vulnerabilities in those if you look at the Android Security bulletins for bugs without a source reference for the fix.

So agreement with your concern, but I'd just like to highlight that custom ROMs are not really a good security solution, as there's just so much to fix (at a kernel level, requiring detailed driver knowledge of the vendor/SoC stuff), and blobs that won't get updated after the vendor abandons the phone.

Ref: https://events.linuxfoundation.org/sites/events/files/slides...

Oh, that's totally true—though for many having some secure updates would be better than none, especially given the number of kernel bugs that are only locally exploitable (i.e., if the rest of the system is up-to-date, locally exploitable issues are less, but not none, of a concern).

One thing I wish there was more visibility of is "is this device still getting security updates", because there's often almost no visibility about that (and while you obviously can't say the vendor won't fix future vulnerabilities, you can say whether the vendor has fixed all known vulnerabilities), even online, yet alone anything pushed to the device to let its user know it is no longer secure.

e.g., http://web.archive.org/web/20161224231459/https://wiki.cyano... is the old CyanogenMod wiki page for the Galaxy S2: the last "development channel" (i.e., unstable) build is 2016-12-18, the last "release channel" (i.e., stable) build is 2015-11-16. There is nothing on the device page to suggest the stable build is known to be insecure (though given the number of Android bugs found in the last year unquestionably is!), yet alone anything about upstream vendors dropping support for the device and the unstable build being known to be insecure too. How is LineageOS going to do better than that? That's a damn low bar.

Absolutely - userland is the easiest to exploit, as it's fairly common across all devices (thanks to CTS and standardisation of the runtime) - that's why stagefright was such a big deal!

Definitely agreed - I've thought about making such a list to give visibility of this before, but it would be more of a user-submitted list (perhaps with link-up to screen scraping of OEM web pages for the ones that list the latest version).

What held me back was the sheer complexity of working out whether a device still gets updates - take Samsung as an example; the user says "I have a Galaxy S6". Depending on their geographical location this might be a carrier-free G920I or G920F. If they are in the US, it could then be one of about 5 or 6 variants, and there's even a G920W8 for Canada.

User wants to know if "Galaxy S6" is safe and secure, but even different regional firmwares of the same SKU might not be getting pushed security updates. And some US carriers (Verizon, ATT) are notorious for not pushing out updates to users. And then finally when you figure out the version on a given phone, you need to try to decide if the fact the device is still on October 2016 security patch means it's unsupported or not.

Often Samsung are lagging 2 to 3 months behind on some SKUs, making it even harder to tell. The same is true for many other OEMs - Sony have a pretty complex system of ROMs for each region, meaning you have carrier and non-carrier ones, and they can be on different versions.

To make this happen, we'd ideally need a single worldwide firmware without carrier changes/tweaks/influence. Until then, I suspect it would be too complex to help users work out if their device was being supported.

Starting from Android 6, it's easy to check if the device is up-to-date from an Android point of view: in "About phone", you have an "Android security patch level" section which should be match the current month.
Yes indeed - the downside of this is that it's hard to gather this information together in a way that lets you show people "which devices" are still maintained.

It is much better for users to know if they are on the latest build. Sadly though for (most/many?) devices, the answer is "it's not", and there's nothing they can really do about it, either due to OEM latency in releasing updates, or the OEM having abandoned their phone.

Samsung [1] and LG [2] pretty much say on their own websites that only certain phones will get updates promptly (or at all) - consider their full product ranges and the cheaper devices not even listed!

[1] http://security.samsungmobile.com/introsm.html

[2] https://lgsecurity.lge.com/security_updates.html > Depending on regions and carriers, updates may be released monthly, quarterly or irregularly.

I presume the OS can at least tell what exact model it's on, which means we could potentially at least have something (an app, given I guess Google will never ship such a thing) that says, "hey, this device isn't secure any more".

Now, obviously there's the problem with the time taken to ship fixes (do you say the device is insecure for the two-to-three months before a patch is shipped? do you say the device is insecure only six months after the exploit becomes public? etc.), so even this isn't that simple.

I still wonder about how well the "Android security patch level" will cope with OEMs and their often slow kernel updates (i.e., the fact there are OEMs that quickly release userland fixes, and very slowly release kernel updates).

The only solution seems to be reverse engineering the blobs and mainlining Linux kernel drivers, without both of those security updates get much much harder to impossible.

I've no idea how to achieve that on volunteer time, maybe a crowdfunded reverse engineering and mainlining org could work?

One problem is that RE is time consuming (regardless of whether or not someone is paid to do it), and the useful life of phones tend to be much shorter than other kinds of devices, so digging apart a blob on one phone is likely to have a limited useful lifetime.

And for phones that the manufacturer actively supports, often a new version (especially if it's a new Android version) means new blobs to RE.

When you consider a lot of phones lose a ton of their user base after 2 or 3 years, it becomes much less attractive to even bother.

The alternative; devices with no updates and no support outside their original OS, doesn't seem very attractive either.

Maybe we can create incentives for manufacturers to do this work themselves, but I doubt that will ever happen, unless maybe we start getting obnoxious viruses like there were on the PC at one point?

Sure, that's not a particularly attractive outcome, either.

I just think it's unrealistic to think paid RE work is going to fill this need.

I think there are two realistic options: 1) the manufacturers suck it up and agree to support devices with timely updates over a longer lifespan, or 2) manufacturers open-source every bit of software that runs on the device.

#2 seems less likely, given that a lot of hardware is driven in part by loadable firmware these days. On the other hand, if that firmware is chipset-specific and not device-specific, and the chipset manufacturer can commit to releasing security updates for those, at least 3rd-party OS images could pull them in without help from the device manufacturer.

But really, it's all about demand: Apple tends to support hardware with new releases for 4-ish years as a matter of course, and i-device users are accustomed to expecting that. Android users just don't expect that, and your average user doesn't understand security enough to get why that's such a big problem. They likely mostly just think, "oh well, I won't get the new shiny Android version Jane has on her new phone, that's ok". If average users can be educated to the point where they will switch manufacturers if they're not getting security updates for the useful life of their phone, the manufacturers will listen to their declining sales. I just don't expect that to happen.

I wouldn't be surprised if it were easier to backport kernel security fixes to the version the blob depended on than reverse engineer the blob, given the relatively short life of devices in general.

That said, neither of those two options is easy!

Yet somehow most stock ROMs are worse at maintaining security patches.
This.

It's actually astounding, albeit hardly surprising, that companies often have zero interest in pushing security patches to devices not being manufactured anymore.

If/when it becomes a risk they must mitigate against (be that a financial or reputational risk), I guess they will.

Heck, until stagefright, Google didn't even release security bulletins. It was nigh-on impossible to keep track of all the vulnerabilities They only released security patches in new version releases. That wasn't good for vendors.

Now Google has pushed forward, and it's the turn of OEMs. They shipped patches to StageFright due to the massive bad PR (headline news in many countries, was a talking point amongst even the vaguely tech savvy).

Unless regulated or they feel they will lose money by not doing so, I don't imagine anything changing soon unfortunately. Qcom and other SoC makers are part of the problem too, since they try to drive chipset sales by only supporting older chipsets for a short time.

This is why for the first time since I started owning smartphones (2010), I am going to switch to iPhone.
That's not going to fix the problem. Installing updates on an iPhone older than a couple years bogs performance down to unusable levels.
I have an iPhone 5 (4 year old phone) running the latest OS and it works fine. No, it's not as fast as the latest devices, but it works adequately if one desires to keep using their 4-year-old device.
While I agree that's been the case in the past, supported devices are fast enough now that it really doesn't slow them down anymore. Even the lowest supported device runs well on iOS 10, and it has gotten official updates years longer than all android devices I know of.

Sure, one could install an updated and more secure rom (assuming it exists for your device). But, the vast majority of users don't care or won't bother to go through that process, rendering it a completely ineffective solution for the general consumer market.

> supported devices are fast enough now that it really doesn't slow them down anymore

That happens to be true now because Apple has dropped support for many iPad models with iOS 10, accounting for up to a third of all iPads in use.

Running iOS 9 on the iPad 2, 3, and iPad mini was frustrating. This was only three months ago, so I wouldn't rule it out for the future yet.

I am aware but at least I won't get hacked which is more important than performance.
But even if Apple may be better than most android manufacturers, Apple still doesn't support devices it considers "Obsolete"...and there is nothing you can do about it other than buy a new phone since the bootloader is locked down. According to https://en.wikipedia.org/wiki/List_of_iOS_devices anything before iPhone 5 is considered "Oboslete" and they don't seem to be releasing updates. On the other hand, I've been happily using my Samsung S3 which cyanogenmod has been pushing latest android updates to nightly...and that is likely to continue as long as there are enough of us who care, even if cyanogen and samsung goes away.
Oh, I'm not denying most OEMs are terrible at this—this is the massive achilles heal of Android.

OTOH, in a sense I have a higher expectation of "aftermarket" OSes insofar as they're actually pushing semi-regular updates which I'd hope would include all needed security updates.

Something I've wondered about: why does Android require you flash a ROM to update the system?

When you buy a windows laptop you'll usually get a bunch of vendor-specific crap, but after doing a clean install you can just pick up the drivers on their website and it'll all work fine. Why isn't this an option with Android?

ARM's SoC lack of standards.

In reality, we were really spoiled with IBM's massive mistake in standardizing the PC (the BIOS), so while Linux may not pick up your monitor, your monitor is made to work to a basic standard (VGA), so you can at least boot up, read the hard-drive, and have basic "keyboard" IO.

This was really important before Win95 made the PC practically MS owned, as you could (and were expected to) install whatever OS you wanted, so they had to have a standard Boot Sector, etc.

ARM SoC are meant to be built by a manufacturer to their specs (their CPU doesn't have to be compatible with anything because nothing is user-serviceable anyways). In other words, PCs were meant to be an Open Standard, while SoC are like old consoles/"dumb" phones.

Unfortunately, due to MS dominance in the PC world, and the collapse of independent computer manufacturers (I remember the days when buying a cheap computer meant not going to Dell but going to the mom-and-pop shop down the block, but you had to know how to fix things if anything went bust), the PC is moving slowly but surely to the appliance world.

As I have been involved with OpenWRT/LEDE and how they make builds to different devices recently, I realized that making unified builds are actually possible. OpenWRT has the same kernel and root image for many different devices that use the same SoC. So once you have OpenWRT flashed on, the content of every OpenWRT update (for every device of the same SOC) is by and large the same. The only check in place is the artificial model check to make sure you don't unintentionally flash the wrong image.

Android can technically do like OpenWRT does, but there is much more stuff involved in the Android world so making a universal build is harder. However, I personally can't think of a hard, technical barrier. New ideas for unifying devices and separating blobs such as device tree or separate vendor partition is getting traction and makers such as Sony are embracing it. Many Sony devices boot the same kernel, and that's why you consistently seeing Sony devices getting support and builds very early on from projects like CM.

BTW, I have been trying to write an answer to you by saying a slightly different version of what dispose54312 said, but then realized it didn't make sense. I came to the conclusion (for myself) this is mostly "how it is done currently for Android." So I hope my unpopular answer is not crazily misguided, and please take my perspective with a grain of salt.

Thanks for your work on OpenWRT...I'm a happy user! I would hope for any expertise on unified builds could be transferred over to android. I'm wondering if android could run on top of OpenWRT, then could use the same system to update the android layer without having to update the lower levels.

Would think one of the whole points of having the many layers in the android software stack is that one layer can be modified without having to reload everything in the lower levels.

> Something I've wondered about: why does Android require you flash a ROM to update the system?

Let me answer that by picking the question apart:

1. To update the system, you need access to write actual system-files. You need system-level access.

2. Android is built entirely up on a "user-space" model where permissions are granted to apps, and almost no apps have system-level access.

3. On rooted phones you can allow system-level access to specific apps, which in theory could install the latest OS version and binaries. In theory that sounds like 1 app which you can use to update the OS, everywhere, right?

4. Except you would need a different installation-strategy for unrooted phones anyway for the installation anyway.

5. To compound the problem the ARM platform has no standardized BIOS/UEFI layer to handle generic booting, so the kernel being booted must be device-specific in order to gain access to the rest of the phone: It can't rely on generic "BIOS"-drivers to load modules on demand from flash storage. And the boot-partition is almost always too small to have a kernel with enough drivers for "everyone" embedded.

6. So you cant make a standard phone-agnostic flashing-app, nor install medium. You do have to build a custom phone-specific initial flash-package for every model you want to support as a bare minimum.

And when you've already put in all that effort, what do you gain by creating an "app" which does this inside a pretty GUI, instead of just rebooting back to recovery to (auto) flash the latest image? Practically nothing.

"Flashing" may sound scary, but in reality it just means installing. There's no technical difference between that and just copying files to a usually write-protected partition.

AFAIK: Apple officially does exactly the same, they just don't call it by that name.

There is a "fastboot" [1] tool from Google that interacts with phone bootloaders that support it and can be used to flash the ROM.

> There's no technical difference between that and just copying files to a usually write-protected partition.

"Flashing" usually means "uploading" a filesystem image onto a partition, not copying single files.

[1] https://source.android.com/source/running.html#booting-into-...

Fastboot is a interface/protocol to talk to the bootloader, in order to flash images to select partitions. On select phones. And it requires an external device to run the process. So it can't be self-hosted.

All that technical whizbang doesn't change or even address the underlying problem: What you boot still needs to be 100% device specific.

There's no UEFI/BIOS on ARM to help bootstrap the boot process.

Also: When you flash custom roms on Android, you format and then unzip a file onto the system partition. Not that I see how that's technically any different from flashing a FS-image, but if it matters to you, now you know.

In conclusion: none of my claims have been near disproven, but a few straw men has been presented to be stricken down ;)

Probably the same people who made sure CyanogenOS users got timely updates: no one.

Last update for my phone was early september. Not holding my breath for this.

I want to read the full story, including how Microsoft was involved with Cyanogen(Inc/Mod).

(yes, it was involved, use Google to search for previous news.

For example at one point CyanogenInc partnered with MSFT so they have a Microsoft sponsored Android flavour with several Microsoft apps preinstalled. https://cyngn.com/press/cyanogen-announces-strategic-partner... http://www.businessinsider.de/why-microsoft-cuddled-up-to-cy... Now it smells like the "embrace, extend and extinguish" strategy that MSFT is using with great success for decades https://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish

So I hope at one point we will get the full story.)