[Andrew_nenakhov]

Remember, people: the real lock-in with Android is not the Google Play. It is the push notification services that are bundled with it.

The Android OS is intentionally crippled to work only with Google Play push notifications, and there are no 'stock' ways to use an alternative without modifying the OS. The best alternative is to do it via persistently running app, which is less reliable and bad for UX.

This notifications problem also applies to iOS: even if users will be eventually allowed to install apps bypassing the AppStore, these apps would just sit idly until opened by a user, which is unacceptable for a very wide range of communication apps.

However, there is a difference with Android: iOS was crippled from the start, but first iterations of Android were relatively open, apps could run in the background without much issues. But then Google started tightening the grip, so Android now closely resembles iOS in this regard.

[dtech]

> first iterations of Android were relatively open, apps could run in the background without much issues. But then Google started tightening the grip

The flip side of this is that there were a lot a badly behaving apps, sitting in the background without a user requirement that would justify that, eating battery and network. Hence nearly all Android phone manufacturers bundling some kind of background app killer.

[NotPractical]

Restricting background apps by default is probably good. But Google didn't have to lock users in to their own service to provide a good push notification system on Android.

The correct solution here would have been a system-level, open source, provider-agnostic push notification API built into AOSP. The Web Push API is a great example of how a push API can be provider-agnostic: https://developer.mozilla.org/en-US/docs/Web/API/Push_API. It makes sense that push notifications should be consolidated to one provider -- this way, your device only needs to maintain one 24/7 TCP connection to a server which it receives all notifications from all apps through, rather than having each app run its own notification service in the background, killing your data and battery life.

Google's push service works by providing an HTTP endpoint to apps that they can use on the backend to deliver notifications. The Web Push API also works this way, but the returned endpoint can be for any provider. If you use Chrome, the domain will be `google.com`, but on Firefox, it's `mozilla.org`. But each provider's endpoint uses the same standardized API, so the backend doesn't have to care what the URL is. And this isn't a security risk, because everything sent through the service is encrypted.

This could have been done for Android, but that would have given Google much less control over the platform, so they decided to do it in a monopolistic way. This is one of the many ways Google aims to maintain their monopolistic control over "open source" Android. Another example is SafetyNet hardware-backed attestation.

Projects like GrapheneOS are really interesting because they are finally providing a real secure, private, de-Googled OS option, with excellent app compatibility thanks to sandboxed Play Services (which allows you to run Play Services without giving it root access to your device [1]). But will we ever be able to fully decouple Google from Android? I'm not sure. I expect most users of custom ROMs to continue installing Play Services on top of them for a long time, if they want something as basic as push notifications to work.

By the way, while I personally have qualms with Google, I believe users should have the choice to use Google apps, if they are comfortable with the inherit privacy risks. This belief is shared by the GrapheneOS developers. [2] My problem with Android is that it was not built to function without Play Services. Community projects that enable it to do so, like microG, will always be a cat-and-mouse game with Google. I think sandboxed Play Services is the most sane approach the problem of Play dependencies the community has come up with so far, but I think if we want real change to happen, we need to target app developers. Each developer has the choice of whether to include Google or not within their app. We just need to convince them that they don't need Google.

Google owns `developer.android.com` and points developers towards using Play Services APIs wherever possible. Perhaps the community should create their own open-source alternatives to the most commonly used APIs (e.g. push notifications) and developer documentation that provides instructions on how to switch from Google (ideally making the transition as easy as possible by emulating the Google API syntax). It would probably be possible to offer an installable platform-agnostic push notification API that developers can use. Providers could be installed as separate apps. Google could be one such provider. Perhaps we could implement a Web Push compatible API, and reverse engineer Chrome's Firebase integration to implement it as an option from the get-go, hoping other providers show up over time. We could perhaps allow users to self-host their push service as well. Modify Gotify, an open source self-hosted push server [3], to accept push notifications in the Web Push format.

In the hacker circles it seems there's two groups of people: those who think Android is a lost cause because it will always be controlled by Google, and think Linux phones are the only real alternative, and those who believe we can actually "steal" Android back from Google and make it into a true open source project. I fall into the latter, but I think a lot more work still needs to be done if we want to achieve this.

[1] https://grapheneos.org/usage#sandboxed-play-services

[2] https://grapheneos.org/features#features

> We aren't against users using Google services but it doesn't belong integrated into the OS in an invasive way. GrapheneOS won't take the shortcut of simply bundling a very incomplete and poorly secured third party reimplementation of Google services into the OS. That wouldn't ever be something users could rely upon. It will also always be chasing a moving target while offering poorer security than the real thing if the focus is on simply getting things working without great care for doing it robustly and securely.

[3] https://gotify.net

[AshamedCaptain]

There is no need for a global push service, either. Just some type of API that batches timer wakeups and network keepalives together is enough to guarantee very high battery life, and you can still keep as many connections as the user wants. Many platforms before Google/Apple worked like this.

Push services are a bit more convenient in that they require less infrastructure both in the server-side as well as client programs (e.g. they can even be shut down and still receive messages), but nothing really prevents having an ecosystem of multiple competing push services.

[NotPractical]

There clearly is a need. It needs to be as easy as possible to switch away from Google, or developers won't do it. Additionally, an ecosystem where each developer is encouraged to go wild and implement their own notification system is not ideal. Sure, some of them will be competent enough not to deplete the user's battery within a few hours, but others will not be, and each individual app that does this contributes to battery drain in a tiny way, which adds up.

> Many platforms before Apple/Google worked like this

Mobile devices are an entirely different beast with strict battery life and data usage requirements.

Another factor to consider is that Android will kill long-running background apps unless you disable battery optimization for said apps through a convoluted process (even if you show a persistent notification). [1] It would be really inconvenient for users to have to manually disable battery optimization for every app that wants to send push notifications.

[1] https://github.com/gotify/android#disable-battery-optimizati...

[qwertox]

I have a custom app which monitors my device 24/7 in the background with a persistent WebSocket connection to my server to receive push notifications. While yes, as a user you need to grant some permissions to the app, it works without a problem.

If by notifications you are referring to Firebase Cloud Messaging, which replaced Google Cloud Messaging and is now integrated in Google Play Services, I do agree with you.

It is my opinion that Google should be forced to decouple all these Google Play Services which are not related to the "Online App-Store" which Google Play is, and ideally be forced to open source it, so that trustworthy alternatives can be used to replace it.

I mean all the components which track you, which basically is what Google Play Services is: it is a system service created to monitor every possible activity you have with your phone (location, physical activity, health) as well as the efficient, persistent data connection to transfer these bits of information to Google (and to your device as FCM), while Google grants you access to use these services in a restricted manner.

For me Amazon's Android devices are basically useless since they don't have Google Play Services installed, and you have to work around it to get it installed, which basically is illegal to do.

[Andrew_nenakhov]

> While yes, as a user you need to grant some permissions to the app, it works without a problem.

Without the problem for YOU, because, presumably, you have a non-shitty phone. It is a much more severe issue on the likes of Samsung / Xiaomi / Huawei. See https://dontkillmyapp.com

[izacus]

> Without the problem for YOU, because, presumably, you have a non-shitty phone. It is a much more severe issue on the likes of Samsung / Xiaomi / Huawei. See https://dontkillmyapp.com

And if you force Google to stop verifying proper behaviour in CTS, do you expect those manufacturers to allow you to run background services at will?

They already kill off anything that's not whitelisted to keep those battery numbers up. The every new major OEM release we developers find bunch of new ways the try and kill background apps.

[Causality1]

I can back that up. Oneplus is infuriating in that they include a way for you to whitelist apps and then kill them anyway. I essentially have to plug my phone in if I want to run, say, a torrent client for more than two minutes.

[NGRhodes]

So the issue is not Android (AOSP) provided by google, but OEMs ?

[WastingMyTime89]

AOSP is not Android. What Google distributes to OEMs and its direct customers is not AOSP. It is AOSP and the different Play Services including the Play Store and the Google applications with contracts mandating that they have to be provided to customers.

If you think people talk about AOSP when they talk about Android, you are always going to miss the point of these conversations.

[schadara]

From the link posted above parent: https://dontkillmyapp.com

> Unfortunately, vendors (e.g. Xiaomi, Huawei, OnePlus or even Samsung…) did not seem to catch that ball and they all have their own battery savers, usually very poorly written, saving battery only superficially, with side effects.

[WastingMyTime89]

Heck, I was following the discussion regarding the notification system and how you couldn’t really do without and thought the link was actually relevant to the discussion without checking.

My apologies to NGRhodes, the link does indeed refer to an issue with OEM and not what Google supplies. I don’t really see what it has to do with the topic we were discussing however.

[Andrew_nenakhov]

The issue is the locked-in requirement to rely on the sole source of push notifications built into the system. Running your own persistent app is an ugly workaround that doesn't work more often than does.

Also, such apps are unloaded from time to time even on stock non-modified Androids.

[ohgodplsno]

AOSP is horribly crippled by Google, as they move more and more features towards play services. Location, push notifications, etc.

[Zanfa]

There's a good reason to not allow 3rd party background networking though. It would work in theory, but in practice every company with more than a few devs would eventually implement their own crappy version of a notification system with terrible power management. Given that even well-funded companies like Uber/Facebook have constant engineering quality issues, I definitely wouldn't want to have 20+ separate active notification connections.

This is not even hypothetical, this was literally happening before background services were heavily limited in earlier Android versions.

[Andrew_nenakhov]

The only thing you really need is an easily accessible tool to explicitly view which apps are allowed to run in the background. This doesn't have to occupy valuable notification area space.

With current notched phones which are limited to 4 notifications (you get four and a dot if there are more), you can have VPN app, XMPP client, Syncthing, and a generous space for one more incoming notification.

[izacus]

> The only thing you really need is an easily accessible tool to explicitly view which apps are allowed to run in the background. This doesn't have to occupy valuable notification area space.

Those tools existed and they didn't help because users had no recurse against poorly behaving apps. Meanwhile Android kept being reamed by reviewers and media for poor battery life and people kept buying locked-down iPhones instead because they lasted longer.

I spent countless of hours trying to get Android devs to not do dumbass things with battery ("oh, I need updates? I''ll just poll the server every 20 minutes and ruin the users battery in hours. Easier than long polling!") and in the end the situation didn't improve until Google stepped on the devs neck and forced them to use GCM/FCM and started actively powering down radio without apps input.

[Andrew_nenakhov]

Oh, so because of (some) dumb users let's punish everyone by making them subject to a monopoly lock-in. Great reasoning.

Users have very effective recurse against poorly behaving apps: uninstall. You just need to inform users that the app X does use much battery. Then it should be up to the users to decide if to allow this behaviour or uninstall this app. Maybe an explicit permission to run in the background. That's it.

Solution that you like is also very beneficial for OS vendor, how convenient.

[izacus]

> Oh, so because of (some) dumb users let's punish everyone by making them subject to a monopoly lock-in. Great reasoning.

In your reasoning, "most" users would be done and "most" apps would be malicious.

But in the end, it's quite simple - dealing with power use on mobile is hard and most developers don't care (same as they don't give a crap about making your web pages fast and slim). Users care about battery life above most of other features, including your freedom. They WILL got and buy a device that lasts the longer amount of time in the smallest and lightest package.

As long as these two things are true, leaving developers to run their polling code without restrictions has a massive effect on sales of both OEM devices and Android ecosystem as a whole. As such, OEMs are actively modifying Android to not allow this - see the wonderfully depressing https://dontkillmyapp.com/ - which is a significantly worse mess than you having to use a proprietary service to send a single device wakeup ping.

[Andrew_nenakhov]

If users care about battery life, give them tools to analyse what's eating it, but handcuff it so they don't hurt themselves. That way users who care about freedom will have it, and users who care about battery life above all will just block the permission to run in the background.

But that doesn't align with Google's interests, so it will never happen, unless they are forced to.

[thebean11]

Every single app that uses notifications needing to run in the background is not a scalable solution..

Even correctly behaving apps will consume much more power and network with this scheme.

[Andrew_nenakhov]

Imagine. Your macOS suddenly decudes that you don't need a web server running on your macbook, so it randomly kills it to free resources for your browser. Would you be happy with this level of care shown by the manufacturer?

It is not up to the OS to decide which apps the user needs running. Be it wifi scanner, vpn or some file sync process. All the user needs is a correct tool to know which app consumes what, and then there User must decide. Not some prick in google or xiaomi or samsung or whoever else.

[Zanfa]

Again, in a world of spherical cows this could work, but in reality it didn't.

I was an Android dev pre-5.0 and believe me, having lived through the s*tshow that was push notifications at the time, there's no way it would've worked any other way. Google's own SDKs were garbage and as with everything else they kept releasing new incompatible rebranded versions annually. Last I checked, the Firebase console for managing push notification subscriptions was still one of the worst SPAs I've come across.

You'd expect a thriving ecosystem of dedicated push notification providers to pop up and outcompete Google, but all of the 3rd party offerings were even worse, in terms of battery life, UX, reliability and even pricing.

It's unfortunate, but no developer cares about your battery life, because no user is going to switch away from using their app solely because of crappy power usage, so Google had to do exactly what they did.

Maybe in a decade or two, with new battery technologies, there won't be a physical limitation and this situation would play out differently.

[Andrew_nenakhov]

> I was an Android dev pre-5.0

I was in Android dev since pre 2.0, developing an app which need to constantly run. The situation got worse since then, not better.

A thriving ecosystem of dedicated push notifications can't pop up because you can't go to Android settings and choose another push notifications provider. You have to rely on ugly crutches to even receive a push notification, and that is why they are 'even worse'.

[joshuamorton]

Can you explain how this would work, does this mean that every app needs to push all notifications to all PN servers? Like there's the Google one and a theoretical Samsung one, my app has to send all push notifications to both, right, since I don't know a priori which provider you use, and if it's a global setting all apps need to use the same one.

So you either need every app developer to maintain the list of all push providers or someone, presumably Google, to maintain the canonical list and the thing that manages sending them to all the push providers.

Oh and then there's the associated suite of privacy issues. Do you really want every push notification prouder to get even metadata about all of your push notifications?

So this can't be a global setting. Instead it has to be a per-app setting where like the app provider needs to register a callback to update the notification server and support that in app. Of course most won't.

[curryst]

Why on Earth wouldn't you just make it a setting on the phone and expose that to apps? Have the apps read it on startup, and save it as part of the user profile server side. It basically becomes an email address.

> So this can't be a global setting. Instead it has to be a per-app setting where like the app provider needs to register a callback to update the notification server and support that in app. Of course most won't.

They seem to do fine with email addresses. Again, I don't see why '[e-mail]' (or an alternative with auth embedded) would be absurdly hard for developers. Someone will write a 'SendPushNotification' function that parses out the domain to send it to and the auth to use and send it, just like we've done with email since forever.

Google will likely know where you're sending your notifications, but they won't manage sending them (though they could probably scrape the contents since they own the OS).

[Aissen]

Corollary: notifications should be a public service whose only purpose would be to maintain that TCP connection and have a server as a broker, with clear control given to the user over which app can use said service and be allowed to wakeup their device.

[Spivak]

So almost literally how iOS and Android notifications work except I guess jointly owned and operated by by Google and Apple? Because public services like this are necessarily funded and operated by the companies that have a financial interest in them existing.

[Aissen]

Not really: you can't use the service alone, without any other software. The software is not open source. And finally, the user has no control over which apps/services can send them a notification (or really, a device wakeup) at a given moment.

[judge2020]

Those are downsides in many situations but the existing systems match your original description perfectly, unless "public" has a special description you didn't communicate.

[jsiepkes]

The current situation means I can't even use my own push notifications for company owned devices running company written software. That's just ridiculous.

[Zanfa]

You could always run a custom Android build on company devices.

[tut-urut-utut]

Sure, and by installing that custom build, you lose support and void warranty from your device provider.

[jsiepkes]

That, and even worse maintaining a custom build with changes and the infrastructure such as maintaining a OTA update server, etc. is a lot of work.

[anthropodie]

That persistent notification for application running in background is the worst. Back in the day, applications on Symbian devices could run in background without any indication. Some would argue that's a risk but then we have been using our PCs like that since forever.

Even if you want users to give indication of something running in background there are far less intrusive ways to achieve that. But if you can't come up with those ways and persistent notification is the best you could come up with then at least give the user the ability to turn it off without turning off background process?

They know the tendency of people to clear notifications as they clutter the phone and they use this small knowledge to push their own Software. At least follow same practice for Google apps but no because then that notification area would be complete mess.

[endless1234]

> Back in the day, applications on Symbian devices could run in background without any indication.

This was absolutely terrible for battery life, though. Accidentally leave the IRC client open in the background? There goes your battery. There are non-nefarious reasons why smartphones work they way they do nowadays.

[anthropodie]

But they did it less for saving battery life and more to push their own Software. There are many things that could have been done to avoid the battery issue.

[izacus]

They changed API to clearly show that an app is using battery to push their own software? Who's "they" and what kind of conspiracy theories have you been reading again?

[easytiger]

> Back in the day, applications on Symbian devices could run in background without any indication. Someone would argue that's a risk but then we have been using our PCs like that since forever.

I still think my Sony Ericson p910i the best smartphone. Multiple human interfaces (the multi directional wheel is the greatest invention to be abandoned) in particular and a near fully functional OS that could do anything my computer could do

[izacus]

> That persistent notification for application running in background is the worst. Back in the day, applications on Symbian devices could run in background without any indication. Some would argue that's a risk but then we have been using our PCs like that since forever.

"My Android phone just died at 1pm with empty battery, I don't know why! I'm buying an iPhone!" complaints are something noone from the Android side of the market wants to continue hearing. Even if that means that you now need to have a notfication clearly telling you your battery is being drained.

[AshamedCaptain]

> However, there is a difference with Android: iOS was crippled from the start, but first iterations of Android were relatively open, apps could run in the background without much issues. But then Google started tightening the grip, so Android now closely resembles iOS in this regard

As far as I know, Android still allows background applications though. The only constraint is that you have to show a persistent notification (that the user can hide). IM clients like Conversations.im will attest to that.

(iOS does not -- and thus creating a real Jabber client that does not depend on push notifications is impossible)

[Vinnl]

Relevant open source project: https://unifiedpush.org

[LeifCarrotson]

Another approach is microG, as used by CalyxOS:

https://calyxos.org/docs/guide/microg/

[j-james]

And yet another approach is sandboxed Play services, as used by GrapheneOS:

https://grapheneos.org/usage#sandboxed-play-services

[Andrew_nenakhov]

"The Distributor is the application you install on your device to get notifications. It receives notifications and distributes them to the other applications."

Such solutions are what I referred to as 'the best alternative'. No, thanks, it's not even close to the real deal.

[Vinnl]

The difference being, of course, that you only need one such application running, rather than every app that needs notifications running its own service. And of course, if a stock method every does become available, it only needs an update of UnifiedPush, rather than of every app that ships its own notification service.

[Andrew_nenakhov]

Captain Obvious, thank you. There problem is, such functionality should be at OS level to connect to chosen push notifications provider if Google Play services don't suit you. Not via some app that is subject to all android restrictions on persistent apps, including occasional unloading from memory.

The OS is designed in such way requires push notifications built into its core to work properly.

[jdiez17]

> Not via some app that is subject to all android restrictions on persistent apps, including occasional unloading from memory

Apps are subject to power and resource restrictions, but the user can easily lift those restrictions manually for certain apps, like UP Distributor. If the app is implemented properly, it will work just as well as Google's push service.

> The OS is designed in such way requires push notifications built into its core to work properly.

Notifications are built into the OS. A network-based push notification service is not part of the OS. It is false to say that (a particular implementation of) network-based push notifications have to be "built into the OS' core to work properly."

[Vinnl]

Of course, but it's not, so it's cool that this project exists and probably relevant to people's interests here, which is why I shared it.

[jdiez17]

While I agree with you that Google is abusing its power in the Android ecosystem to "encourage" (to put it nicely) developers to use their APIs, it is actually possible to manage this imbalance between what Google wants and what we, as users, want.

Specifically with regards to notifications the main things I care about are instant messaging apps (WhatsApp, Telegram, Signal and Element) and email notifications. All of these apps fall back to either their own implementation of a persistent connection or use the OS' APIs for background job scheduling and polling. In my experience, having a handful of persistent background connections does not significantly impact battery life (<5%/charge). Also, modern Android has Doze[1] mode. Unless an app is granted a battery optimization exception, its scheduled tasks (polling) will run within a brief period every N minutes, where N depends on how long the device has been inactive. This is what I want in most cases. I personally only give IM apps a battery optimization exception. Notifications from other apps are usually less urgent/important.

However if you want the convenience of having a single connection for all your notification delivery, GrapheneOS[2] (an AOSP-based OS with many security and privacy improvements) has a compatibility layer for Google Play services[3] that allows it to be run with non-system privileges (i.e. the same as every app the user installs). That means you can choose exactly which permissions are granted to it. For example, you can revoke all permissions except Network[4]. This lets you use Google's FCM service without Google knowing anything about you except your IP address (and the contents of your notifications).

Using Google as the notifications middleman is an ugly crutch though[5], but this setup is solid. This is only necessary because they have convinced Android developers to use their APIs, but alternatives can and do exist. I think UnifiedPush[6] is going in the right direction, but they haven't gained much traction yet.

[1] https://source.android.com/devices/tech/power/platform_mgmt

[2] https://grapheneos.org

[3] https://grapheneos.org/usage#sandboxed-play-services

[4] https://grapheneos.org/features (ctrl-f Network permission toggle)

[5] similar to using Apple as the notifications middleman, although the incentives are slightly different

[6] https://unifiedpush.org/

[havkd]

> The Android OS is intentionally crippled to work only with Google Play push notifications, and there are no 'stock' ways to use an alternative without modifying the OS.

So modify the OS… it’s open.

[oblio]

Modify the OS, install it on your phone, hope that if you have an issue you still have support from the vendor or the carrier...

If it were that easy as an overall solution, non-OEM Android forks would be mainstream. They're not. I hardly know any people using non-OEM Android.

[josefx]

And distribute the modified OS to all your customers how exactly? You could probably sell your own android based phone as long as you find a single manufacturer that isn't part of Googles OpenHandset alliance or bound by the Play Store license to only sell Google approved Android. Alternatively you could limit yourself to India and China where Google cannot enforce these terms. There really isn't any point in pretending that Android is an open OS in the west. It is Googles proprietary stack or nothing.