Doing this with eBPF is definitely an improvement, but when I look at some of the sidecars we run in production, I often wonder why we can't just... integrate them into the application.
You can! There are downsides though for any sufficiently polyglot organization, which is maintaining all the different client SDK's that need to use that.
Sidecars are often useful for platform-centric teams that would like to have access to help manage something like secrets, mTLS, or traffic shaping in the case of Envoy. The team that's responsible for that just needs to maintain a single sidecar rather than all of the potential SDK's for teams.
Especially if you have specific sidecars that only work on a specific infrastructure, for example if you have a Vault sidecar that deals with secrets for your service over EKS IAM permissions, you suddenly can't start your service without a decent amount of mocking and feature flags. Its nice to not have to burden your client code with all of that.
Also, there is a decent amount of work being done on gRPC to speak XDS which also removes the need for the sidecar [0].
Another thing too is that if your main application artifact can be static while your sidecar can react to configuration changes/patches/vulns/updates. Depending on your architecture it can make some components last for years without a change even though the sidecar/surrounding configuration is doing all sorts of stuff. Back when more people ran Java environments there were all sorts of settings you can do with just the JVM without the bytecode moving for how JCE worked which was extraordinarily helpful.
It depends on your environment and architecture combined with how fast you can move especially with third party components. Having the microservice be 'dumb' can save everything.
> Especially if you have specific sidecars that only work on a specific infrastructure, for example if you have a Vault sidecar that deals with secrets for your service over EKS IAM permissions, you suddenly can't start your service without a decent amount of mocking and feature flags. Its nice to not have to burden your client code with all of that.
Could you please elaborate on this? I don't fully understand what you mean. Especially, I don't understand if "Its nice to not have to burden your client code with all of that" applies to a setup with or without sidecars.
Take vault for example. Rather than have to toggle a flag in your service to get a secret, you could have the vault sidecar inject the secret automatically into your container, as opposed to having to pass a configuration flag `USE_VAULT` to your application, which will conditionally have a baked in vault client that fetches your secret for you.
Your service doesn't really care where the secret comes from, as long it can use that secret to connect to some database, API or whatever. So IMO it makes your application code a bit cleaner knowing that it doesn't have to worry about where to fetch a secret from.
This is actually exactly the case I was thinking of. We have a few applications that use vault in a much more in-depth way than fetching a couple of secrets, so have the need to interact with it directly. We then have the much more common case of applications that use vault to fetch their database credentials, and for those we use a sidecar to fetch them at startup and another to renew them every 30 minutes.
The 2x sidecars do with 150 lines of YAML configuration what could be done with a library and 10 lines of java. And I don't buy the other theoretical benefits either. Easier to update? Each service can reference the library centrally from our monorepo whereas the YAML is copy-pasted to every service. It's also statically type-checked. Polyglot? Yeah, fair, but we're an almost entirely JVM shop.
Some of this could maybe be made easier with something like Kubevela but I don't think you're actually eliminating any complexity that way, just hiding it.
Author of linkerd argues that splitting this responsibility will improve stability as you'll have a homogeneous interface (sidecar proxy) over a heterogeneous group of pods. Updating a sidecar-container (or using the same across all applications) is possible, whereas if it's integrated into the application you'll encounter much more barriers and need much wider coordination.
Like it or not the socket has become the demarcation mechanism we use.
Therefore all software ends up deployed as a thing that talks on sockets.
Therefore you can't/shouldn't put functionality that belongs on the other end of the socket inside that thing. If you do that it's no longer the kind of thing you wanted (a discrete unit of software that does something). It's now a larger kind of component (software that does something, plus elements of the environment that software runs within). You probably don't want that.
Sidecars are often useful for platform-centric teams that would like to have access to help manage something like secrets, mTLS, or traffic shaping in the case of Envoy. The team that's responsible for that just needs to maintain a single sidecar rather than all of the potential SDK's for teams.
Especially if you have specific sidecars that only work on a specific infrastructure, for example if you have a Vault sidecar that deals with secrets for your service over EKS IAM permissions, you suddenly can't start your service without a decent amount of mocking and feature flags. Its nice to not have to burden your client code with all of that.
Also, there is a decent amount of work being done on gRPC to speak XDS which also removes the need for the sidecar [0].
[0] https://istio.io/latest/blog/2021/proxyless-grpc/