[btreecat]

I like this approach!

I am struggling to lock down a pod in my home cluster to allow local connections to it's web UI but force all other connections through a VPN client. I'm going to investigate if I could use squid for this.

My next approach is going to involve using a sidecar.

One heads up to the author, the text based charts didn't render well on FF mobile. Text is meant to reflow based on screen size, typeface etc. I feel this is a great case for using a drawing/image instead.

[fsmunoz]

Thank you!

Depending on what want for "lock down", this or something like this could work: you are essentially defining a single outbound communication path. In a way, your scenario was one of the reasons behind this experiment.

I'll take a look a the overflow thing, although I'm not sure if I will be able to fix it: I do have an image at the start which is an alternative to the text-based drawing, so nothing is lost. I use my own blogging solution that is essentially Texinfo (https://interlaye.red/Texiblog.html) so these blocks are the result of using an @example block (which is then converted into a preformatted block). I'm not sure this can be improved, apart from (as you said) using alternative images.

[baobun]

Using an http proxy like squid (or apache/haproxy/caddy/envoy/trafficserver/freenginx) does sound like what you should do next.

If you need the pod to do outbound connections as well as receive incoming traffic, usually that would be two different proxies (forward and reverse, respectively). Unless you do some fancy p2p service mesh.

[brynx97]

I had challenges with split-DNS in my homelab k3s cluster trying to do this. I ended up just putting the apps in docker-compose on a VM that has static routes for my local homelab networks. I looked at tailscale to solve this since it has a kubernetes operator, but tailscale doesn't fit my use cases or work well with all of my devices.

[btreecat]

> I had challenges with split-DNS in my homelab k3s cluster trying to do this. I ended up just putting the apps in docker-compose on a VM that has static routes for my local homelab networks. I looked at tailscale to solve this since it has a kubernetes operator, but tailscale doesn't fit my use cases or work well with all of my devices.

I don't need tails scale for this, seems like overkill.

I would like to better understand why my combination of marked packets and SOCK5 proxy are not fully working for certain UDP traffic. I also need to investigate if disabling ipv6 will help.

Using a VM or docker compose when I have k3s feels like admitting defeat with out understanding why.

[brynx97]

To each their own. I mostly figured out why, and I did not want to create too much tech debt in my homelab with brittle split-DNS and PostUp/PostUp wireguard configurations. I already had ansible and templates setup to move back to the VM and docker-compose. I did learn a fair bit on CoreDNS, so that was a worthwhile experiment.

[btreecat]

I didn't mean for you, I meant for me. I have truenas providing storage to my cluster but can easily just run a VM there.

I think you're approach is absolutely valid and didn't mean to seem like I was dismissive. Apologies.

[baobun]

> I would like to better understand why my combination of marked packets and SOCK5 proxy are not fully working for certain UDP traffic

I think UDP support for SOCKS5 proxies and clients is very spotty, especially beyond DNS. Probably some bugs out there. That might go for UDP in more or less esoteric container networking setups too...

If everything else fails, I've had the least hassle with socat, as well as just chucking workloads in full vm (if in container with --network=host) and using ip routes and policies.

[hosh]

I'm not sure I understand the issue.

Wouldn't the pattern be to use a reverse proxy for ingress and everything goes through there into the backends? Keep the pod ips range that is not directly reachable from outside the cluster?

[btreecat]

> I'm not sure I understand the issue. > > Wouldn't the pattern be to use a reverse proxy for ingress and everything goes through there into the backends? Keep the pod ips range that is not directly reachable from outside the cluster?

If all connections were inbound that would work fine. I'm trying to control traffic flowing in and out.