It's a generic setup where you can do _anything_ you want via firing off an HTTP request as a clone of the original, and re-handling the request after getting back the response.
Big shoutout to the Authelia team for helping out with testing and confirming that it works. Will be merged very soon (today?) and some follow-up work is coming to make the config for it even shorter (some sugar for the common forward_auth usecase).
Me too. Thanks for being so helpful! Using Tailscale has been a pleasure, and you've got a great team over there, so integrating with TS has also been a dream.
Neat to see this up here! When I saw xena's initial post about Grafana I saw an opportunity to make it work with an existing, well known, basically bullet proof proxy (nginx). Xena took my sketchy POC and made it great :)
One interesting enhancement, which is probably not within scope of this tool, would be a way to logically AND a bunch of these auth tools together. With that you could use this tool without changes to assert that someone is allowed on the tailnet then pass those results to a second process that checks what groups they belong to and authorizes them for the particular upstream being checked. One could accomplish that with one nginx proxy per application, of course, and lean into Tailscale's ACLs for authz.
You might just want to integrate a policy rules engine like open policy agent: https://www.openpolicyagent.org/ It can act as a server which you bounce a subrequest against to get an authorization answer from a policy you defined ahead of time with a simple language.
And if you don't have time or want to do that, check out Pomerium it's basically a forward auth proxy with OPA policy engine integrated into it already: https://www.pomerium.com/
Yeah I've been thinking about that too. Something I've been wondering about is tying things to the ACL file through something like "capabilities"[1], but this would probably require a fair bit of per-service hacking. I think it'd be worth it, but it would be a lot of work. The main problem here is that Tailscale ACLs only really have "can connect to port" as the main capability they provide. I think I could end up telling the nginx-auth proxy if the person is a network admin or not (I'm not sure if that capability reliably shows up in whois responses, will need to check), that may be a starting point but it certainly won't scale.
I.e. pass the caps through an nginx variable up to the `/auth` location, then out to `nginx-auth`, then nginx-auth passes all(?) of the user's caps to the upstream.
This is pretty cool, but does potentially open any of these services not just to the browser but to any malware running on your clients. Probably not a huge deal in most cases but something to keep in mind.
Author of the post here. Realistically this is about as dangerous as what you have already with anything behind an SSH server. If you really need to be sure an actual human is making a request, use a yubikey 2fa challenge. I'll update the post to include this on Monday (I'm off for the rest of the week).
Are there any additional issues with CSRF? I assume if websites are already protected against CSRF then there are no problems and if they don't have CSRF protection then they already have a problem they need to fix.
I think if you are checking the host header on the server there is no problem. But I think if you are not checking the host header then there are some cute DNS rebinding attacks that will let an evil website perform arbitrary actions on behalf of a user if they are tricked into navigating onto the evil website. i've seen this same attack on sites hosted locally where authentication is assumed because only the localhost can connect to the site. DNS rebinding breaks this assumption. i think there is a similar thing going on here.
the good thing is a lot of stuff kind of implicitly checks the host header. like if you are using vhosting (without a default) then you have an implicit host header check even if you didn't set it up explicitly for security.
EDIT: and of course if you are using HTTPS then it should not be an issue because the server will not be able to serve a certificate that matches the hostname.
It’s good to protect against DNS rebinding, and the host header or TLS will protect against this, but rebinding isn’t the main source of CSRF issues. You can still blindly perform CSRF attacks via XHR or <form>.
The disadvantage to Tailscale’s implicit authentication is that it can’t take advantage of modern features like SameSite cookies, which can be a strong defense against CSRF. You would need to implement CSRF tokens everywhere, or try to rely on Origin/Referer (which is sketchy).
This is awesome! I was looking to set up auth on a tailnet-exposed Nginx server and now I know how I'll handle it :) Saves me a ton of efforts!
Now all I want from them is a way to bind an auth key to a specific IP address (for containers when they restart they keep the same DNS record). I only mention this because I know the devs read HN and I'm hoping they'll see this :)
Can you email me at xe at tailscale dot com? I'd be happy to learn more about your setup and make suggestions for you, might even turn that into one of these posts!
This is such a damn elegant solution for an Auth Proxy, using Wireguard and all the existing, solid abstractions. You guys are just on fire.
I may have missed this in the post, but is there any plans to make a general purpose one rather than Grafana, etc? like tailscaled --proxy --to or (and I saw mholt's post) just rely on something like Caddy for that?
I'm not sure yet. A lot of this is still experimental to feel out the problem space. Doing this as sidecars/extra things you run on the side lets us learn more about how people want to use this until we make this an actual product. If you have feedback/suggestions though, I am welcome to hear it and forward it along to the team.
Just thinking "out loud" here – this isn't standards compliant or anything.
I'd love to be able to control some of these kinds of things right in the Admin UI.
Like being able to say "create new proxy, use this relay that gets me into this network, look at this acl to decide who gets in"
I could see this as really useful for the long tail of "admin" type services. Like admin UIs for Sidekiq (Ruby bg jobs) or Oban (Elixir bg job), our HashiCorp Nomad or Consul admin screens, etc without having to mess around with extra tokens.
I’m a big fan of using subrequest authentication with nginx. I’ve been using it for years, but in my case, it uses plain http-auth to get the credentials from the user (with an ldap backend).
For this implementation, how does the request work for the user? Is it http-auth, or some other SSO web login form?
It's transparent. The user is already authenticated to the tailnet by dint of having a tailscale IP. This implementation asks the local tailscale daemon for who the request's IP belongs to and just passes that back up to nginx. It does some filtering to make sure it's an actual user instead of a service machine (i.e. does the IP have tags or not, basically) but otherwise leans entirely on Tailscale's ACLs.
Yep! As the author alluded to elsewhere in the comments, if you want to make _really sure_ that the user is a person in a browser instead of a rogue malware process on their machine you can combine this with a yubikey tap or webauthn attestation step.
I've been meaning to try something similar for Kubernetes/OpenShift-- you can set up authentication (technically an identity provider) through a configurable HTTP header. My idea was having the reverse proxy only listen on the tailscale IP, but this is even cooler.
It would be great if authentication via SSO would be possible on clients that are not running the Tailscale client. A bit like Cloudflare Access. Currently, I’m using both to manage access to my servers.
Basically a public access endpoint to services which authorizes with SSO to the same endpoint which your tailscale account authorizes and could let you access HTTPS services.
You could do other things too like exposing services publicly without auth (by tailscale anyway) like ngrok.
I can conceptualize it well, but, I'm not a huge fan of the idea honestly primarily because tailscale is so easy to use.
Fascinating post. I have been reading a lot on Tailscale lately and this kind of application came to mind immediately.
The post briefly mentions that the proxy can be set to listen a unix socket instead of a TCP socket. Is that referring only to the subauth socket, or the entire nginx? It seems like the real security value is in the latter, but that would be nginx config right? Setting it to only accept traffic from Tailscale rather than from :80?
The subauth socket. I generally suggest people run all their services on Unix sockets as much as possible to prevent accidentally exposing things to the internet unintentionally, however it's not the best idea to run your reverse proxy on a Unix socket unless you have even more cursed things going on :)
Thanks for the quick reply, that helps a lot. In the case where we're not intending to handle Internet traffic on the proxy--only Tailscale--it would still be possible to bind the Nginx listener to only the Tailscale adapter though right?
I'm imagining a use-case where Tailscale + Nginx is acting kind of like an Azure App Proxy where you want to ensure that all access to the upstream service is validated against the Tailscale ACL. Maybe that's already implicitly the case since the auth_request goes to nginx-auth, so any traffic that isn't coming from a Tailscale IP is going to be denied anyways.
Thanks! I knew they worked similarly, but I wasn't sure if they were actually speaking the same protocol or not, since Traefik doesn't seem to deign to mention nginx in their docs.
This would be cool together with a kubernetes operator / ingress kind of thing that would expose services to tailscale, and maybe some kind of tailscale load balancer, with magicdns