You're right in a sense, but there's no aws-managed bastion. Session manager communicates with your instance via an outbound-created websocket connection. Inputs and outputs are piped through it.
I'd love to learn how you're using Session Manager or what other features/integrations you'd like to see us explore. Also if the terraform module packaging is useful. There are additional Session Manager features like port forwarding that I plan to write about soon.
Can you write one about port forwarding? Specifically, I would like to understand how various web interfaces on EMR cluster can be accessed through Sessions Manager. (Ganglia, Spark history server, etc.)
Our organization recently looked into AWS Session Manager for tunneling but couldn't find documentation on how to make it work for our usecase. We were trying to tunnel into our VPC in order to be able to connect to an Amazon DocumentDB cluster. We don't have any EC2 instances which seems to be the only thing Session Manager has support for. Despite the callouts that Session Manager replaces bastion servers, that didn't seem to be the case for us. Did we miss something in our research?
Last I checked the "tunneling" only works to redirect traffic to a different port on the same SSM managed instance. The tunnel cannot be established with another box in the same VPC. So I don't think you can call it tunneling until they add that feature. Here's the GitHub issue where they discuss the limitation and a workaround: https://github.com/aws/amazon-ssm-agent/issues/208
I never see mention of Windows with Session Manager. I have a mixed infrastructure with a number of Windows (IIS) app servers running various things.
We currently connect via SSH to a bastionhost, then tunnel from there to various systems, which allows connecting to SSH (linux instances), RDP (Windows), or basically any other network services like Redis or a database. I ended up writing some scripts to automate all this, so as long as you have the right certificates and IAM permissions, you can connect with a single command -- for Windows instances, it even retrieves the randomized password from the EC2 API. The end result is for any EC2 instances you're instantly popped into a shell/RDP session without having to enter credentials.
I'd love to replace this with something better (eg Session Manager), but I've not seen how to do this for RDP, and haven't had the time to go experimenting on my own to see if it's even possible. If I can't 100% replace the bastionhosts, having two entirely different connection methods doesn't solve anything (and in fact makes it worse, because it's harder to use).
Be careful with SSM in general. The documentation suggests adding the AmazonEC2RoleforSSM policy to the role of the EC2 instances you want to access via Session Manager. This role grants read/write to all S3 buckets in your account (amongst other things). See this article for better steps and unavoidable risky things: https://cloudonaut.io/aws-ssm-is-a-trojan-horse-fix-it-now/
There are so many AWS managed policies that provide access far beyond what one might suspect given the policy name and description.
Implementing least privilege with IAM can be really difficult in any moderately complex environment.
And close to impossible in highly complex environments. We're looking to adopt using AWS accounts to achieve a few objectives but one being reducing the blast radius of potential breaches and outages.
Good call to watch out for this stuff. The examples in the repo we set up use the AmazonSSMManagedInstanceCore
managed policy, which does not grant any S3 permissions, just various ssm, ssmmessages, and ec2messages permissions.
> The documentation suggests adding the AmazonEC2RoleforSSM policy to the role of the EC2 instances
Which documentation do you mean? The article mentions the policy AmazonSSMManagedInstanceCore, which is the same as what's mentioned in the SSM setup guide:
Thanks for clarifying, I didn’t recheck since we rolled out SSM in mid-2019 and then scrambled when we realised we’d granted account wide S3 permissions. The article I linked to also has a recommended minimal IAM policy for Run Command and SSM. I’ll update my comment to mention this.
AmazonSSMManagedInstanceCore is still too much access, it has unscoped ssm:GetParameter! I hope you weren't trying to protect any secrets in ParameterStore!
Does anyone know how this works with other utils that use SSH protocol, like rsync? What about tunneling other services to or from a local host? I'd love to have fewer hosts to maintain and a smaller network/attack surface, but we use SSH for more than just gaining commandline access to our instances.
>"No more bastion hosts required! Session Manager uses AWS APIs to communicate with your instances, so you can remove the administrative burden of maintaining bastion hosts."
Does this presume the EC2 instances have a public IP or is there a way this would also work with EC2 instances on private subnets?
It's great for managing active SSH sessions, but not so much for the other purpose for bastions: fine-grained network access control+routing. It would be cool if they made a more specific version of this just for network traffic without the SSH component.
FWIW, the project is open source, so you could build a modified agent for your purposes and inject it via cloud-init or your favorite config management tool: https://github.com/aws/amazon-ssm-agent
Would be interesting to lock down the session manager agent (if possible) so that the only way to privileged access is through sudo-like priv esc that uses 2fa.
As far as I know, SSH over SSM doesn't do anything regarding user management. It just establishes an SSH connection. Management of users on the host, authorized SSH keys, etc. is totally out of scope for SSM.
So if you already have access control setup on your host, then SSM doesn't do anything to undermine it. If you don't have it, you'll still need to add it.