Our organization recently looked into AWS Session Manager for tunneling but couldn't find documentation on how to make it work for our usecase. We were trying to tunnel into our VPC in order to be able to connect to an Amazon DocumentDB cluster. We don't have any EC2 instances which seems to be the only thing Session Manager has support for. Despite the callouts that Session Manager replaces bastion servers, that didn't seem to be the case for us. Did we miss something in our research?
Last I checked the "tunneling" only works to redirect traffic to a different port on the same SSM managed instance. The tunnel cannot be established with another box in the same VPC. So I don't think you can call it tunneling until they add that feature. Here's the GitHub issue where they discuss the limitation and a workaround: https://github.com/aws/amazon-ssm-agent/issues/208
If you're doing a federated login with Okta or another provider, you need to set up the hardware MFA there.
There is SSH tunneling support as well, will add an update on that soon.