How to securely connect EC2 via SSH with AWS Systems Manager

Modern best practice to connect Amazon EC2 instance via SSH without SSH key/password and with closed inbound 22 port. AWS Systems Manager Session Manager tutorial.

4 min readJan 3, 2020

Step 1: Launch Amazon EC2 Instance
Step 2: Create AWS IAM Role
Step 3: Connect Amazon EC2 via SSH
Extra: Connect using SSH command and SSH key
Sources

Disclaimer: I do not represent my current/previous employers on my personal Medium blog.

Step 1: Launch Amazon EC2 Instance

Navigate to Amazon EC2 and start launching new instance. In this tutorial I will use Amazon Linux 2 AMI (HVM) operating system.

Create a new security group with no rules (e.g. MediumSG) and assign to your EC2 instance:

Proceed without SSH key pair:

Finally copy and save Instance ID (e.g. i-06fd9f063a7cf53fd).

Step 2: Create AWS IAM Role

Navigate to AWS IAM and create new role. Choose EC2 service and click Next: Permissions:

Select AmazonSSMManagedInstanceCore policy. Click Next: Tags. Click Next: Review. Enter Role name (e.g. MediumRole) and Role description (e.g. AWS Systems Manager Session Manager for EC2 Instance). Click Create role:

Attach your new role (e.g. MediumRole) to your EC2 instance from Step 1. Navigate to Amazon EC2 select your instance (e.g. i-06fd9f063a7cf53fd) and click Actions -> Instance Settings -> Attach/Replace IAM Role. Select the role and click Apply:

Step 3: Connect Amazon EC2 via SSH

You can connect EC2 instance with AWS Systems Manager Session Manager in multiple ways: E2 console, Systems Manager console, AWS CLI.

3.1 Connect using Amazon EC2 console

Navigate to Amazon EC2 select your instance and click Connect. Select Session Manager and click Connect:

Note: If you saw We weren’t able to connect to your instance error, navigate to AWS Systems Manager. Select the same region as for your EC2 instance (e.g. N. Virginia). Click Get Started with Systems Manager. Finish Systems Manager Quick Setup.

3.2 Connect using AWS Systems Manager console

Navigate to AWS Systems Manager and select Instances & Nodes -> Session Manager. Click Start session. Select your instance and click Start session:

3.3 Connect using AWS CLI

$ aws ssm start-session --target INSTANCE_ID

Example:

$ aws ssm start-session --target i-06fd9f063a7cf53fd

Extra: Connect using SSH command and SSH key

Launch new EC2 instance (e.g. i-077b1f947c98988d5) and download SSH key (e.g. key.pem).

The Amazon EC2 instance must have the latest SSM Agent installed to allow connection using SSH. You can install the agent on both Windows instances and Linux instances.

On Amazon Linux 2, you can install the latest SSM Agent by using the following command:

$ sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm

Let’s prepare your local machine. First, install the latest Session Manager Plugin on you local machine.

Second, update ~/.ssh/config file on your local machine and add the code below:

Third, start a session by using the following ssh command:

$ ssh -i SSH_KEY ec2-user@INSTANCE_ID