Level 2: AWS Q13

KodeKloud Engineer
#1

I tried to configure the same by myself but was not able to find the issue. Then used the Ai to help me then to i was not able to find the issue.

Please find the step i used below. (I think i will need a PEM file to login to the server but there is none configured in the instance and iam not able to login without the pem too)

VPC Peering Setup Guide for Nautilus DevOps (AWS Console)

Overview

This guide demonstrates setting up VPC Peering between a default public VPC and a private VPC using the AWS Management Console to enable communication between EC2 instances.

Prerequisites (Already Existing)

  • Public EC2: devops-public-ec2 in default VPC
  • Private VPC: devops-private-vpc (CIDR: 10.1.0.0/16)
  • Private Subnet: devops-private-subnet (CIDR: 10.1.1.0/24)
  • Private EC2: devops-private-ec2 in private subnet

Step 1: Create VPC Peering Connection

Navigate to VPC Console:

  1. Login to AWS Management Console
  2. Go to Services → VPC (or search for “VPC”)
  3. In the VPC Dashboard, click Peering connections in the left sidebar

Create the Peering Connection:

  1. Click Create peering connection button
  2. Fill in the configuration:
    • Peering connection name tag: devops-vpc-peering
    • VPC (Requester):
      • Select My VPCs
      • Choose your Default VPC from dropdown
    • Select another VPC to peer with: Choose My VPCs
    • Region: This region (us-east-1 or your current region)
    • VPC (Accepter): Select devops-private-vpc from dropdown
  3. Click Create peering connection

Accept the Peering Request:

  1. You’ll see a success message with the Peering Connection ID
  2. In the Peering Connections list, select your newly created devops-vpc-peering
  3. Click Actions → Accept request
  4. In the confirmation dialog, click Accept request
  5. You’ll see the status change from “Pending Acceptance” to “Active”

![Peering Connection Status should show “Active”]

Step 2: Identify Default VPC CIDR Block

Find Your Default VPC CIDR:

  1. In VPC Dashboard, click Your VPCs in the left sidebar
  2. Look for the VPC with Default VPC: “Yes”
  3. Note down the IPv4 CIDR (typically something like 172.31.0.0/16)
  4. Important: You’ll need this CIDR for the route table configuration

![Note your Default VPC CIDR - typically 172.31.0.0/16]

Step 3: Configure Route Tables

Update Default VPC Route Table:

  1. In VPC Dashboard, click Route tables in the left sidebar
  2. Identify the Default VPC Route Table:
    • Look for route table where VPC column shows your default VPC ID
    • Usually has Main: “Yes”
    • You can also filter by VPC ID to find it easily
  3. Select the default VPC route table
  4. Go to Routes tab (bottom panel)
  5. Click Edit routes button
  6. Click Add route button
  7. Configure the new route:
    • Destination: 10.1.0.0/16 (private VPC CIDR)
    • Target: Click the dropdown → Select Peering Connection
    • Choose devops-vpc-peering from the list
  8. Click Save changes

Update Private VPC Route Table:

  1. Filter or Find Private VPC Route Table:
    • Still in Route tables page
    • Look for route table where VPC column shows devops-private-vpc
    • Or use the search bar to filter by VPC name
  2. Select the private VPC route table
  3. Go to Routes tab (bottom panel)
  4. Click Edit routes button
  5. Click Add route button
  6. Configure the new route:
    • Destination: [Your Default VPC CIDR] (e.g., 172.31.0.0/16)
    • Target: Click the dropdown → Select Peering Connection
    • Choose devops-vpc-peering from the list
  7. Click Save changes

Verify Route Tables:

Both route tables should now show the new routes with:

  • Status: Active
  • Target: pcx-xxxxxxxxx (your peering connection ID)

Step 4: Update Security Groups

Find Private EC2 Instance’s Security Group:

  1. Go to Services → EC2 (or search for “EC2”)
  2. In EC2 Dashboard, click Instances in the left sidebar
  3. Find devops-private-ec2 instance in the list
  4. Select the instance by checking the box
  5. In the bottom panel, go to Security tab
  6. Note down the Security Group ID (starts with “sg-”)
  7. Click on the Security Group ID link to open it

Add ICMP Rule to Private EC2’s Security Group:

  1. You should now be in Security Groups page with your private EC2’s security group selected
  2. Go to Inbound rules tab (bottom panel)
  3. Click Edit inbound rules button
  4. Click Add rule button
  5. Configure the ICMP rule:
    • Type: Select All ICMP - IPv4 from dropdown
    • Protocol: ICMP (auto-filled)
    • Port range: -1 (auto-filled)
    • Source: Select Custom from dropdown
    • CIDR blocks: Enter your Default VPC CIDR (e.g., 172.31.0.0/16)
    • Description: Allow ping from default VPC (optional)
  6. Click Save rules

Verify Security Group Rule:

You should now see the new inbound rule listed:

  • Type: All ICMP - IPv4
  • Protocol: ICMP
  • Port range: 0 - 65535
  • Source: [Your Default VPC CIDR]/16

Step 5: Configure SSH Access to Public EC2 (Complete Guide)

Understanding the SSH Key Situation:

Current State: Your public EC2 (devops-public-ec2) was launched with some SSH key pair, and you want to add your AWS client host’s SSH key (/root/.ssh/id_rsa.pub) so you can SSH directly without needing the original key.

Part A: Find How You Currently Access the Public EC2

First, determine how the public EC2 was set up:

  1. Go to EC2 Dashboard → Instances
  2. Select devops-public-ec2
  3. In the bottom panel, look for Key pair name
  4. Note down the key pair name (e.g., my-ec2-key, devops-key, etc.)

Part B: Locate Your Current Access Method

You have a few scenarios:

Scenario 1: You Have the Original Key Pair (.pem file)

If you have the .pem file that was used to launch the public EC2:

  • Usually stored in ~/.ssh/ or Downloads folder
  • Named something like my-ec2-key.pem, devops-key.pem
  • Location: /path/to/your-key.pem

Scenario 2: You Don’t Have the Original Key

If you don’t have the original key file, you have options:

  • Use AWS Systems Manager Session Manager (if enabled)
  • Use EC2 Instance Connect (if available)
  • Ask your team/admin for the key file

Scenario 3: Password/Other Authentication

Some setups might use password authentication or other methods.

Part C: Get Public EC2 Details

  1. Still in EC2 Dashboard → Instances
  2. Select devops-public-ec2
  3. Note down:
    • Public IPv4 address (e.g., 54.123.45.67)
    • Key pair name
    • Security group (click on it to verify SSH is allowed)

Part D: Get Your SSH Public Key Content

On your AWS client host (where you want to SSH from), run:

cat /root/.ssh/id_rsa.pub

Copy the entire output - it looks like:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7... (long string) ...root@your-hostname

Part E: Connect to Public EC2 (Various Methods)

Method 1: Using Original Key Pair (.pem file)

# If you have the original .pem file
ssh -i /path/to/your-key.pem [email protected]

# Example:
ssh -i ~/.ssh/devops-key.pem [email protected]
ssh -i ~/Downloads/my-ec2-key.pem [email protected]

Method 2: Using EC2 Instance Connect (Browser-based)

  1. In EC2 Console, select devops-public-ec2
  2. Click Connect button (top right)
  3. Choose EC2 Instance Connect tab
  4. Click Connect - opens browser-based terminal

Method 3: Using AWS Systems Manager Session Manager

  1. In EC2 Console, select devops-public-ec2
  2. Click Connect button
  3. Choose Session Manager tab
  4. Click Connect

Method 4: If Password Authentication is Enabled

ssh [email protected]
# Enter password when prompted

Part F: Add Your SSH Key (Once Connected)

Once you’re connected to the public EC2 (using any method above):

  1. Check if .ssh directory exists:

    ls -la ~/.ssh/

  2. Create .ssh directory if needed:

    mkdir -p ~/.ssh

  3. Add your public key to authorized_keys:

    # Open the authorized_keys file in nano editor
nano ~/.ssh/authorized_keys

    OR use echo command:

    # Replace the ssh-rsa string below with your actual public key
echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7vQ... root@your-hostname" >> ~/.ssh/authorized_keys

  4. Set proper permissions:

    chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

  5. Verify the key was added:

    cat ~/.ssh/authorized_keys

  6. Exit the SSH session:

    exit

Part G: Test New SSH Access

From your AWS client host, test the new SSH access:

ssh [email protected]

You should now connect WITHOUT needing the original .pem file!

Troubleshooting Common Issues:

Issue 1: “Permission denied (publickey)”

Solutions:

  • Verify your public key was added correctly
  • Check file permissions: chmod 600 ~/.ssh/authorized_keys
  • Ensure you’re using the correct username (ec2-user for Amazon Linux)

Issue 2: “Connection timed out”

Solutions:

  • Check security group allows SSH (port 22) from your IP
  • Verify the public IP address is correct
  • Check if instance is running

Issue 3: Can’t Find Original .pem File

Solutions:

  • Check Downloads folder: ls ~/Downloads/*.pem
  • Check .ssh folder: ls ~/.ssh/*.pem
  • Use EC2 Instance Connect from AWS Console
  • Contact your team/admin for the key

Issue 4: “Host key verification failed”

Solutions:

ssh-keygen -R 54.123.45.67  # Remove old host key
ssh [email protected]   # Try again

Summary of What This Step Accomplishes:

:white_check_mark: Identifies how to currently access your public EC2
:white_check_mark: Adds your AWS client host’s SSH public key to the public EC2
:white_check_mark: Enables direct SSH access without the original .pem file
:white_check_mark: Prepares you for Step 6 testing

Step 6: Test the VPC Peering Connection

Get Private EC2 IP Address:

  1. In EC2 Dashboard → Instances
  2. Find devops-private-ec2 instance
  3. Note down its Private IPv4 address (should be something like 10.1.1.x)

SSH into Public EC2 and Test Ping:

  1. SSH into the public EC2 from your AWS client host:

    ssh ec2-user@[PUBLIC_EC2_PUBLIC_IP]

  2. From the public EC2, ping the private EC2:

    ping -c 4 [PRIVATE_EC2_IP]
# For example: ping -c 4 10.1.1.100

Expected Successful Output:

PING 10.1.1.100 (10.1.1.100) 56(84) bytes of data.
64 bytes from 10.1.1.100: icmp_seq=1 ttl=64 time=1.12 ms
64 bytes from 10.1.1.100: icmp_seq=2 ttl=64 time=0.892 ms
64 bytes from 10.1.1.100: icmp_seq=3 ttl=64 time=0.901 ms
64 bytes from 10.1.1.100: icmp_seq=4 ttl=64 time=0.895 ms

--- 10.1.1.100 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 0.892/0.951/1.120/0.098 ms

Additional Testing:

You can also test network connectivity:

# Test traceroute to see the path
traceroute [PRIVATE_EC2_IP]

# Check routing table
netstat -rn

Verification Through AWS Console

1. Verify Peering Connection Status:

  1. VPC Dashboard → Peering connections
  2. Find devops-vpc-peering
  3. Status should show: Active
  4. State should show: Active

2. Verify Route Tables:

  1. VPC Dashboard → Route tables
  2. Check Default VPC Route Table:
    • Should have route: 10.1.0.0/16 → pcx-xxxxxxxxx (Active)
  3. Check Private VPC Route Table:
    • Should have route: [Default VPC CIDR]/16 → pcx-xxxxxxxxx (Active)

3. Verify Security Group Rules:

  1. EC2 Dashboard → Security Groups
  2. Find private EC2’s security group
  3. Inbound rules should include:
    • Type: All ICMP - IPv4
    • Source: [Default VPC CIDR]
    • Description: Allow ping from default VPC

4. Verify EC2 Instance Details:

  1. EC2 Dashboard → Instances
  2. Public EC2 (devops-public-ec2):
    • Should have Public IP
    • Should be in default VPC
  3. Private EC2 (devops-private-ec2):
    • Should have only Private IP (10.1.1.x)
    • Should be in devops-private-vpc

Troubleshooting Common Issues

Issue 1: Ping Times Out or Fails

Possible Causes:

  • Security group doesn’t allow ICMP traffic
  • Route tables not configured correctly
  • Peering connection not active

Solutions:

  1. Check Security Group: Ensure ICMP rule is added with correct source CIDR
  2. Check Route Tables: Verify both VPCs have routes to each other via peering connection
  3. Check Peering Status: Ensure peering connection status is “Active”

Issue 2: SSH Access Denied to Public EC2

Possible Causes:

  • Public key not added correctly
  • SSH permissions incorrect

Solutions:

  1. Verify Key Addition: SSH using existing method and check ~/.ssh/authorized_keys
  2. Check Permissions:
    chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Issue 3: Cannot Find Default VPC

Solutions:

  1. In VPC Dashboard: Look for VPC with “Default VPC” = “Yes”
  2. Check Region: Ensure you’re in the correct AWS region

Issue 4: Route Addition Fails

Possible Causes:

  • CIDR block conflicts
  • Peering connection not active

Solutions:

  1. Verify CIDR Blocks: Ensure no overlap between VPC CIDRs
  2. Check Peering: Ensure connection is accepted and active

Debug Commands (from Public EC2):

# Check routing table
netstat -rn

# Test connectivity path
traceroute [PRIVATE_EC2_IP]

# Check if ICMP is reaching
tcpdump -i any icmp

Summary of What You’ve Accomplished

:white_check_mark: VPC Peering Connection: Created devops-vpc-peering between default VPC and devops-private-vpc

:white_check_mark: Bidirectional Routing: Configured route tables to allow traffic flow both ways

:white_check_mark: Security Configuration: Updated private EC2’s security group to allow ICMP from default VPC

:white_check_mark: SSH Access: Added your SSH public key to public EC2 for easy access

:white_check_mark: Connectivity Testing: Verified ping works from public EC2 to private EC2

Network Architecture:

┌─────────────────────────────────────┐    ┌─────────────────────────────────────┐
│           Default VPC               │    │        devops-private-vpc           │
│        (e.g., 172.31.0.0/16)       │    │          (10.1.0.0/16)             │
│                                     │    │                                     │
│  ┌─────────────────────────────┐   │    │   ┌─────────────────────────────┐   │
│  │     devops-public-ec2       │   │    │   │     devops-private-ec2      │   │
│  │     (Public IP)             │   │◄──►│   │     (Private IP: 10.1.1.x) │   │
│  │     (Private IP: 172.31.x.x)│   │    │   │                             │   │
│  └─────────────────────────────┘   │    │   └─────────────────────────────┘   │
└─────────────────────────────────────┘    └─────────────────────────────────────┘
                    │                                          ▲
                    └─────── VPC Peering Connection ──────────┘
                            (devops-vpc-peering)

Key Benefits:

  • Secure Communication: Private resources remain isolated from internet
  • Network Segmentation: Different VPCs for different purposes
  • Cost-Effective: VPC Peering has minimal costs compared to other connectivity options
  • High Performance: Low latency communication between VPCs

Your VPC Peering setup is now complete and functional! The public EC2 instance can successfully communicate with the private EC2 instance while maintaining proper network isolation and security.

#2

Hi @amrith_2006

Are you still facing any issues with it, or did you just want to share the solution?