Task: Integrating AWS SQS and SNS for Reliable Messaging

Getting below error while creating resources using CloudFormation template.

The resource LambdaExecutionRole is in a CREATE_FAILED state

This AWS::IAM::Role resource is in a CREATE_FAILED state.

“Resource handler returned message: “User: arn:aws:iam::637423569811:user/kk_labs_user_608339 is not authorized to perform: iam:PutRolePolicy on resource: role lambda_execution_role because no identity-based policy allows the iam:PutRolePolicy action (Service: Iam, Status Code: 403, Request ID: 637a8802-9dd1-46a1-92fb-7793e3be50a6)” (RequestToken: 9cc94140-83a4-fe3a-2c64-e0823ae6cce7, HandlerErrorCode: AccessDenied)”

cat xfusion-priority-stack.yml
AWSTemplateFormatVersion: ‘2010-09-09’
Description: CloudFormation stack to create priority queuing system with SNS, SQS, and Lambda.

Resources:

High-Priority SQS Queue

HighPriorityQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: xfusion-High-Priority-Queue

Low-Priority SQS Queue

LowPriorityQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: xfusion-Low-Priority-Queue

SNS Topic

PriorityQueuesTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: xfusion-Priority-Queues-Topic

High-Priority Queue Subscription to SNS

HighPriorityQueueSubscription:
Type: AWS::SNS::Subscription
Properties:
Protocol: sqs
TopicArn: !Ref PriorityQueuesTopic
Endpoint: !GetAtt HighPriorityQueue.Arn
FilterPolicy:
priority: [“high”]

Low-Priority Queue Subscription to SNS

LowPriorityQueueSubscription:
Type: AWS::SNS::Subscription
Properties:
Protocol: sqs
TopicArn: !Ref PriorityQueuesTopic
Endpoint: !GetAtt LowPriorityQueue.Arn
FilterPolicy:
priority: [“low”]

Lambda Execution Role

LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: lambda_execution_role
AssumeRolePolicyDocument:
Version: ‘2012-10-17’
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: LambdaExecutionPolicy
PolicyDocument:
Version: ‘2012-10-17’
Statement:
- Effect: Allow
Action:
- sqs:ReceiveMessage
- sqs:DeleteMessage
- sqs:GetQueueAttributes
Resource:
- !GetAtt HighPriorityQueue.Arn
- !GetAtt LowPriorityQueue.Arn
- Effect: Allow
Action:
- sns:Publish
Resource: !Ref PriorityQueuesTopic
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: “arn:aws:logs:::*”

Lambda Function

PriorityQueueLambdaFunction:
Type: AWS::Lambda::Function
Properties:
FunctionName: xfusion-priorities-queue-function
Runtime: python3.9
Handler: index.lambda_handler
Code:
ZipFile: |
#!/usr/bin/env python3
import boto3
import os
sqs = boto3.client(‘sqs’)
def delete_message(queue_url, receipt_handle, message):
response = sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=receipt_handle)
return “Message " + “'” + message + “'” + " deleted”

     def poll_messages(queue_url):
         QueueUrl=queue_url
         response = sqs.receive_message(
             QueueUrl=QueueUrl,
             AttributeNames=[],
             MaxNumberOfMessages=1,
             MessageAttributeNames=['All'],
             WaitTimeSeconds=3
         )
         if "Messages" in response:
             receipt_handle=response['Messages'][0]['ReceiptHandle']
             message = response['Messages'][0]['Body']
             delete_response = delete_message(QueueUrl,receipt_handle,message)
             return delete_response
         else:
             return "No more messages to poll"
     def lambda_handler(event, context):
         response = poll_messages(os.environ['high_priority_queue'])
         if response == "No more messages to poll":
             response = poll_messages(os.environ['low_priority_queue'])
         return response
  Role: !GetAtt LambdaExecutionRole.Arn
  Environment:
    Variables:
      HIGH_PRIORITY_QUEUE_URL: !Ref HighPriorityQueue
      LOW_PRIORITY_QUEUE_URL: !Ref LowPriorityQueue

Outputs:
PriorityQueuesTopicArn:
Description: ARN of the SNS Topic
Value: !Ref PriorityQueuesTopic

HighPriorityQueueURL:
Description: URL of the High-Priority Queue
Value: !Ref HighPriorityQueue

LowPriorityQueueURL:
Description: URL of the Low-Priority Queue
Value: !Ref LowPriorityQueue

LambdaFunctionName:
Description: Name of the Lambda Function
Value: !Ref PriorityQueueLambdaFunction

Hi,

Please share the AWS level and task title, and I’ll look into it.

Level 4 Cloud (AWS) and task number 4 Integrating AWS SQS and SNS for Reliable Messaging

Hi,

This is a valid issue. It seems the user doesn’t have enough permissions to create the resources as required by the task. I’ve forwarded this to the team, and we’ll look into it. I’ll let you know if there is any new feedback.

Hi Raymond, Please test now and confirm if this is working now

Hi guys,

Please try again using managed policies to create the Lambda role successfully. For example:

  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: 'lambda_execution_role'
      AssumeRolePolicyDocument:
        Statement:
          - Action:
            - sts:AssumeRole
            Effect: Allow
            Principal:
              Service:
              - lambda.amazonaws.com
        Version: 2012-10-17
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonSQSFullAccess
        - arn:aws:iam::aws:policy/AmazonSNSFullAccess
      Path: /

Hi ,
Now I am getting the below error, I have attached a screenshot for the reference
High-priority message was not processed.

Hi @avimore2010

Could you please share more details about the error and your CloudFormation setup? It’s hard to understand the root cause without more context.

Hi @raymond.baoly
Following is my CloudFormation file, using this file I can create resources and run the provided commands. Still, after submitting that task, I am getting an error High-priority message was not processed.

AWSTemplateFormatVersion: ‘2010-09-09’
Description: SQS priority queues template

Resources:
SQSHighPriorityQueue:
Type: AWS::SQS::Queue
Properties:
VisibilityTimeout: 180
QueueName: xfusion-High-Priority-Queue

SQSLowPriorityQueue:
Type: AWS::SQS::Queue
Properties:
VisibilityTimeout: 180
DelaySeconds: 20
QueueName: xfusion-Low-Priority-Queue

PriorityQueuesTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: xfusion-Priority-Queues-Topic

SQSHighQueuePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
Queues:
- !Ref SQSHighPriorityQueue
PolicyDocument:
Id: AllowIncomingMessageFromSNS
Statement:
-
Effect: Allow
Principal: ‘*’
Action:
- sqs:SendMessage
Resource:
- !GetAtt SQSHighPriorityQueue.Arn
Condition:
ArnEquals:
aws:SourceArn: !Ref PriorityQueuesTopic

SQSLowQueuePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
Queues:
- !Ref SQSLowPriorityQueue
PolicyDocument:
Id: AllowIncomingMessageFromSNS
Statement:
-
Effect: Allow
Principal: ‘*’
Action:
- sqs:SendMessage
Resource:
- !GetAtt SQSLowPriorityQueue.Arn
Condition:
ArnEquals:
aws:SourceArn: !Ref PriorityQueuesTopic

SNSHighSubscription:
Type: AWS::SNS::Subscription
Properties:
TopicArn: !Ref PriorityQueuesTopic
Endpoint: !GetAtt SQSHighPriorityQueue.Arn
Protocol: sqs
RawMessageDelivery: true
FilterPolicy: {“priority”: [“high”]}

SNSLowSubscription:
Type: AWS::SNS::Subscription
Properties:
TopicArn: !Ref PriorityQueuesTopic
Endpoint: !GetAtt SQSLowPriorityQueue.Arn
Protocol: sqs
RawMessageDelivery: true
FilterPolicy: {“priority”: [“low”]}

LambdaRole:
Type: AWS::IAM::Role
Properties:
RoleName: lambda_execution_role
AssumeRolePolicyDocument:
Version: ‘2012-10-17’
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonSQSFullAccess
- arn:aws:iam::aws:policy/AmazonSNSFullAccess
- arn:aws:iam::aws:policy/AWSLambda_FullAccess

LambdaFunction:
Type: AWS::Lambda::Function
Properties:
FunctionName:
Fn::Sub: xfusion-priorities-queue-function
Description: Priority queue function
Runtime: python3.9
Code:
ZipFile: >
import boto3
import os
sqs = boto3.client(‘sqs’)
def delete_message(queue_url, receipt_handle, message):
response = sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=receipt_handle)
return “Message " + “'” + message + “'” + " deleted”

      def poll_messages(queue_url):
          QueueUrl=queue_url
          response = sqs.receive_message(
              QueueUrl=QueueUrl,
              AttributeNames=[],
              MaxNumberOfMessages=1,
              MessageAttributeNames=['All'],
              WaitTimeSeconds=3
          )
          if "Messages" in response:
              receipt_handle=response['Messages'][0]['ReceiptHandle']
              message = response['Messages'][0]['Body']
              delete_response = delete_message(QueueUrl,receipt_handle,message)
              return delete_response
          else:
              return "No more messages to poll"

      def lambda_handler(event, context):
          response = poll_messages(os.environ['high_priority_queue'])
          if response == "No more messages to poll":
              response = poll_messages(os.environ['low_priority_queue'])
          return response

  Handler: index.lambda_handler
  MemorySize: 128
  Timeout: 10
  Role:
    Fn::GetAtt:
      - LambdaRole
      - Arn
  Environment:
    Variables:
      high_priority_queue: !Ref SQSHighPriorityQueue
      low_priority_queue: !Ref SQSLowPriorityQueue

Outputs:
SNSTopicARN:
Value: !Ref PriorityQueuesTopic