The step by step procedure to create an Amazon SQS queue and later test it with Amazon SNS.
What is an Amazon SQS?
Amazon SQS (Simple Queue Service) is the very first AWS service. It’s a message queueing service that helps with decoupling the architecture. It’s widely used in microservices designs. It’s a poll based service. So basically, it receives messages from some source, maintains messages in the queue for consumers to poll for it, consume it, and delete it after processing.
Using SQS, one need not keep consumer compute/infrastructure up and running all the time. SQS takes care of buffering and queueing messages and consumers can poll them once a while, processes them, and exists/shuts till the next poll. It also holds messages if consumers are not available or busy and scales accordingly so you don’t lose any messages.
Amazon SQS offers two types on queue and their differences are as below –
|Standard Queue||FIFO Queue|
|Default SQS queue||First in first out queue|
|Messages order is not preserved||Preserved messages order which is first in first out|
|message can be processed twice||Each messages processed exactly once|
|It can scale indefinitely||It can processes 3000 messages per seconds only|
|Best fit for scaling and high throughput requirement||Best fit where order is important and duplicates cant be tolerated|
|No naming restrictions||Queue name must ends with .fifo suffix.|
Lets dive into creating our first SQS queue.
How to create Amazon SQS queue?
- Log in to an Amazon SQS dashboard
- Click on the Create queue button on the introduction page.
- Create a queue wizard should open up –
- Choose the type of queue
- Name: Enter the queue name. For the FIFO queue, the name should have a .fifo suffix.
- Choose the type of queue
- Visibility timeout: Time for which message won’t be visible to other consumers when picked up by consumer for processing. Range 0 secs to 12 hours
- Messages retention period: Duration for which messages live in the queue if not gets deleted by any consumers. Range 1 min to 14 days
- Delivery delay: Time for which message won’t be visible to consumers to pick once it is received by the queue. Range 0 sec to 15 min. 0 means the message will be available immediately for processing once received by the queue.
- Maximum message size: Message size. Range 1KB to 256KB
- Receive message wait time: Max time for polling to wait for messages to be available. 0 means short polling (increases empty responses) and 20 means long polling. Range 0 to 20 secs.
- Access policy
- Simple bullet selections for access to send and receive messages to/from the queue. JSON will be generated accordingly.
- Use your own JSON to define policies
Three optional configurations for SQS queue.
- Encryption: Server side encryption offered by AWS.
- Dead-letter queue: Home for undeliverable messages.
- Tags: For identification purposes.
Click on Create queue button.
SQS should create a queue and present you with queue details.
You can click on the Send and receive messages button to manually send, receive and delete messages in this queue.
Enter the message body and click on the Send message button. Message will be sent to SQS queue.
On the same screen, if you scroll down you can click the Poll for messages button and receive the messages from the SQS queue which you sent in the earlier step.
Click on message ID to view message details. Under the message body tab you can see message body.
Once messages are processed (i.e. read in our case) you can delete messages from the queue by choosing a message and clicking the Delete button.
Sending messages to SQS queue from SNS
We walked through sending and receiving messages manually to and from the SQS queue. Now we will do it using SNS. SNS is a Simple notification Service offered by AWS. SNS topics are the channels where you publish messages for recipients. It’s a push-based service.
I already created an SNS topic named sns-topic-001 to which the SQS queue will be subscribed.
On an SQS queue details page click on the Subscribe to Amazon SNS topic button.
You should be able to select the existing SNS topic from the drop-down on the next page.
Click Save button.
- Login to the SNS dashboard
- Click on the Topic you created
- In the Topic details screen, click on the Publish message button
- Make sure you see the SQS queue under Subscriptions.
Fill in message details like Subject and body and click the Publish message button.
Now the message is published to the SNS topic. We have an SQS queue is subscribed to SNS Topic. That means this published message should appear in the SQS queue.
Lets go back to SQS queue and check for it.
You should see 1 message available to receive in SQS. And when clicked the Poll for messages button you will see that message in the messages details section below.
If you check message body you will see all SNS formatted message details.
In this way you can check the functionality of the SQS queue by sending and receiving messages manually or using SNS Topic. You can even receive messages automatically using the Lambda function and then publish it to another SNS Topic which can mail it to you for testing! We might cover this in another article. But this is it for now!