# No Bottlenecks. Just SQS.

When building modern applications, we often encounter a dilemma:

* Should we process tasks immediately within the API call?
    
* Or offload them and return quickly?
    

As your application scales, tasks like sending emails, processing payments, resizing images, or syncing data can introduce latency and risk. That’s where **queues** shine — by allowing systems to decouple work and handle it **asynchronously**.

## What is a Queue in Software?

A **queue** is a data structure that follows **FIFO** (First-In, First-Out) — the first item added is the first to be removed.

In distributed systems:

* A **Producer** adds jobs/tasks/messages to the queue.
    
* One or more **Consumers** process those messages independently.
    

![Distributed Messaging Queue: Everything You Need to Know About | by Arslan  Ahmad | Geek Culture | Medium](https://miro.medium.com/v2/resize:fit:1382/0*TYGDLcBNJM49PpPV align="left")

This separation improves responsiveness, fault tolerance, and scalability.

### Why Queues Are Critical in Modern Architectures

1. **Decoupling Components:** Queues allow services to operate independently. (Ex, If the email service is slow, the signup API still works.)
    
2. **Improved Resilience:** If a consumer crashes, the queue persists tasks until it's back.
    
3. **Auto-Scaling:** You can increase consumers to process high loads dynamically.
    
4. **Retry & Error Handling:** Failed jobs can be retried or moved to a Dead Letter Queue (DLQ).
    
5. **Rate Limiting at Scale:** You can control how fast background jobs are processed, avoiding downstream overload.
    

## Why Choose AWS SQS as Your Queueing System?

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1744527503375/2a8bb9c9-34e3-4b84-833f-40ce0267855f.png align="center")

Amazon SQS stands out as a robust, fully managed message queuing service that eliminates the overhead of managing message brokers and scaling infrastructure manually. Designed for durability, flexibility, and performance, SQS offers features that make it ideal for both startups and enterprise-grade applications. Its native integration with the AWS ecosystem (like Lambda, SNS, and CloudWatch) allows for powerful event-driven architectures. You don’t have to worry about provisioning servers or handling failover — SQS scales automatically with your workload.

Features like visibility timeouts, dead-letter queues, long polling, and encryption make it a secure and intelligent choice when building modern, asynchronous backend systems. Amazon SQS (Simple Queue Service) is a fully managed message queuing service that:

* Scales automatically
    
* Offers at-least-once delivery
    
* Works seamlessly with other AWS services
    

![Diagram detailing how SQS works.](https://a.b.cdn.console.awsstatic.com/a/v1/XKO4B6QDVFJTQWTL2XED4H2RRKZ73K7O4MYWLEE2XWGFYJ53GTLA/assets/images/55fc40afe501ed4ed4ed-SQS-diagram.svg align="left")

Amazon SQS allows producers to send messages to a queue. Messages are then stored in an SQS Queue. When consumers are ready to process new messages they poll them from the queue. Applications, micro-services, and multiple AWS services can take the role of producers or consumers.

### Benefits and features of SQS

1. **Highly scalable Standard and FIFO queues:** Queues scale elastically with your application. Nearly unlimited throughput and no limit to the number of messages per queue in Standard queues. First-In-First-Out delivery and exactly once processing in FIFO queues.
    
2. **Durability and availability:** Your queues are distributed on multiple servers. Redundant infrastructure provides highly concurrent access to messages.
    
3. **Security:** Protection in transit and at rest. Transmit sensitive data in encrypted queues. Send messages in a Virtual Private Cloud.
    
4. **Batching:** Send, receive, or delete messages in batches of up to 10 messages or 256KB to save costs.
    

## What We're Building Today

We're walking through a real-world implementation of an **event-driven Node.js system using Amazon SQS**. You’ll learn how we decoupled a user signup workflow by pushing email notifications to a queue — making our backend more scalable, performant, and resilient.

This guide is based on the project:

%[https://github.com/kartikmehta8/node-aws-sqs-events] 

> This repo demonstrates a simple and scalable system where user registration triggers an asynchronous welcome email using AWS SQS and Nodemailer.

We'll explore:

* A full walkthrough of the codebase structure
    
* How to set up SQS in your AWS account
    
* How to test your implementation under load
    

Let's dive into the **power of queues** and how SQS unlocks a world of scalable, asynchronous architectures.

## The Plot

Sending welcome emails directly during user registration can slow down your app, especially under load. A better approach is to **offload the email task** to a **background worker using AWS SQS** — giving your API a major performance boost.

### 1\. Project Structure

```bash
.
├── config/                   # MongoDB connection.
│   └── db.js
├── controllers/
│   └── authController.js     # Signup & Signin with SQS enqueue.
├── middleware/
│   └── rateLimitMiddleware.js
├── models/
│   └── User.js               # Mongoose schema.
├── routes/
│   └── authRoutes.js
├── services/
│   ├── emailService.js       # (Not shown here) Worker to send actual emails.
│   └── sqsService.js         # AWS SQS integration (Producer)
├── utils/
│   └── validateInput.js
├── worker/
│   └── emailWorker.js        # (Consumer) Polls SQS and sends emails.
├── .env                      # Environment variables.
├── server.js                 # Entry point.
└── package.json
```

This structure separates concerns: auth logic, SQS handling, email delivery, and background processing — all decoupled and scalable. I recommend readers check this out, as the code details are not discussed, the blog is focused on SQS.

### 2\. Environment Variables

Check out the sample `.env` file to see how the environment variables are structured.

```bash
# This file contains all the environment variables that are required by the application.
PORT=5001
JWT_SECRET=kartikmehta
MONGO_URI=mongodb://localhost:27017

# Email
EMAIL_USER=
EMAIL_PASS=

# AWS
AWS_REGION=ap-south-1
AWS_SQS_QUEUE_URL=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
```

[This](https://docs.thedaotool.com/local-setup/mail-setup) is how you can set up `EMAIL_USER` and `EMAIL_PASS`.

## Setting Up AWS SQS

### Create the SQS Queue

1. Go to [https://console.aws.amazon.com/sqs](https://console.aws.amazon.com/sqs)
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1752339742439/66c94dc2-1956-49cf-8dde-e4a3732f06b1.png align="center")

2. Click **Create Queue**
    
3. Choose **Standard**
    
4. Name it `email-queue`
    
5. Leave defaults and create.
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1752339808509/6f1ea5c9-6789-46ba-949c-a47205cbeb2b.png align="center")

The SQS Queue URI can be obtained from the following location:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1752340238270/40042446-0b0b-418a-834e-cc51cd4f30d6.png align="center")

### Create IAM Access Keys

1. Go to **AWS Console > IAM**.
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1742364960291/eb9c174b-1ad9-4b1f-a0a1-19426ffc3bab.png?auto=compress,format&format=webp align="left")

2. On the left-hand side, under **Access Management**, click on **Users**. This is where you can create users and generate their access keys and secrets. Click **Create user**.
    
3. Enter a **username** and click **Next** to proceed.
    
4. In the **Permissions** section, select **Attach policies directly**.
    
    Scroll down to view the available policies. We only need to attach two policies:
    
    * `AmazonSQSFullAccess` (for now)
        
5. Click **Create user**. You have successfully created the user.
    
6. Now, to generate the **Access Key ID** and **Secret Access Key** for this user, select the user you just created. Navigate to the **Security Credentials** tab.
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1742366753723/1c704660-16f9-47f0-987e-b1d1c74cd4e8.png?auto=compress,format&format=webp align="left")

7. Scroll down to the **Access Keys** section and click **Create access key**.
    
8. Select **Command Line Interface (CLI)** as the use case, acknowledge the warning at the bottom, and click **Next** to proceed.
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1742367032639/8c8c9358-9d55-49ce-a423-5421c7e60aa1.png?auto=compress,format&format=webp align="left")

9. Click **Create access key**. You can see your **Access Key ID** and **Secret Access Key**.
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1742367235473/e78fce88-01da-402b-9f74-5bb3dc4b5f25.png?auto=compress,format&format=webp align="left")

Now we have all the environment keys needed to run this project. Let’s go through the code to understand what’s being done.

## Before vs After AWS SQS Integration

Let’s look at **signup logic before and after** integrating SQS.

### Before (Synchronous Email Sending)

```javascript
const nodemailer = require('nodemailer');

const transporter = nodemailer.createTransport({ /* config */ });

const registerUser = async (req, res) => {
  const { name, email, password } = req.body;
  const user = await User.create({ name, email, password });

  await transporter.sendMail({
    from: 'support@example.com',
    to: email,
    subject: 'Welcome!',
    html: `<h1>Hi ${name}</h1>`,
  });

  res.status(201).json({ user });
};
```

**Problems:**

* Email sending delays the response
    
* If SMTP fails, user signup breaks
    
* Harder to retry or monitor delivery failures
    

### After (Async SQS Job Enqueue)

```javascript
// controllers/authController.js
const registerUser = async (req, res) => {
  const { name, email, password } = req.body;
  const user = await User.create({ name, email, password });

  // Send to queue instead of SMTP directly
  await sendToQueue({ email: user.email, name: user.name });

  res.status(201).json({ user });
};
```

and

```javascript
const processMessages = async () => {
  const params = { QueueUrl: QUEUE_URL, MaxNumberOfMessages: 10 };

  const data = await sqs.receiveMessage(params).promise();
  if (data.Messages) {
    for (const message of data.Messages) {
      const { email, name } = JSON.parse(message.Body);
      await sendWelcomeEmail(email, name);
      await sqs.deleteMessage({ QueueUrl: QUEUE_URL, ReceiptHandle: message.ReceiptHandle }).promise();
      console.log(`Message processed: ${message.Body}`);
    }
  }
};

// code...

setInterval(processMessages, 5000);
```

**Result:**

* Signup completes instantly
    
* Email handled **out-of-band**
    
* You can now add metrics, retries, and even scale with AWS Lambda
    

| Benefit | Explanation |
| --- | --- |
| **Fast API Responses** | Signup API doesn’t wait for email to be sent |
| **Reliable Job Handling** | Emails won’t be lost on server crash |
| **Retry Support** | AWS SQS allows retry attempts or dead letter queue |
| **Async Scalability** | Workers scale separately — 1 or 100 workers |
| **Better User Experience** | Emails sent quickly in parallel with user interaction |
| **Clear Audit Trail** | Log every queued job for debugging or tracking |
| **Language/Platform Agnostic** | Multiple services in different stacks can share the same queue |

### TL;DR: Why This Pattern Rocks

* The frontend and API remain **lightweight and blazing fast**
    
* Failures in your SMTP provider **won’t break user signups**
    
* Your system is now **ready for scale**, reliability, and observability
    

## Conclusion

In this tutorial, we implemented AWS SQS to offload email sending from the signup flow, improving speed and reliability. While this was a basic use case, SQS can power much more—like async tasks for report generation, payment processing, syncing services, or triggering workflows. With built-in features like retry policies, dead letter queues, visibility timeouts, and high throughput, AWS SQS ensures fault-tolerant, scalable job handling. By decoupling time-consuming operations from the main app, you keep user interactions fast and systems clean. Queues aren't just for email—they're essential for building modern, event-driven, production-grade backend architecture.

**Time to offload the heavy lifting.**

If you ever need help or just want to chat, **DM me on Twitter / X or LinkedIn.**

[**Kartik Mehta**](https://www.mrmehta.in/)

[***X***](https://x.com/kartik_mehta8) */* [***LinkedIn***](https://www.linkedin.com/in/kartikmehta17)