Skip to content

Commit b44d9a6

Browse files
pgrzesikpgrzesik
committed
feat: Add aws-node-sqs-worker template
1 parent 0f39c6a commit b44d9a6

File tree

7 files changed

+225
-0
lines changed

7 files changed

+225
-0
lines changed

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,7 @@ serverless install -u https://github.com/serverless/examples/tree/master/folder-
8989
| [Aws Node Simple Transcribe S3](https://github.com/serverless/examples/tree/master/aws-node-simple-transcribe-s3) <br/> Example demonstrates how to setup a lambda function to transcribe audio file | nodeJS |
9090
| [Aws Single Page App Via Cloudfront](https://github.com/serverless/examples/tree/master/aws-node-single-page-app-via-cloudfront) <br/> Demonstrating how to deploy a Single Page Application with Serverless | nodeJS |
9191
| [Serverless Single Page App Plugin](https://github.com/serverless/examples/tree/master/aws-node-single-page-app-via-cloudfront/serverless-single-page-app-plugin) <br/> A plugin to simplify deploying Single Page Application using S3 and CloudFront | nodeJS |
92+
| [Aws Node Sqs Worker](https://github.com/serverless/examples/tree/master/aws-node-sqs-worker) <br/> Serverless Framework Node SQS Producer-Consumer on AWS | nodeJS |
9293
| [Aws Node Stripe Integration](https://github.com/serverless/examples/tree/master/aws-node-stripe-integration) <br/> This example for Stripe integration using AWS Lambda and API Gateway. | nodeJS |
9394
| [Aws Node Line Echo Bot](https://github.com/serverless/examples/tree/master/aws-node-telegram-echo-bot) <br/> This is a simple echo bot on Telegram | nodeJS |
9495
| [Aws Text Analysis Via Sns Post Processing](https://github.com/serverless/examples/tree/master/aws-node-text-analysis-via-sns-post-processing) <br/> Example demonstrates how to setup a simple data processing pipeline | nodeJS |

‎aws-node-sqs-worker/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
node_modules
2+
.serverless

‎aws-node-sqs-worker/README.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
<!--
2+
title: 'Serverless Framework Node SQS Producer-Consumer on AWS'
3+
description: 'This template demonstrates how to develop and deploy a simple SQS-based producer-consumer service running on AWS Lambda using the traditional Serverless Framework.'
4+
layout: Doc
5+
framework: v2
6+
platform: AWS
7+
language: nodeJS
8+
authorLink: 'https://github.com/serverless'
9+
authorName: 'Serverless, inc.'
10+
authorAvatar: 'https://avatars1.githubusercontent.com/u/13742415?s=200&v=4'
11+
-->
12+
13+
# Serverless Framework Node SQS Producer-Consumer on AWS
14+
15+
This template demonstrates how to develop and deploy a simple SQS-based producer-consumer service running on AWS Lambda using the traditional Serverless Framework. It allows to accept messages, for which computation might be time or resource intensive, and offload their processing to an asynchronous background process for a faster and more resilient system.
16+
17+
## Anatomy of the template
18+
19+
This template defines two functions, `producer` and `consumer`. First of them, `producer`, is triggered by `http` event type, accepts JSON payload and sends it to a corresponding SQS queue for further processing. To learn more about `http` event configuration options, please refer to [http event docs](https://www.serverless.com/framework/docs/providers/aws/events/apigateway/). Second function, `consumer`, is responsible for processing messages from SQS queue thanks to its `sqs` trigger definition. To learn more about `sqs` event configuration options, please refer to [sqs event docs](https://www.serverless.com/framework/docs/providers/aws/events/sqs/). Additionally, the template takes care of provisioning underlying SQS queue along with corresponding SQS dead-letter queue, which are defined in `resources` section. The dead-letter queue is defined in order to prevent processing invalid messages over and over. In our case, if message is delivered to the source queue more than 5 times, it will be moved to dead-letter queue. For more details, please refer to official [AWS documentation](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html). To learn more about `resources`, please refer to [our docs](https://www.serverless.com/framework/docs/providers/aws/guide/resources/).
20+
21+
## Usage
22+
23+
### Deployment
24+
25+
This example is made to work with the Serverless Framework dashboard, which includes advanced features such as CI/CD, monitoring, metrics, etc.
26+
27+
In order to deploy with dashboard, you need to first login with:
28+
29+
```
30+
serverless login
31+
```
32+
33+
and then perform deployment with:
34+
35+
```
36+
serverless deploy
37+
```
38+
39+
After running deploy, you should see output similar to:
40+
41+
```bash
42+
Serverless: Packaging service...
43+
Serverless: Excluding development dependencies...
44+
Serverless: Creating Stack...
45+
Serverless: Checking Stack create progress...
46+
........
47+
Serverless: Stack create finished...
48+
Serverless: Uploading CloudFormation file to S3...
49+
Serverless: Uploading artifacts...
50+
Serverless: Uploading service aws-node-sqs-worker.zip file to S3 (1.04 KB)...
51+
Serverless: Validating template...
52+
Serverless: Updating Stack...
53+
Serverless: Checking Stack update progress...
54+
................................................
55+
Serverless: Stack update finished...
56+
Service Information
57+
service: aws-node-sqs-worker
58+
stage: dev
59+
region: us-east-1
60+
stack: aws-node-sqs-worker-dev
61+
resources: 17
62+
api keys:
63+
None
64+
endpoints:
65+
POST - https://xxxxxxx.execute-api.us-east-1.amazonaws.com/dev/produce
66+
functions:
67+
producer: aws-node-sqs-worker-dev-producer
68+
consumer: aws-node-sqs-worker-dev-consumer
69+
layers:
70+
None
71+
```
72+
73+
_Note_: In current form, after deployment, your API is public and can be invoked by anyone. For production deployments, you might want to configure an authorizer. For details on how to do that, refer to [http event docs](https://www.serverless.com/framework/docs/providers/aws/events/apigateway/).
74+
75+
### Invocation
76+
77+
After successful deployment, you can now call the created API endpoint with `POST` request to invoke `producer` function:
78+
79+
```bash
80+
curl --request POST 'https://xxxxxxx.execute-api.us-east-1.amazonaws.com/dev/produce' --header 'Content-Type: application/json' --data-raw '{"name": "John"}'
81+
```
82+
83+
In response, you should see output similar to:
84+
85+
```bash
86+
{"message": "Message accepted!"}
87+
```

‎aws-node-sqs-worker/handler.js‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
const { SQS } = require("aws-sdk");
2+
3+
const sqs = new SQS();
4+
5+
const producer = async (event) => {
6+
let statusCode = 200;
7+
let message;
8+
9+
if (!event.body) {
10+
return {
11+
statusCode: 400,
12+
body: JSON.stringify({
13+
message: "No body was found",
14+
}),
15+
};
16+
}
17+
18+
try {
19+
await sqs
20+
.sendMessage({
21+
QueueUrl: process.env.QUEUE_URL,
22+
MessageBody: event.body,
23+
MessageAttributes: {
24+
AttributeName: {
25+
StringValue: "Attribute Value",
26+
DataType: "String",
27+
},
28+
},
29+
})
30+
.promise();
31+
32+
message = "Message accepted!";
33+
} catch (error) {
34+
console.log(error);
35+
message = error;
36+
statusCode = 500;
37+
}
38+
39+
return {
40+
statusCode,
41+
body: JSON.stringify({
42+
message,
43+
}),
44+
};
45+
};
46+
47+
const consumer = async (event) => {
48+
try {
49+
for (const record of event.Records) {
50+
const messageAttributes = record.messageAttributes;
51+
console.log(
52+
"Message Attributte: ",
53+
messageAttributes.AttributeName.stringValue
54+
);
55+
console.log("Message Body: ", record.body);
56+
}
57+
} catch (error) {
58+
console.log(error);
59+
}
60+
};
61+
62+
module.exports = {
63+
producer,
64+
consumer,
65+
};

‎aws-node-sqs-worker/package.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"name": "aws-node-sqs-worker",
3+
"version": "1.0.0",
4+
"description": "Serverless Framework Node SQS Producer-Consumer on AWS",
5+
"author": "",
6+
"license": "MIT"
7+
}

‎aws-node-sqs-worker/serverless.template.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
name: aws-node-sqs-worker
2+
org: serverlessinc
3+
description: Deploys a Node SQS Producer-Consumer service with traditional Serverless Framework
4+
keywords: aws, serverless, faas, lambda, node, sqs
5+
repo: https://github.com/serverless/examples/aws-node-sqs-worker
6+
license: MIT

‎aws-node-sqs-worker/serverless.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
service: aws-node-sqs-worker
2+
3+
frameworkVersion: '2'
4+
variablesResolutionMode: '20210219'
5+
configValidationMode: 'error'
6+
7+
8+
provider:
9+
name: aws
10+
runtime: nodejs12.x
11+
lambdaHashingVersion: '20201221'
12+
apiGateway:
13+
shouldStartNameWithService: true
14+
iam:
15+
role:
16+
statements:
17+
- Effect: Allow
18+
Action:
19+
- sqs:SendMessage
20+
Resource:
21+
- Fn::GetAtt: [ WorkerQueue, Arn ]
22+
23+
functions:
24+
producer:
25+
handler: handler.producer
26+
events:
27+
- http:
28+
method: post
29+
path: produce
30+
environment:
31+
QUEUE_URL: !Ref WorkerQueue
32+
33+
consumer:
34+
handler: handler.consumer
35+
events:
36+
- sqs:
37+
arn:
38+
Fn::GetAtt:
39+
- WorkerQueue
40+
- Arn
41+
42+
resources:
43+
Resources:
44+
WorkerQueue:
45+
Type: AWS::SQS::Queue
46+
Properties:
47+
QueueName: workerQueue
48+
RedrivePolicy:
49+
deadLetterTargetArn:
50+
Fn::GetAtt:
51+
- WorkerDLQ
52+
- Arn
53+
maxReceiveCount: 5
54+
WorkerDLQ:
55+
Type: AWS::SQS::Queue
56+
Properties:
57+
QueueName: workerDLQ

0 commit comments

Comments
 (0)