Feature request: Enhance Parser Utility with JSON String Handling

[garysassano]

Use case

Problem Statement

When working with AWS Lambda through API Gateway or Lambda Function URLs, request bodies are commonly sent as stringified JSON. The current Parser utility doesn't handle this case elegantly, requiring developers to implement manual parsing or workarounds that compromise type safety.

Current Limitations

1. Stringified JSON Handling: When receiving a stringified JSON payload (common in API Gateway and Lambda Function URLs), we need to:

   • Either manually parse the JSON before schema validation
   • Or create complex nested schemas that don't preserve type information effectively

2. Type Inference: When using the parser with API Gateway or Function URL handlers:

   • We lose the automatic response type inference that comes with APIGatewayProxyHandlerV2 or APIGatewayProxyEventV2
   • Need to explicitly specify response types, adding unnecessary boilerplate

Example of Current Issues

// Current approach with manual parsing - API Gateway or Function URL
const handler: APIGatewayProxyHandlerV2 = async (event) => {
  const body = JSON.parse(event.body ?? "{}");
  const filename = body.filename ?? "unknown";
  // ... rest of the handler
};

// Current attempt with parser (suboptimal)
const schema = z.object({
  body: z.string(), // Have to define as string, losing type information
  // ... other event properties
});

const handler = middy()
  .use(parser({ schema }))
  .handler(async (event) => {
    // Need manual parsing inside handler
    const body = JSON.parse(event.body);
    // Types are lost
  });

Solution/User Experience

Proposed Solutions

1. Built-in JSON String Preprocessing

Add a built-in preprocessor for JSON string fields, similar to the approach discussed in zod discussions:

const schema = z.object({
  body: jsonString(z.object({
    filename: z.string()
  }))
});

// Would automatically parse JSON and validate against schema

2. Enhanced Envelope Support

Add specific envelopes for common Lambda HTTP patterns:

const handler = middy()
  .use(parser({ 
    schema: mySchema,
    envelope: HttpJsonBodyEnvelope // New envelope type that works for both API Gateway and Function URLs
  }))
  .handler(async (event) => {
    // event.body is already parsed and typed
  });

Benefits

• Remove the need for manual JSON parsing in handlers
• Single validation step that handles both parsing and schema validation
• Reduce boilerplate code in Lambda functions
• Fewer places where JSON parsing errors need to be handled
• Unified approach for both API Gateway and Lambda Function URLs
• Better developer experience with simplified request handling

Alternative solutions

No response

Acknowledgment

• This feature request meets Powertools for AWS Lambda (TypeScript) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[dreamorosi]

Hi @garysassano, if I understand the request correctly, this is already possible with the current API.

A couple months ago we have added a helper called JSONStringified that does what I think you're describing. In the docs we don't have a code sample for API Gateway specifically, but we do for ALB and SQS, which follow the same idea: a json stringified field that you want to parse and validate.

I have tested the same setup you showed and it works:

Click on the image to enlarge it ⬆️

This is the relevant piece of code:

import { APIGatewayProxyEventV2Schema } from '@aws-lambda-powertools/parser/schemas/api-gatewayv2';
import { JSONStringified } from '@aws-lambda-powertools/parser/helpers';
import { z } from 'zod';

const mySchema = APIGatewayProxyEventV2Schema.extend({
  body: JSONStringified(
    z.object({
      name: z.string(),
      age: z.number(),
    })
  ),
});

and below you have the full code.

import { Logger } from '@aws-lambda-powertools/logger';
import { injectLambdaContext } from '@aws-lambda-powertools/logger/middleware';
import { APIGatewayProxyEventV2Schema } from '@aws-lambda-powertools/parser/schemas/api-gatewayv2';
import { JSONStringified } from '@aws-lambda-powertools/parser/helpers';
import { z } from 'zod';
import middy from '@middy/core';
import { parser } from '@aws-lambda-powertools/parser/middleware';

const logger = new Logger();
const mySchema = APIGatewayProxyEventV2Schema.extend({
  body: JSONStringified(
    z.object({
      name: z.string(),
      age: z.number(),
    })
  ),
});
type MyEvent = z.infer<typeof mySchema>;

export const handler = middy(async (event: MyEvent) => {
  logger.info('body!', { body: event.body });

  return {
    statusCode: 200,
    body: JSON.stringify('Hello, World!'),
  };
})
  .use(injectLambdaContext(logger, { logEvent: true }))
  .use(parser({ schema: mySchema }));

I have opened a PR to add the helper to the API docs and create a standalone section in the Parser docs so that it's more discoverable.

[garysassano]

Your solution works for validating the event schema, but it doens't parse the stringified JSON, so you cannot access the body object properties inside the Lambda handler, unless you parse it manually first.

index.ts

import { Logger } from "@aws-lambda-powertools/logger";
import { injectLambdaContext } from "@aws-lambda-powertools/logger/middleware";
import { JSONStringified } from "@aws-lambda-powertools/parser/helpers";
import { APIGatewayProxyEventV2Schema } from "@aws-lambda-powertools/parser/schemas";
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import middy from "@middy/core";
import { z } from "zod";

const logger = new Logger({ serviceName: "presignLambda" });
const s3Client = new S3Client();
const rekognitionBucketName = process.env.REKOGNITION_BUCKET_NAME;

const presignRequestSchema = APIGatewayProxyEventV2Schema.extend({
  body: JSONStringified(
    z.object({
      filename: z.string().min(1).max(255),
    }),
  ),
});
type PresignRequest = z.infer<typeof presignRequestSchema>;

const lambdaHandler = async (event: PresignRequest) => {
  logger.info("Received event body", { body: event.body });
  const { filename } = event.body;
  logger.info("Extracted filename", { filename });
};

log-1

{
    "cold_start": true,
    "function_arn": "arn:aws:lambda:eu-central-1::function:presign-lambda-c8018142",
    "function_memory_size": "1024",
    "function_name": "presign-lambda-c8018142",
    "function_request_id": "718a6ce2-a1d3-4db6-a2f3-0acb6c7c319e",
    "level": "INFO",
    "message": "Received event body",
    "sampling_rate": 0,
    "service": "presignLambda",
    "timestamp": "2024-10-04T15:56:29.995Z",
    "xray_trace_id": "1-6700102d-293675c61163cffe0698113f",
    "body": "{\"filename\":\"cat.jpg\"}"
}

log-2

{
    "cold_start": true,
    "function_arn": "arn:aws:lambda:eu-central-1::function:presign-lambda-c8018142",
    "function_memory_size": "1024",
    "function_name": "presign-lambda-c8018142",
    "function_request_id": "718a6ce2-a1d3-4db6-a2f3-0acb6c7c319e",
    "level": "INFO",
    "message": "Extracted filename",
    "sampling_rate": 0,
    "service": "presignLambda",
    "timestamp": "2024-10-04T15:56:29.996Z",
    "xray_trace_id": "1-6700102d-293675c61163cffe0698113f"
}

[dreamorosi]

At least in your code sample shared above, you are never using presignRequestSchema anywhere after declaring it, nor parsing the event.

[garysassano]

I confirm the issue was on my side (forgot to add the use for parser on Middy), and now everything works wonderfully.