diff --git a/.github/workflows/publish-artifacts-examples-tests.yml b/.github/workflows/publish-artifacts-examples-tests.yml index 5eb4e1d24..d9c76a9a2 100644 --- a/.github/workflows/publish-artifacts-examples-tests.yml +++ b/.github/workflows/publish-artifacts-examples-tests.yml @@ -133,6 +133,7 @@ jobs: --name github \ --username ${{ github.actor }} \ --password ${{ secrets.GITHUB_TOKEN }} + --store-password-in-clear-text - name: Publish packages to GitHub Packages run: | diff --git a/docs/core/logging-v1.md b/docs/core/logging-v1.md new file mode 100644 index 000000000..ba06f8e39 --- /dev/null +++ b/docs/core/logging-v1.md @@ -0,0 +1,808 @@ +--- +title: Logging v1 - Legacy +description: Core utility +--- + +!!! warning + Version 1.x.x will continue to be supported until **end of July 2025** for critical bug fixes and security updates in very exceptional cases where you cannot update to v2, but no new features will be added to this version. + + We recommend you upgrade to the latest version. + + The latest version is available at [Logging v2](https://docs.powertools.aws.dev/lambda/dotnet/core/logging-v2/). + + +The logging utility provides a Lambda optimized logger with output structured as JSON. + + +## Key features + +* Capture key fields from Lambda context, cold start and structures logging output as JSON +* Log Lambda event when instructed (disabled by default) +* Log sampling enables DEBUG log level for a percentage of requests (disabled by default) +* Append additional keys to structured log at any point in time +* Ahead-of-Time compilation to native code support [AOT](https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html) from version 1.6.0 + +## Installation + +Powertools for AWS Lambda (.NET) are available as NuGet packages. You can install the packages from [NuGet Gallery](https://www.nuget.org/packages?q=AWS+Lambda+Powertools*){target="_blank"} or from Visual Studio editor by searching `AWS.Lambda.Powertools*` to see various utilities available. + +* [AWS.Lambda.Powertools.Logging](https://www.nuget.org/packages?q=AWS.Lambda.Powertools.Logging): + + `dotnet add package AWS.Lambda.Powertools.Logging --version 1.6.5` + +## Getting started + +!!! info + + AOT Support + If loooking for AOT specific configurations navigate to the [AOT section](#aot-support) + + +Logging requires two settings: + +Setting | Description | Environment variable | Attribute parameter +------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- +**Service** | Sets **Service** key that will be present across all log statements | `POWERTOOLS_SERVICE_NAME` | `Service` +**Logging level** | Sets how verbose Logger should be (Information, by default) | `POWERTOOLS_LOG_LEVEL` | `LogLevel` + +### Service Property Priority Resolution + +The root level Service property now correctly follows this priority order: + +1. LoggingAttribute.Service (property value set in the decorator) +2. POWERTOOLS_SERVICE_NAME (environment variable) + + +### Example using AWS Serverless Application Model (AWS SAM) + +You can override log level by setting **`POWERTOOLS_LOG_LEVEL`** environment variable in the AWS SAM template. + +You can also explicitly set a service name via **`POWERTOOLS_SERVICE_NAME`** environment variable. This sets **Service** key that will be present across all log statements. + +Here is an example using the AWS SAM [Globals section](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-specification-template-anatomy-globals.html). + +=== "template.yaml" + + ```yaml hl_lines="13 14" + # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + # SPDX-License-Identifier: MIT-0 + AWSTemplateFormatVersion: "2010-09-09" + Transform: AWS::Serverless-2016-10-31 + Description: > + Example project for Powertools for AWS Lambda (.NET) Logging utility + + Globals: + Function: + Timeout: 10 + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: powertools-dotnet-logging-sample + POWERTOOLS_LOG_LEVEL: Debug + POWERTOOLS_LOGGER_LOG_EVENT: true + POWERTOOLS_LOGGER_CASE: PascalCase # Allowed values are: CamelCase, PascalCase and SnakeCase + POWERTOOLS_LOGGER_SAMPLE_RATE: 0 + ``` + +### Full list of environment variables + +| Environment variable | Description | Default | +| ------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- | +| **POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | `"service_undefined"` | +| **POWERTOOLS_LOG_LEVEL** | Sets logging level | `Information` | +| **POWERTOOLS_LOGGER_CASE** | Override the default casing for log keys | `SnakeCase` | +| **POWERTOOLS_LOGGER_LOG_EVENT** | Logs incoming event | `false` | +| **POWERTOOLS_LOGGER_SAMPLE_RATE** | Debug log sampling | `0` | + + +### Using AWS Lambda Advanced Logging Controls (ALC) + +!!! question "When is it useful?" + When you want to set a logging policy to drop informational or verbose logs for one or all AWS Lambda functions, regardless of runtime and logger used. + +With [AWS Lambda Advanced Logging Controls (ALC)](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-advanced){target="_blank"}, you can enforce a minimum log level that Lambda will accept from your application code. + +When enabled, you should keep `Logger` and ALC log level in sync to avoid data loss. + +!!! warning "When using AWS Lambda Advanced Logging Controls (ALC)" + - When Powertools Logger output is set to `PascalCase` **`Level`** property name will be replaced by **`LogLevel`** as a property name. + - ALC takes precedence over **`POWERTOOLS_LOG_LEVEL`** and when setting it in code using **`[Logging(LogLevel = )]`** + +Here's a sequence diagram to demonstrate how ALC will drop both `Information` and `Debug` logs emitted from `Logger`, when ALC log level is stricter than `Logger`. + +```mermaid +sequenceDiagram + title Lambda ALC allows WARN logs only + participant Lambda service + participant Lambda function + participant Application Logger + + Note over Lambda service: AWS_LAMBDA_LOG_LEVEL="WARN" + Note over Application Logger: POWERTOOLS_LOG_LEVEL="DEBUG" + Lambda service->>Lambda function: Invoke (event) + Lambda function->>Lambda function: Calls handler + Lambda function->>Application Logger: Logger.Warning("Something happened") + Lambda function-->>Application Logger: Logger.Debug("Something happened") + Lambda function-->>Application Logger: Logger.Information("Something happened") + + Lambda service->>Lambda service: DROP INFO and DEBUG logs + + Lambda service->>CloudWatch Logs: Ingest error logs +``` + +**Priority of log level settings in Powertools for AWS Lambda** + +We prioritise log level settings in this order: + +1. AWS_LAMBDA_LOG_LEVEL environment variable +2. Setting the log level in code using `[Logging(LogLevel = )]` +3. POWERTOOLS_LOG_LEVEL environment variable + +If you set `Logger` level lower than ALC, we will emit a warning informing you that your messages will be discarded by Lambda. + +> **NOTE** +> With ALC enabled, we are unable to increase the minimum log level below the `AWS_LAMBDA_LOG_LEVEL` environment variable value, see [AWS Lambda service documentation](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-log-level){target="_blank"} for more details. + +## Standard structured keys + +Your logs will always include the following keys to your structured logging: + +Key | Type | Example | Description +------------------------------------------------- | ------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- +**Timestamp** | string | "2020-05-24 18:17:33,774" | Timestamp of actual log statement +**Level** | string | "Information" | Logging level +**Name** | string | "Powertools for AWS Lambda (.NET) Logger" | Logger name +**ColdStart** | bool | true| ColdStart value. +**Service** | string | "payment" | Service name defined. "service_undefined" will be used if unknown +**SamplingRate** | int | 0.1 | Debug logging sampling rate in percentage e.g. 10% in this case +**Message** | string | "Collecting payment" | Log statement value. Unserializable JSON values will be cast to string +**FunctionName**| string | "example-powertools-HelloWorldFunction-1P1Z6B39FLU73" +**FunctionVersion**| string | "12" +**FunctionMemorySize**| string | "128" +**FunctionArn**| string | "arn:aws:lambda:eu-west-1:012345678910:function:example-powertools-HelloWorldFunction-1P1Z6B39FLU73" +**XRayTraceId**| string | "1-5759e988-bd862e3fe1be46a994272793" | X-Ray Trace ID when Lambda function has enabled Tracing +**FunctionRequestId**| string | "899856cb-83d1-40d7-8611-9e78f15f32f4" | AWS Request ID from lambda context + +## Logging incoming event + +When debugging in non-production environments, you can instruct Logger to log the incoming event with `LogEvent` parameter or via `POWERTOOLS_LOGGER_LOG_EVENT` environment variable. + +!!! warning + Log event is disabled by default to prevent sensitive info being logged. + +=== "Function.cs" + + ```c# hl_lines="6" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(LogEvent = true)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + } + } + ``` + +## Setting a Correlation ID + +You can set a Correlation ID using `CorrelationIdPath` parameter by passing a [JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03){target="_blank"}. + +!!! Attention + The JSON Pointer expression is `case sensitive`. In the bellow example `/headers/my_request_id_header` would work but `/Headers/my_request_id_header` would not find the element. + + +=== "Function.cs" + + ```c# hl_lines="6" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(CorrelationIdPath = "/headers/my_request_id_header")] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + } + } + ``` +=== "Example Event" + + ```json hl_lines="3" + { + "headers": { + "my_request_id_header": "correlation_id_value" + } + } + ``` + +=== "Example CloudWatch Logs excerpt" + + ```json hl_lines="15" + { + "cold_start": true, + "xray_trace_id": "1-61b7add4-66532bb81441e1b060389429", + "function_name": "test", + "function_version": "$LATEST", + "function_memory_size": 128, + "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", + "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72", + "timestamp": "2021-12-13T20:32:22.5774262Z", + "level": "Information", + "service": "lambda-example", + "name": "AWS.Lambda.Powertools.Logging.Logger", + "message": "Collecting payment", + "sampling_rate": 0.7, + "correlation_id": "correlation_id_value", + } + ``` +We provide [built-in JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03){target="_blank"} +for known event sources, where either a request ID or X-Ray Trace ID are present. + +=== "Function.cs" + + ```c# hl_lines="6" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(CorrelationIdPath = CorrelationIdPaths.ApiGatewayRest)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + } + } + ``` + +=== "Example Event" + + ```json hl_lines="3" + { + "RequestContext": { + "RequestId": "correlation_id_value" + } + } + ``` + +=== "Example CloudWatch Logs excerpt" + + ```json hl_lines="15" + { + "cold_start": true, + "xray_trace_id": "1-61b7add4-66532bb81441e1b060389429", + "function_name": "test", + "function_version": "$LATEST", + "function_memory_size": 128, + "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", + "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72", + "timestamp": "2021-12-13T20:32:22.5774262Z", + "level": "Information", + "service": "lambda-example", + "name": "AWS.Lambda.Powertools.Logging.Logger", + "message": "Collecting payment", + "sampling_rate": 0.7, + "correlation_id": "correlation_id_value", + } + ``` + +## Appending additional keys + +!!! info "Custom keys are persisted across warm invocations" + Always set additional keys as part of your handler to ensure they have the latest value, or explicitly clear them with [`ClearState=true`](#clearing-all-state). + +You can append your own keys to your existing logs via `AppendKey`. Typically this value would be passed into the function via the event. Appended keys are added to all subsequent log entries in the current execution from the point the logger method is called. To ensure the key is added to all log entries, call this method as early as possible in the Lambda handler. + +=== "Function.cs" + + ```c# hl_lines="21" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(LogEvent = true)] + public async Task FunctionHandler(APIGatewayProxyRequest apigwProxyEvent, + ILambdaContext context) + { + var requestContextRequestId = apigwProxyEvent.RequestContext.RequestId; + + var lookupInfo = new Dictionary() + { + {"LookupInfo", new Dictionary{{ "LookupId", requestContextRequestId }}} + }; + + // Appended keys are added to all subsequent log entries in the current execution. + // Call this method as early as possible in the Lambda handler. + // Typically this is value would be passed into the function via the event. + // Set the ClearState = true to force the removal of keys across invocations, + Logger.AppendKeys(lookupInfo); + + Logger.LogInformation("Getting ip address from external service"); + + } + ``` +=== "Example CloudWatch Logs excerpt" + + ```json hl_lines="4 5 6" + { + "cold_start": false, + "xray_trace_id": "1-622eede0-647960c56a91f3b071a9fff1", + "lookup_info": { + "lookup_id": "4c50eace-8b1e-43d3-92ba-0efacf5d1625" + }, + "function_name": "PowertoolsLoggingSample-HelloWorldFunction-hm1r10VT3lCy", + "function_version": "$LATEST", + "function_memory_size": 256, + "function_arn": "arn:aws:lambda:ap-southeast-2:538510314095:function:PowertoolsLoggingSample-HelloWorldFunction-hm1r10VT3lCy", + "function_request_id": "96570b2c-f00e-471c-94ad-b25e95ba7347", + "timestamp": "2022-03-14T07:25:20.9418065Z", + "level": "Information", + "service": "powertools-dotnet-logging-sample", + "name": "AWS.Lambda.Powertools.Logging.Logger", + "message": "Getting ip address from external service" + } + ``` + +### Removing additional keys + +You can remove any additional key from entry using `Logger.RemoveKeys()`. + +=== "Function.cs" + + ```c# hl_lines="21 22" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(LogEvent = true)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + Logger.AppendKey("test", "willBeLogged"); + ... + var customKeys = new Dictionary + { + {"test1", "value1"}, + {"test2", "value2"} + }; + + Logger.AppendKeys(customKeys); + ... + Logger.RemoveKeys("test"); + Logger.RemoveKeys("test1", "test2"); + ... + } + } + ``` + +## Extra Keys + +Extra keys allow you to append additional keys to a log entry. Unlike `AppendKey`, extra keys will only apply to the current log entry. + +Extra keys argument is available for all log levels' methods, as implemented in the standard logging library - e.g. Logger.Information, Logger.Warning. + +It accepts any dictionary, and all keyword arguments will be added as part of the root structure of the logs for that log statement. + +!!! info + Any keyword argument added using extra keys will not be persisted for subsequent messages. + +=== "Function.cs" + + ```c# hl_lines="16" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(LogEvent = true)] + public async Task FunctionHandler(APIGatewayProxyRequest apigwProxyEvent, + ILambdaContext context) + { + var requestContextRequestId = apigwProxyEvent.RequestContext.RequestId; + + var lookupId = new Dictionary() + { + { "LookupId", requestContextRequestId } + }; + + // Appended keys are added to all subsequent log entries in the current execution. + // Call this method as early as possible in the Lambda handler. + // Typically this is value would be passed into the function via the event. + // Set the ClearState = true to force the removal of keys across invocations, + Logger.AppendKeys(lookupId); + } + ``` + +### Clearing all state + +Logger is commonly initialized in the global scope. Due to [Lambda Execution Context reuse](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html), this means that custom keys can be persisted across invocations. If you want all custom keys to be deleted, you can use `ClearState=true` attribute on `[Logging]` attribute. + +=== "Function.cs" + + ```cs hl_lines="6 13" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(ClearState = true)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + if (apigProxyEvent.Headers.ContainsKey("SomeSpecialHeader")) + { + Logger.AppendKey("SpecialKey", "value"); + } + + Logger.LogInformation("Collecting payment"); + ... + } + } + ``` +=== "#1 Request" + + ```json hl_lines="11" + { + "level": "Information", + "message": "Collecting payment", + "timestamp": "2021-12-13T20:32:22.5774262Z", + "service": "payment", + "cold_start": true, + "function_name": "test", + "function_memory_size": 128, + "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", + "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72", + "special_key": "value" + } + ``` + +=== "#2 Request" + + ```json + { + "level": "Information", + "message": "Collecting payment", + "timestamp": "2021-12-13T20:32:22.5774262Z", + "service": "payment", + "cold_start": true, + "function_name": "test", + "function_memory_size": 128, + "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", + "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72" + } + ``` + +## Sampling debug logs + +You can dynamically set a percentage of your logs to **DEBUG** level via env var `POWERTOOLS_LOGGER_SAMPLE_RATE` or +via `SamplingRate` parameter on attribute. + +!!! info + Configuration on environment variable is given precedence over sampling rate configuration on attribute, provided it's in valid value range. + +=== "Sampling via attribute parameter" + + ```c# hl_lines="6" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(SamplingRate = 0.5)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + } + } + ``` + +=== "Sampling via environment variable" + + ```yaml hl_lines="8" + + Resources: + HelloWorldFunction: + Type: AWS::Serverless::Function + Properties: + ... + Environment: + Variables: + POWERTOOLS_LOGGER_SAMPLE_RATE: 0.5 + ``` + +## Configure Log Output Casing + +By definition Powertools for AWS Lambda (.NET) outputs logging keys using **snake case** (e.g. *"function_memory_size": 128*). This allows developers using different Powertools for AWS Lambda (.NET) runtimes, to search logs across services written in languages such as Python or TypeScript. + +If you want to override the default behavior you can either set the desired casing through attributes, as described in the example below, or by setting the `POWERTOOLS_LOGGER_CASE` environment variable on your AWS Lambda function. Allowed values are: `CamelCase`, `PascalCase` and `SnakeCase`. + +=== "Output casing via attribute parameter" + + ```c# hl_lines="6" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(LoggerOutputCase = LoggerOutputCase.CamelCase)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + } + } + ``` + +Below are some output examples for different casing. + +=== "Camel Case" + + ```json + { + "level": "Information", + "message": "Collecting payment", + "timestamp": "2021-12-13T20:32:22.5774262Z", + "service": "payment", + "coldStart": true, + "functionName": "test", + "functionMemorySize": 128, + "functionArn": "arn:aws:lambda:eu-west-1:12345678910:function:test", + "functionRequestId": "52fdfc07-2182-154f-163f-5f0f9a621d72" + } + ``` + +=== "Pascal Case" + + ```json + { + "Level": "Information", + "Message": "Collecting payment", + "Timestamp": "2021-12-13T20:32:22.5774262Z", + "Service": "payment", + "ColdStart": true, + "FunctionName": "test", + "FunctionMemorySize": 128, + "FunctionArn": "arn:aws:lambda:eu-west-1:12345678910:function:test", + "FunctionRequestId": "52fdfc07-2182-154f-163f-5f0f9a621d72" + } + ``` + +=== "Snake Case" + + ```json + { + "level": "Information", + "message": "Collecting payment", + "timestamp": "2021-12-13T20:32:22.5774262Z", + "service": "payment", + "cold_start": true, + "function_name": "test", + "function_memory_size": 128, + "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", + "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72" + } + ``` + +## Custom Log formatter (Bring Your Own Formatter) + +You can customize the structure (keys and values) of your log entries by implementing a custom log formatter and override default log formatter using ``Logger.UseFormatter`` method. You can implement a custom log formatter by inheriting the ``ILogFormatter`` class and implementing the ``object FormatLogEntry(LogEntry logEntry)`` method. + +=== "Function.cs" + + ```c# hl_lines="11" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + /// + /// Function constructor + /// + public Function() + { + Logger.UseFormatter(new CustomLogFormatter()); + } + + [Logging(CorrelationIdPath = "/headers/my_request_id_header", SamplingRate = 0.7)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + } + } + ``` +=== "CustomLogFormatter.cs" + + ```c# + public class CustomLogFormatter : ILogFormatter + { + public object FormatLogEntry(LogEntry logEntry) + { + return new + { + Message = logEntry.Message, + Service = logEntry.Service, + CorrelationIds = new + { + AwsRequestId = logEntry.LambdaContext?.AwsRequestId, + XRayTraceId = logEntry.XRayTraceId, + CorrelationId = logEntry.CorrelationId + }, + LambdaFunction = new + { + Name = logEntry.LambdaContext?.FunctionName, + Arn = logEntry.LambdaContext?.InvokedFunctionArn, + MemoryLimitInMB = logEntry.LambdaContext?.MemoryLimitInMB, + Version = logEntry.LambdaContext?.FunctionVersion, + ColdStart = logEntry.ColdStart, + }, + Level = logEntry.Level.ToString(), + Timestamp = logEntry.Timestamp.ToString("o"), + Logger = new + { + Name = logEntry.Name, + SampleRate = logEntry.SamplingRate + }, + }; + } + } + ``` + +=== "Example CloudWatch Logs excerpt" + + ```json + { + "Message": "Test Message", + "Service": "lambda-example", + "CorrelationIds": { + "AwsRequestId": "52fdfc07-2182-154f-163f-5f0f9a621d72", + "XRayTraceId": "1-61b7add4-66532bb81441e1b060389429", + "CorrelationId": "correlation_id_value" + }, + "LambdaFunction": { + "Name": "test", + "Arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", + "MemorySize": 128, + "Version": "$LATEST", + "ColdStart": true + }, + "Level": "Information", + "Timestamp": "2021-12-13T20:32:22.5774262Z", + "Logger": { + "Name": "AWS.Lambda.Powertools.Logging.Logger", + "SampleRate": 0.7 + } + } + ``` + +## AOT Support + +!!! info + + If you want to use the `LogEvent`, `Custom Log Formatter` features, or serialize your own types when Logging events, you need to make changes in your Lambda `Main` method. + +!!! info + + Starting from version 1.6.0, it is required to update the Amazon.Lambda.Serialization.SystemTextJson NuGet package to version 2.4.3 in your csproj. + +### Configure + +Replace `SourceGeneratorLambdaJsonSerializer` with `PowertoolsSourceGeneratorSerializer`. + +This change enables Powertools to construct an instance of `JsonSerializerOptions` used to customize the serialization and deserialization of Lambda JSON events and your own types. + +=== "Before" + + ```csharp + Func> handler = FunctionHandler; + await LambdaBootstrapBuilder.Create(handler, new SourceGeneratorLambdaJsonSerializer()) + .Build() + .RunAsync(); + ``` + +=== "After" + + ```csharp hl_lines="2" + Func> handler = FunctionHandler; + await LambdaBootstrapBuilder.Create(handler, new PowertoolsSourceGeneratorSerializer()) + .Build() + .RunAsync(); + ``` + +For example when you have your own Demo type + +```csharp +public class Demo +{ + public string Name { get; set; } + public Headers Headers { get; set; } +} +``` + +To be able to serialize it in AOT you have to have your own `JsonSerializerContext` + +```csharp +[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))] +[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))] +[JsonSerializable(typeof(Demo))] +public partial class MyCustomJsonSerializerContext : JsonSerializerContext +{ +} +``` + +When you update your code to use `PowertoolsSourceGeneratorSerializer`, we combine your `JsonSerializerContext` with Powertools' `JsonSerializerContext`. This allows Powertools to serialize your types and Lambda events. + +### Custom Log Formatter + +To use a custom log formatter with AOT, pass an instance of `ILogFormatter` to `PowertoolsSourceGeneratorSerializer` instead of using the static `Logger.UseFormatter` in the Function constructor as you do in non-AOT Lambdas. + +=== "Function Main method" + + ```csharp hl_lines="5" + + Func> handler = FunctionHandler; + await LambdaBootstrapBuilder.Create(handler, + new PowertoolsSourceGeneratorSerializer + ( + new CustomLogFormatter() + ) + ) + .Build() + .RunAsync(); + + ``` + +=== "CustomLogFormatter.cs" + + ```csharp + public class CustomLogFormatter : ILogFormatter + { + public object FormatLogEntry(LogEntry logEntry) + { + return new + { + Message = logEntry.Message, + Service = logEntry.Service, + CorrelationIds = new + { + AwsRequestId = logEntry.LambdaContext?.AwsRequestId, + XRayTraceId = logEntry.XRayTraceId, + CorrelationId = logEntry.CorrelationId + }, + LambdaFunction = new + { + Name = logEntry.LambdaContext?.FunctionName, + Arn = logEntry.LambdaContext?.InvokedFunctionArn, + MemoryLimitInMB = logEntry.LambdaContext?.MemoryLimitInMB, + Version = logEntry.LambdaContext?.FunctionVersion, + ColdStart = logEntry.ColdStart, + }, + Level = logEntry.Level.ToString(), + Timestamp = logEntry.Timestamp.ToString("o"), + Logger = new + { + Name = logEntry.Name, + SampleRate = logEntry.SamplingRate + }, + }; + } + } + ``` + +### Anonymous types + +!!! note + + While we support anonymous type serialization by converting to a `Dictionary`, this is **not** a best practice and is **not recommended** when using native AOT. + + We recommend using concrete classes and adding them to your `JsonSerializerContext`. diff --git a/docs/core/logging-v2.md b/docs/core/logging-v2.md deleted file mode 100644 index 93b6d51b3..000000000 --- a/docs/core/logging-v2.md +++ /dev/null @@ -1,1529 +0,0 @@ ---- -title: Logging V2 -description: Core utility ---- - -The logging utility provides a Lambda optimized logger with output structured as JSON. - -## Key features - -* Capture key fields from Lambda context, cold start and structures logging output as JSON -* Log Lambda event when instructed (disabled by default) -* Log sampling enables DEBUG log level for a percentage of requests (disabled by default) -* Append additional keys to structured log at any point in time -* Ahead-of-Time compilation to native code - support [AOT](https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html) -* Custom log formatter to override default log structure -* Support - for [AWS Lambda Advanced Logging Controls (ALC)](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs-advanced.html) - {target="_blank"} -* Support for Microsoft.Extensions.Logging - and [ILogger](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.logging.ilogger?view=dotnet-plat-ext-7.0) - interface -* Support - for [ILoggerFactory](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.logging.iloggerfactory?view=dotnet-plat-ext-7.0) - interface -* Support for message templates `{}` and `{@}` for structured logging - -## Installation - -Powertools for AWS Lambda (.NET) are available as NuGet packages. You can install the packages -from [NuGet Gallery](https://www.nuget.org/packages?q=AWS+Lambda+Powertools*){target="_blank"} or from Visual Studio -editor by searching `AWS.Lambda.Powertools*` to see various utilities available. - -* [AWS.Lambda.Powertools.Logging](https://www.nuget.org/packages?q=AWS.Lambda.Powertools.Logging): - - `dotnet add package AWS.Lambda.Powertools.Logging` - -## Getting started - -!!! info - - AOT Support - If loooking for AOT specific configurations navigate to the [AOT section](#aot-support) - -Logging requires two settings: - - Setting | Description | Environment variable | Attribute parameter --------------------|---------------------------------------------------------------------|---------------------------|--------------------- - **Service** | Sets **Service** key that will be present across all log statements | `POWERTOOLS_SERVICE_NAME` | `Service` - **Logging level** | Sets how verbose Logger should be (Information, by default) | `POWERTOOLS_LOG_LEVEL` | `LogLevel` - -### Full list of environment variables - -| Environment variable | Description | Default | -|-----------------------------------|----------------------------------------------------------------------------------------|-----------------------| -| **POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | `"service_undefined"` | -| **POWERTOOLS_LOG_LEVEL** | Sets logging level | `Information` | -| **POWERTOOLS_LOGGER_CASE** | Override the default casing for log keys | `SnakeCase` | -| **POWERTOOLS_LOGGER_LOG_EVENT** | Logs incoming event | `false` | -| **POWERTOOLS_LOGGER_SAMPLE_RATE** | Debug log sampling | `0` | - -### Setting up the logger - -You can set up the logger in different ways. The most common way is to use the `Logging` attribute on your Lambda. -You can also use the `ILogger` interface to log messages. This interface is part of the Microsoft.Extensions.Logging. - -=== "Using decorator" - - ```c# hl_lines="6 10" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(Service = "payment", LogLevel = LogLevel.Debug)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Logger.LogInformation("Collecting payment"); - ... - } - } - ``` - -=== "Logger Factory" - - ```c# hl_lines="6 10-17 23" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - private readonly ILogger _logger; - - public Function(ILoggerFactory loggerFactory) - { - _logger = loggerFactory.Create(builder => - { - builder.AddPowertoolsLogger(config => - { - config.Service = "TestService"; - config.LoggerOutputCase = LoggerOutputCase.PascalCase; - }); - }).CreatePowertoolsLogger(); - } - - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - _logger.LogInformation("Collecting payment"); - ... - } - } - ``` - -=== "With Builder" - - ```c# hl_lines="6 10-13 19" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - private readonly ILogger _logger; - - public Function(ILogger logger) - { - _logger = logger ?? new PowertoolsLoggerBuilder() - .WithService("TestService") - .WithOutputCase(LoggerOutputCase.PascalCase) - .Build(); - } - - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - _logger.LogInformation("Collecting payment"); - ... - } - } - ``` - -### Customizing the logger - -You can customize the logger by setting the following properties in the `Logger.Configure` method: - -| Property | Description | -|:----------------------|--------------------------------------------------------------------------------------------------| -| `Service` | The name of the service. This is used to identify the service in the logs. | -| `MinimumLogLevel` | The minimum log level to log. This is used to filter out logs below the specified level. | -| `LogFormatter` | The log formatter to use. This is used to customize the structure of the log entries. | -| `JsonOptions` | The JSON options to use. This is used to customize the serialization of logs.| -| `LogBuffering` | The log buffering options. This is used to configure log buffering. | -| `TimestampFormat` | The format of the timestamp. This is used to customize the format of the timestamp in the logs.| -| `SamplingRate` | Sets a percentage (0.0 to 1.0) of logs that will be dynamically elevated to DEBUG level | -| `LoggerOutputCase` | The output casing of the logger. This is used to customize the casing of the log entries. | -| `LogOutput` | Specifies the console output wrapper used for writing logs. This property allows redirecting log output for testing or specialized handling scenarios. | - - -### Configuration - -You can configure Powertools Logger using the static `Logger` class. This class is a singleton and is created when the -Lambda function is initialized. You can configure the logger using the `Logger.Configure` method. - -=== "Configure static Logger" - -```c# hl_lines="5-9" - public class Function - { - public Function() - { - Logger.Configure(options => - { - options.MinimumLogLevel = LogLevel.Information; - options.LoggerOutputCase = LoggerOutputCase.CamelCase; - }); - } - - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Logger.LogInformation("Collecting payment"); - ... - } - } -``` - -### ILogger -You can also use the `ILogger` interface to log messages. This interface is part of the Microsoft.Extensions.Logging. -With this approach you get more flexibility and testability using dependency injection (DI). - -=== "Configure with LoggerFactory or Builder" - - ```c# hl_lines="5-12" - public class Function - { - public Function(ILogger logger) - { - _logger = logger ?? LoggerFactory.Create(builder => - { - builder.AddPowertoolsLogger(config => - { - config.Service = "TestService"; - config.LoggerOutputCase = LoggerOutputCase.PascalCase; - }); - }).CreatePowertoolsLogger(); - } - - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Logger.LogInformation("Collecting payment"); - ... - } - } - ``` - -## Standard structured keys - -Your logs will always include the following keys to your structured logging: - - Key | Type | Example | Description -------------------------|--------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------ - **Level** | string | "Information" | Logging level - **Message** | string | "Collecting payment" | Log statement value. Unserializable JSON values will be cast to string - **Timestamp** | string | "2020-05-24 18:17:33,774" | Timestamp of actual log statement - **Service** | string | "payment" | Service name defined. "service_undefined" will be used if unknown - **ColdStart** | bool | true | ColdStart value. - **FunctionName** | string | "example-powertools-HelloWorldFunction-1P1Z6B39FLU73" - **FunctionMemorySize** | string | "128" - **FunctionArn** | string | "arn:aws:lambda:eu-west-1:012345678910:function:example-powertools-HelloWorldFunction-1P1Z6B39FLU73" - **FunctionRequestId** | string | "899856cb-83d1-40d7-8611-9e78f15f32f4" | AWS Request ID from lambda context - **FunctionVersion** | string | "12" - **XRayTraceId** | string | "1-5759e988-bd862e3fe1be46a994272793" | X-Ray Trace ID when Lambda function has enabled Tracing - **Name** | string | "Powertools for AWS Lambda (.NET) Logger" | Logger name - **SamplingRate** | int | 0.1 | Debug logging sampling rate in percentage e.g. 10% in this case - **Customer Keys** | | | - -## Message templates - -You can use message templates to extract properties from your objects and log them as structured data. - -!!! info - - Override the `ToString()` method of your object to return a meaningful string representation of the object. - - This is especially important when using `{}` to log the object as a string. - - ```csharp - public class User - { - public string FirstName { get; set; } - public string LastName { get; set; } - public int Age { get; set; } - - public override string ToString() - { - return $"{LastName}, {FirstName} ({Age})"; - } - } - ``` - -If you want to log the object as a JSON object, use `{@}`. This will serialize the object and log it as a JSON object. - -=== "Message template {@}" - - ```c# hl_lines="7-14" - public class Function - { - [Logging(Service = "user-service", LogLevel = LogLevel.Information)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - var user = new User - { - FirstName = "John", - LastName = "Doe", - Age = 42 - }; - - logger.LogInformation("User object: {@user}", user); - ... - } - } - ``` - -=== "{@} Output" - - ```json hl_lines="3 8-12" - { - "level": "Information", - "message": "User object: Doe, John (42)", - "timestamp": "2025-04-07 09:06:30.708", - "service": "user-service", - "coldStart": true, - "name": "AWS.Lambda.Powertools.Logging.Logger", - "user": { - "firstName": "John", - "lastName": "Doe", - "age": 42 - }, - ... - } - ``` - -If you want to log the object as a string, use `{}`. This will call the `ToString()` method of the object and log it as -a string. - -=== "Message template {} ToString" - - ```c# hl_lines="7-12 14 18 19" - public class Function - { - [Logging(Service = "user", LogLevel = LogLevel.Information)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - var user = new User - { - FirstName = "John", - LastName = "Doe", - Age = 42 - }; - - logger.LogInformation("User data: {user}", user); - - // Also works with numbers, dates, etc. - - logger.LogInformation("Price: {price:0.00}", 123.4567); // will respect decimal places - logger.LogInformation("Percentage: {percent:0.0%}", 0.1234); - ... - } - } - ``` - -=== "Output {} ToString" - - ```json hl_lines="3 8 12 17 21 26" - { - "level": "Information", - "message": "User data: Doe, John (42)", - "timestamp": "2025-04-07 09:06:30.689", - "service": "user-servoice", - "coldStart": true, - "name": "AWS.Lambda.Powertools.Logging.Logger", - "user": "Doe, John (42)" - } - { - "level": "Information", - "message": "Price: 123.46", - "timestamp": "2025-04-07 09:23:01.235", - "service": "user-servoice", - "cold_start": true, - "name": "AWS.Lambda.Powertools.Logging.Logger", - "price": 123.46 - } - { - "level": "Information", - "message": "Percentage: 12.3%", - "timestamp": "2025-04-07 09:23:01.260", - "service": "user-servoice", - "cold_start": true, - "name": "AWS.Lambda.Powertools.Logging.Logger", - "percent": "12.3%" - } - ``` - - -## Logging incoming event - -When debugging in non-production environments, you can instruct Logger to log the incoming event with `LogEvent` -parameter or via `POWERTOOLS_LOGGER_LOG_EVENT` environment variable. - -!!! warning -Log event is disabled by default to prevent sensitive info being logged. - -=== "Function.cs" - - ```c# hl_lines="6" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(LogEvent = true)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - } - } - ``` - -## Setting a Correlation ID - -You can set a Correlation ID using `CorrelationIdPath` parameter by passing -a [JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03){target="_blank"}. - -!!! Attention -The JSON Pointer expression is `case sensitive`. In the bellow example `/headers/my_request_id_header` would work but -`/Headers/my_request_id_header` would not find the element. - -=== "Function.cs" - - ```c# hl_lines="6" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(CorrelationIdPath = "/headers/my_request_id_header")] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - } - } - ``` - -=== "Example Event" - - ```json hl_lines="3" - { - "headers": { - "my_request_id_header": "correlation_id_value" - } - } - ``` - -=== "Example CloudWatch Logs excerpt" - - ```json hl_lines="15" - { - "level": "Information", - "message": "Collecting payment", - "timestamp": "2021-12-13T20:32:22.5774262Z", - "service": "lambda-example", - "cold_start": true, - "function_name": "test", - "function_memory_size": 128, - "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", - "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72", - "function_version": "$LATEST", - "xray_trace_id": "1-61b7add4-66532bb81441e1b060389429", - "name": "AWS.Lambda.Powertools.Logging.Logger", - "sampling_rate": 0.7, - "correlation_id": "correlation_id_value", - } - ``` - -We provide [built-in JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03) -{target="_blank"} -for known event sources, where either a request ID or X-Ray Trace ID are present. - -=== "Function.cs" - - ```c# hl_lines="6" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(CorrelationIdPath = CorrelationIdPaths.ApiGatewayRest)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - } - } - ``` - -=== "Example Event" - - ```json hl_lines="3" - { - "RequestContext": { - "RequestId": "correlation_id_value" - } - } - ``` - -=== "Example CloudWatch Logs excerpt" - - ```json hl_lines="15" - { - "level": "Information", - "message": "Collecting payment", - "timestamp": "2021-12-13T20:32:22.5774262Z", - "service": "lambda-example", - "cold_start": true, - "function_name": "test", - "function_memory_size": 128, - "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", - "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72", - "function_version": "$LATEST", - "xray_trace_id": "1-61b7add4-66532bb81441e1b060389429", - "name": "AWS.Lambda.Powertools.Logging.Logger", - "sampling_rate": 0.7, - "correlation_id": "correlation_id_value", - } - ``` - -## Appending additional keys - -!!! info "Custom keys are persisted across warm invocations" -Always set additional keys as part of your handler to ensure they have the latest value, or explicitly clear them with [ -`ClearState=true`](#clearing-all-state). - -You can append your own keys to your existing logs via `AppendKey`. Typically this value would be passed into the -function via the event. Appended keys are added to all subsequent log entries in the current execution from the point -the logger method is called. To ensure the key is added to all log entries, call this method as early as possible in the -Lambda handler. - -=== "Function.cs" - - ```c# hl_lines="21" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(LogEvent = true)] - public async Task FunctionHandler(APIGatewayProxyRequest apigwProxyEvent, - ILambdaContext context) - { - var requestContextRequestId = apigwProxyEvent.RequestContext.RequestId; - - var lookupInfo = new Dictionary() - { - {"LookupInfo", new Dictionary{{ "LookupId", requestContextRequestId }}} - }; - - // Appended keys are added to all subsequent log entries in the current execution. - // Call this method as early as possible in the Lambda handler. - // Typically this is value would be passed into the function via the event. - // Set the ClearState = true to force the removal of keys across invocations, - Logger.AppendKeys(lookupInfo); - - Logger.LogInformation("Getting ip address from external service"); - - } - ``` - -=== "Example CloudWatch Logs excerpt" - - ```json hl_lines="4 5 6" - { - "level": "Information", - "message": "Getting ip address from external service" - "timestamp": "2022-03-14T07:25:20.9418065Z", - "service": "powertools-dotnet-logging-sample", - "cold_start": false, - "function_name": "PowertoolsLoggingSample-HelloWorldFunction-hm1r10VT3lCy", - "function_memory_size": 256, - "function_arn": "arn:aws:lambda:function:PowertoolsLoggingSample-HelloWorldFunction-hm1r10VT3lCy", - "function_request_id": "96570b2c-f00e-471c-94ad-b25e95ba7347", - "function_version": "$LATEST", - "xray_trace_id": "1-622eede0-647960c56a91f3b071a9fff1", - "name": "AWS.Lambda.Powertools.Logging.Logger", - "lookup_info": { - "lookup_id": "4c50eace-8b1e-43d3-92ba-0efacf5d1625" - }, - } - ``` - -### Removing additional keys - -You can remove any additional key from entry using `Logger.RemoveKeys()`. - -=== "Function.cs" - - ```c# hl_lines="21 22" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(LogEvent = true)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - Logger.AppendKey("test", "willBeLogged"); - ... - var customKeys = new Dictionary - { - {"test1", "value1"}, - {"test2", "value2"} - }; - - Logger.AppendKeys(customKeys); - ... - Logger.RemoveKeys("test"); - Logger.RemoveKeys("test1", "test2"); - ... - } - } - ``` - -## Extra Keys - -Extra keys allow you to append additional keys to a log entry. Unlike `AppendKey`, extra keys will only apply to the -current log entry. - -Extra keys argument is available for all log levels' methods, as implemented in the standard logging library - e.g. -Logger.Information, Logger.Warning. - -It accepts any dictionary, and all keyword arguments will be added as part of the root structure of the logs for that -log statement. - -!!! info -Any keyword argument added using extra keys will not be persisted for subsequent messages. - -=== "Function.cs" - - ```c# hl_lines="16" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(LogEvent = true)] - public async Task FunctionHandler(APIGatewayProxyRequest apigwProxyEvent, - ILambdaContext context) - { - var requestContextRequestId = apigwProxyEvent.RequestContext.RequestId; - - var lookupId = new Dictionary() - { - { "LookupId", requestContextRequestId } - }; - - // Appended keys are added to all subsequent log entries in the current execution. - // Call this method as early as possible in the Lambda handler. - // Typically this is value would be passed into the function via the event. - // Set the ClearState = true to force the removal of keys across invocations, - Logger.AppendKeys(lookupId); - } - ``` - -### Clearing all state - -Logger is commonly initialized in the global scope. Due -to [Lambda Execution Context reuse](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html), this means that -custom keys can be persisted across invocations. If you want all custom keys to be deleted, you can use -`ClearState=true` attribute on `[Logging]` attribute. - -=== "Function.cs" - - ```cs hl_lines="6 13" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(ClearState = true)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - if (apigProxyEvent.Headers.ContainsKey("SomeSpecialHeader")) - { - Logger.AppendKey("SpecialKey", "value"); - } - - Logger.LogInformation("Collecting payment"); - ... - } - } - ``` - -=== "#1 Request" - - ```json hl_lines="11" - { - "level": "Information", - "message": "Collecting payment", - "timestamp": "2021-12-13T20:32:22.5774262Z", - "service": "payment", - "cold_start": true, - "function_name": "test", - "function_memory_size": 128, - "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", - "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72", - "special_key": "value" - } - ``` - -=== "#2 Request" - - ```json - { - "level": "Information", - "message": "Collecting payment", - "timestamp": "2021-12-13T20:32:22.5774262Z", - "service": "payment", - "cold_start": true, - "function_name": "test", - "function_memory_size": 128, - "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", - "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72" - } - ``` - -## Sampling debug logs - -You can dynamically set a percentage of your logs to **DEBUG** level via env var `POWERTOOLS_LOGGER_SAMPLE_RATE` or -via `SamplingRate` parameter on attribute. - -!!! info -Configuration on environment variable is given precedence over sampling rate configuration on attribute, provided it's -in valid value range. - -=== "Sampling via attribute parameter" - - ```c# hl_lines="6" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(SamplingRate = 0.5)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - } - } - ``` - -=== "Sampling via environment variable" - - ```yaml hl_lines="8" - - Resources: - HelloWorldFunction: - Type: AWS::Serverless::Function - Properties: - ... - Environment: - Variables: - POWERTOOLS_LOGGER_SAMPLE_RATE: 0.5 - ``` - -## Configure Log Output Casing - -By definition Powertools for AWS Lambda (.NET) outputs logging keys using **snake case** (e.g. *"function_memory_size": -128*). This allows developers using different Powertools for AWS Lambda (.NET) runtimes, to search logs across services -written in languages such as Python or TypeScript. - -If you want to override the default behavior you can either set the desired casing through attributes, as described in -the example below, or by setting the `POWERTOOLS_LOGGER_CASE` environment variable on your AWS Lambda function. Allowed -values are: `CamelCase`, `PascalCase` and `SnakeCase`. - -=== "Output casing via attribute parameter" - - ```c# hl_lines="6" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - [Logging(LoggerOutputCase = LoggerOutputCase.CamelCase)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - } - } - ``` - -Below are some output examples for different casing. - -=== "Camel Case" - - ```json - { - "level": "Information", - "message": "Collecting payment", - "timestamp": "2021-12-13T20:32:22.5774262Z", - "service": "payment", - "coldStart": true, - "functionName": "test", - "functionMemorySize": 128, - "functionArn": "arn:aws:lambda:eu-west-1:12345678910:function:test", - "functionRequestId": "52fdfc07-2182-154f-163f-5f0f9a621d72" - } - ``` - -=== "Pascal Case" - - ```json - { - "Level": "Information", - "Message": "Collecting payment", - "Timestamp": "2021-12-13T20:32:22.5774262Z", - "Service": "payment", - "ColdStart": true, - "FunctionName": "test", - "FunctionMemorySize": 128, - "FunctionArn": "arn:aws:lambda:eu-west-1:12345678910:function:test", - "FunctionRequestId": "52fdfc07-2182-154f-163f-5f0f9a621d72" - } - ``` - -=== "Snake Case" - - ```json - { - "level": "Information", - "message": "Collecting payment", - "timestamp": "2021-12-13T20:32:22.5774262Z", - "service": "payment", - "cold_start": true, - "function_name": "test", - "function_memory_size": 128, - "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", - "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72" - } - ``` - - -## Advanced - -### Log Levels - -The default log level is `Information` and can be set using the `MinimumLogLevel` property option or by using the `POWERTOOLS_LOG_LEVEL` environment variable. - -We support the following log levels: - -| Level | Numeric value | Lambda Level | -|---------------|---------------|--------------| -| `Trace` | 0 | `trace` | -| `Debug` | 1 | `debug` | -| `Information` | 2 | `info` | -| `Warning` | 3 | `warn` | -| `Error` | 4 | `error` | -| `Critical` | 5 | `fatal` | -| `None` | 6 | | - -### Using AWS Lambda Advanced Logging Controls (ALC) - -!!! question "When is it useful?" -When you want to set a logging policy to drop informational or verbose logs for one or all AWS Lambda functions, -regardless of runtime and logger used. - -With [AWS Lambda Advanced Logging Controls (ALC)](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-advanced) -{target="_blank"}, you can enforce a minimum log level that Lambda will accept from your application code. - -When enabled, you should keep `Logger` and ALC log level in sync to avoid data loss. - -!!! warning "When using AWS Lambda Advanced Logging Controls (ALC)" -- When Powertools Logger output is set to `PascalCase` **`Level`** property name will be replaced by **`LogLevel`** as - a property name. -- ALC takes precedence over **`POWERTOOLS_LOG_LEVEL`** and when setting it in code using **`[Logging(LogLevel = )]`** - -Here's a sequence diagram to demonstrate how ALC will drop both `Information` and `Debug` logs emitted from `Logger`, -when ALC log level is stricter than `Logger`. - -```mermaid -sequenceDiagram - title Lambda ALC allows WARN logs only - participant Lambda service - participant Lambda function - participant Application Logger - - Note over Lambda service: AWS_LAMBDA_LOG_LEVEL="WARN" - Note over Application Logger: POWERTOOLS_LOG_LEVEL="DEBUG" - Lambda service->>Lambda function: Invoke (event) - Lambda function->>Lambda function: Calls handler - Lambda function->>Application Logger: Logger.Warning("Something happened") - Lambda function-->>Application Logger: Logger.Debug("Something happened") - Lambda function-->>Application Logger: Logger.Information("Something happened") - - Lambda service->>Lambda service: DROP INFO and DEBUG logs - - Lambda service->>CloudWatch Logs: Ingest error logs -``` - -**Priority of log level settings in Powertools for AWS Lambda** - -We prioritise log level settings in this order: - -1. AWS_LAMBDA_LOG_LEVEL environment variable -2. Setting the log level in code using `[Logging(LogLevel = )]` -3. POWERTOOLS_LOG_LEVEL environment variable - -If you set `Logger` level lower than ALC, we will emit a warning informing you that your messages will be discarded by -Lambda. - -> **NOTE** -> With ALC enabled, we are unable to increase the minimum log level below the `AWS_LAMBDA_LOG_LEVEL` environment -> variable value, -> see [AWS Lambda service documentation](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-log-level) -> {target="_blank"} for more details. - -### Using JsonSerializerOptions - -Powertools supports customizing the serialization and deserialization of Lambda JSON events and your own types using -`JsonSerializerOptions`. -You can do this by creating a custom `JsonSerializerOptions` and passing it to the `JsonOptions` of the Powertools -Logger. - -Supports `TypeInfoResolver` and `DictionaryKeyPolicy` options. These two options are the most common ones used to -customize the serialization of Powertools Logger. - -- `TypeInfoResolver`: This option allows you to specify a custom `JsonSerializerContext` that contains the types you - want to serialize and deserialize. This is especially useful when using AOT compilation, as it allows you to specify - the types that should be included in the generated assembly. -- `DictionaryKeyPolicy`: This option allows you to specify a custom naming policy for the properties in the JSON output. - This is useful when you want to change the casing of the property names or use a different naming convention. - -!!! info -If you want to preserve the original casing of the property names (keys), you can set the `DictionaryKeyPolicy` to -`null`. - -```csharp -builder.Logging.AddPowertoolsLogger(options => -{ - options.JsonOptions = new JsonSerializerOptions - { - DictionaryKeyPolicy = JsonNamingPolicy.CamelCase, // Override output casing - TypeInfoResolver = MyCustomJsonSerializerContext.Default // Your custom JsonSerializerContext - }; -}); -``` - -### Custom Log formatter (Bring Your Own Formatter) - -You can customize the structure (keys and values) of your log entries by implementing a custom log formatter and -override default log formatter using ``LogFormatter`` property in the `configure` options. - -You can implement a custom log formatter by -inheriting the ``ILogFormatter`` class and implementing the ``object FormatLogEntry(LogEntry logEntry)`` method. - -=== "Function.cs" - - ```c# hl_lines="11" - /** - * Handler for requests to Lambda function. - */ - public class Function - { - /// - /// Function constructor - /// - public Function() - { - Logger.Configure(options => - { - options.LogFormatter = new CustomLogFormatter(); - }); - } - - [Logging(CorrelationIdPath = "/headers/my_request_id_header", SamplingRate = 0.7)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - } - } - ``` - -=== "CustomLogFormatter.cs" - - ```c# - public class CustomLogFormatter : ILogFormatter - { - public object FormatLogEntry(LogEntry logEntry) - { - return new - { - Message = logEntry.Message, - Service = logEntry.Service, - CorrelationIds = new - { - AwsRequestId = logEntry.LambdaContext?.AwsRequestId, - XRayTraceId = logEntry.XRayTraceId, - CorrelationId = logEntry.CorrelationId - }, - LambdaFunction = new - { - Name = logEntry.LambdaContext?.FunctionName, - Arn = logEntry.LambdaContext?.InvokedFunctionArn, - MemoryLimitInMB = logEntry.LambdaContext?.MemoryLimitInMB, - Version = logEntry.LambdaContext?.FunctionVersion, - ColdStart = logEntry.ColdStart, - }, - Level = logEntry.Level.ToString(), - Timestamp = logEntry.Timestamp.ToString("o"), - Logger = new - { - Name = logEntry.Name, - SampleRate = logEntry.SamplingRate - }, - }; - } - } - ``` - -=== "Example CloudWatch Logs excerpt" - - ```json - { - "Message": "Test Message", - "Service": "lambda-example", - "CorrelationIds": { - "AwsRequestId": "52fdfc07-2182-154f-163f-5f0f9a621d72", - "XRayTraceId": "1-61b7add4-66532bb81441e1b060389429", - "CorrelationId": "correlation_id_value" - }, - "LambdaFunction": { - "Name": "test", - "Arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", - "MemorySize": 128, - "Version": "$LATEST", - "ColdStart": true - }, - "Level": "Information", - "Timestamp": "2021-12-13T20:32:22.5774262Z", - "Logger": { - "Name": "AWS.Lambda.Powertools.Logging.Logger", - "SampleRate": 0.7 - } - } - ``` - -### Buffering logs - -Log buffering enables you to buffer logs for a specific request or invocation. Enable log buffering by passing `LogBufferingOptions` when configuring a Logger instance. You can buffer logs at the `Warning`, `Information`, `Debug` or `Trace` level, and flush them automatically on error or manually as needed. - -!!! tip "This is useful when you want to reduce the number of log messages emitted while still having detailed logs when needed, such as when troubleshooting issues." - -=== "LogBufferingOptions" - - ```csharp hl_lines="5-14" - public class Function - { - public Function() - { - Logger.Configure(logger => - { - logger.Service = "MyServiceName"; - logger.LogBuffering = new LogBufferingOptions - { - BufferAtLogLevel = LogLevel.Debug, - MaxBytes = 20480, // Default is 20KB (20480 bytes) - FlushOnErrorLog = true // default true - }; - }); - - Logger.LogDebug('This is a debug message'); // This is NOT buffered - } - - [Logging] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Logger.LogDebug('This is a debug message'); // This is buffered - Logger.LogInformation('This is an info message'); - - // your business logic here - - Logger.LogError('This is an error message'); // This also flushes the buffer - } - } - - ``` - -#### Configuring the buffer - -When configuring the buffer, you can set the following options to fine-tune how logs are captured, stored, and emitted. You can configure the following options in the `logBufferOptions` constructor parameter: - -| Parameter | Description | Configuration | Default | -|---------------------|------------------------------------------------- |--------------------------------------------|---------| -| `MaxBytes` | Maximum size of the log buffer in bytes | `number` | `20480` | -| `BufferAtLogLevel` | Minimum log level to buffer | `Trace`, `Debug`, `Information`, `Warning` | `Debug` | -| `FlushOnErrorLog` | Automatically flush buffer when logging an error | `True`, `False` | `True` | - -=== "BufferAtLogLevel" - - ```csharp hl_lines="10" - public class Function - { - public Function() - { - Logger.Configure(logger => - { - logger.Service = "MyServiceName"; - logger.LogBuffering = new LogBufferingOptions - { - BufferAtLogLevel = LogLevel.Warning - }; - }); - } - - [Logging] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - // All logs below are buffered - Logger.LogDebug('This is a debug message'); - Logger.LogInformation('This is an info message'); - Logger.LogWarning('This is a warn message'); - - Logger.ClearBuffer(); // This will clear the buffer without emitting the logs - } - } - ``` - - 1. Setting `BufferAtLogLevel: 'Warning'` configures log buffering for `Warning` and all lower severity levels like `Information`, `Debug`, and `Trace`. - 2. Calling `Logger.ClearBuffer()` will clear the buffer without emitting the logs. - -=== "FlushOnErrorLog" - - ```csharp hl_lines="10" - public class Function - { - public Function() - { - Logger.Configure(logger => - { - logger.Service = "MyServiceName"; - logger.LogBuffering = new LogBufferingOptions - { - FlushOnErrorLog = false - }; - }); - } - - [Logging] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Logger.LogDebug('This is a debug message'); // this is buffered - - try - { - throw new Exception(); - } - catch (Exception e) - { - Logger.LogError(e.Message); // this does NOT flush the buffer - } - - Logger.LogDebug("Debug!!"); // this is buffered - - try - { - throw new Exception(); - } - catch (Exception e) - { - Logger.LogError(e.Message); // this does NOT flush the buffer - Logger.FlushBuffer(); // Manually flush - } - } - } - ``` - - 1. Disabling `FlushOnErrorLog` will not flush the buffer when logging an error. This is useful when you want to control when the buffer is flushed by calling the `Logger.FlushBuffer()` method. - -#### Flushing on errors - -When using the `Logger` decorator, you can configure the logger to automatically flush the buffer when an error occurs. This is done by setting the `FlushBufferOnUncaughtError` option to `true` in the decorator. - -=== "FlushBufferOnUncaughtError" - - ```csharp hl_lines="15" - public class Function - { - public Function() - { - Logger.Configure(logger => - { - logger.Service = "MyServiceName"; - logger.LogBuffering = new LogBufferingOptions - { - BufferAtLogLevel = LogLevel.Debug - }; - }); - } - - [Logging(FlushBufferOnUncaughtError = true)] - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Logger.LogDebug('This is a debug message'); - - throw new Exception(); // This causes the buffer to be flushed - } - } - ``` - -#### Buffering workflows - -##### Manual flush - -
-```mermaid -sequenceDiagram - participant Client - participant Lambda - participant Logger - participant CloudWatch - Client->>Lambda: Invoke Lambda - Lambda->>Logger: Initialize with DEBUG level buffering - Logger-->>Lambda: Logger buffer ready - Lambda->>Logger: Logger.LogDebug("First debug log") - Logger-->>Logger: Buffer first debug log - Lambda->>Logger: Logger.LogInformation("Info log") - Logger->>CloudWatch: Directly log info message - Lambda->>Logger: Logger.LogDebug("Second debug log") - Logger-->>Logger: Buffer second debug log - Lambda->>Logger: Logger.FlushBuffer() - Logger->>CloudWatch: Emit buffered logs to stdout - Lambda->>Client: Return execution result -``` -Flushing buffer manually -
- -##### Flushing when logging an error - -
-```mermaid -sequenceDiagram - participant Client - participant Lambda - participant Logger - participant CloudWatch - Client->>Lambda: Invoke Lambda - Lambda->>Logger: Initialize with DEBUG level buffering - Logger-->>Lambda: Logger buffer ready - Lambda->>Logger: Logger.LogDebug("First log") - Logger-->>Logger: Buffer first debug log - Lambda->>Logger: Logger.LogDebug("Second log") - Logger-->>Logger: Buffer second debug log - Lambda->>Logger: Logger.LogDebug("Third log") - Logger-->>Logger: Buffer third debug log - Lambda->>Lambda: Exception occurs - Lambda->>Logger: Logger.LogError("Error details") - Logger->>CloudWatch: Emit buffered debug logs - Logger->>CloudWatch: Emit error log - Lambda->>Client: Raise exception -``` -Flushing buffer when an error happens -
- -##### Flushing on error - -This works only when using the `Logger` decorator. You can configure the logger to automatically flush the buffer when an error occurs by setting the `FlushBufferOnUncaughtError` option to `true` in the decorator. - -
-```mermaid -sequenceDiagram - participant Client - participant Lambda - participant Logger - participant CloudWatch - Client->>Lambda: Invoke Lambda - Lambda->>Logger: Using decorator - Logger-->>Lambda: Logger context injected - Lambda->>Logger: Logger.LogDebug("First log") - Logger-->>Logger: Buffer first debug log - Lambda->>Logger: Logger.LogDebug("Second log") - Logger-->>Logger: Buffer second debug log - Lambda->>Lambda: Uncaught Exception - Lambda->>CloudWatch: Automatically emit buffered debug logs - Lambda->>Client: Raise uncaught exception -``` -Flushing buffer when an uncaught exception happens -
- -#### Buffering FAQs - -1. **Does the buffer persist across Lambda invocations?** - No, each Lambda invocation has its own buffer. The buffer is initialized when the Lambda function is invoked and is cleared after the function execution completes or when flushed manually. - -2. **Are my logs buffered during cold starts?** - No, we never buffer logs during cold starts. This is because we want to ensure that logs emitted during this phase are always available for debugging and monitoring purposes. The buffer is only used during the execution of the Lambda function. - -3. **How can I prevent log buffering from consuming excessive memory?** - You can limit the size of the buffer by setting the `MaxBytes` option in the `LogBufferingOptions` constructor parameter. This will ensure that the buffer does not grow indefinitely and consume excessive memory. - -4. **What happens if the log buffer reaches its maximum size?** - Older logs are removed from the buffer to make room for new logs. This means that if the buffer is full, you may lose some logs if they are not flushed before the buffer reaches its maximum size. When this happens, we emit a warning when flushing the buffer to indicate that some logs have been dropped. - -5. **How is the log size of a log line calculated?** - The log size is calculated based on the size of the serialized log line in bytes. This includes the size of the log message, the size of any additional keys, and the size of the timestamp. - -6. **What timestamp is used when I flush the logs?** - The timestamp preserves the original time when the log record was created. If you create a log record at 11:00:10 and flush it at 11:00:25, the log line will retain its original timestamp of 11:00:10. - -7. **What happens if I try to add a log line that is bigger than max buffer size?** - The log will be emitted directly to standard output and not buffered. When this happens, we emit a warning to indicate that the log line was too big to be buffered. - -8. **What happens if Lambda times out without flushing the buffer?** - Logs that are still in the buffer will be lost. If you are using the log buffer to log asynchronously, you should ensure that the buffer is flushed before the Lambda function times out. You can do this by calling the `Logger.FlushBuffer()` method at the end of your Lambda function. - -### Timestamp formatting - -You can customize the timestamp format by setting the `TimestampFormat` property in the `Logger.Configure` method. The default format is `o`, which is the ISO 8601 format. -You can use any valid [DateTime format string](https://docs.microsoft.com/en-us/dotnet/standard/base-types/custom-date-and-time-format-strings) to customize the timestamp format. -For example, to use the `yyyy-MM-dd HH:mm:ss` format, you can do the following: - -```csharp -Logger.Configure(logger => -{ - logger.TimestampFormat = "yyyy-MM-dd HH:mm:ss"; -}); -``` -This will output the timestamp in the following format: - -```json -{ - "level": "Information", - "message": "Test Message", - "timestamp": "2021-12-13 20:32:22", - "service": "lambda-example", - ... -} -``` - -## AOT Support - -!!! info - - If you want to use the `LogEvent`, `Custom Log Formatter` features, or serialize your own types when Logging events, you need to either pass `JsonSerializerContext` or make changes in your Lambda `Main` method. - -!!! info - - Starting from version 1.6.0, it is required to update the Amazon.Lambda.Serialization.SystemTextJson NuGet package to version 2.4.3 in your csproj. - -### Using JsonSerializerOptions - -To be able to serializer your own types, you need to pass your `JsonSerializerContext` to the `TypeInfoResolver` of the `Logger.Configure` method. - -```csharp -Logger.Configure(logger => -{ - logger.JsonOptions = new JsonSerializerOptions - { - TypeInfoResolver = YourJsonSerializerContext.Default - }; -}); -``` - -### Using PowertoolsSourceGeneratorSerializer - -Replace `SourceGeneratorLambdaJsonSerializer` with `PowertoolsSourceGeneratorSerializer`. - -This change enables Powertools to construct an instance of `JsonSerializerOptions` used to customize the serialization -and deserialization of Lambda JSON events and your own types. - -=== "Before" - - ```csharp - Func> handler = FunctionHandler; - await LambdaBootstrapBuilder.Create(handler, new SourceGeneratorLambdaJsonSerializer()) - .Build() - .RunAsync(); - ``` - -=== "After" - - ```csharp hl_lines="2" - Func> handler = FunctionHandler; - await LambdaBootstrapBuilder.Create(handler, new PowertoolsSourceGeneratorSerializer()) - .Build() - .RunAsync(); - ``` - -For example when you have your own Demo type - -```csharp -public class Demo -{ - public string Name { get; set; } - public Headers Headers { get; set; } -} -``` - -To be able to serialize it in AOT you have to have your own `JsonSerializerContext` - -```csharp -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))] -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))] -[JsonSerializable(typeof(Demo))] -public partial class MyCustomJsonSerializerContext : JsonSerializerContext -{ -} -``` - -When you update your code to use `PowertoolsSourceGeneratorSerializer`, we combine your -`JsonSerializerContext` with Powertools' `JsonSerializerContext`. This allows Powertools to serialize your types and -Lambda events. - -### Custom Log Formatter - -To use a custom log formatter with AOT, pass an instance of `ILogFormatter` to `PowertoolsSourceGeneratorSerializer` -instead of using the static `Logger.UseFormatter` in the Function constructor as you do in non-AOT Lambdas. - -=== "Function Main method" - - ```csharp hl_lines="5" - - Func> handler = FunctionHandler; - await LambdaBootstrapBuilder.Create(handler, - new PowertoolsSourceGeneratorSerializer - ( - new CustomLogFormatter() - ) - ) - .Build() - .RunAsync(); - - ``` - -=== "CustomLogFormatter.cs" - - ```csharp - public class CustomLogFormatter : ILogFormatter - { - public object FormatLogEntry(LogEntry logEntry) - { - return new - { - Message = logEntry.Message, - Service = logEntry.Service, - CorrelationIds = new - { - AwsRequestId = logEntry.LambdaContext?.AwsRequestId, - XRayTraceId = logEntry.XRayTraceId, - CorrelationId = logEntry.CorrelationId - }, - LambdaFunction = new - { - Name = logEntry.LambdaContext?.FunctionName, - Arn = logEntry.LambdaContext?.InvokedFunctionArn, - MemoryLimitInMB = logEntry.LambdaContext?.MemoryLimitInMB, - Version = logEntry.LambdaContext?.FunctionVersion, - ColdStart = logEntry.ColdStart, - }, - Level = logEntry.Level.ToString(), - Timestamp = logEntry.Timestamp.ToString("o"), - Logger = new - { - Name = logEntry.Name, - SampleRate = logEntry.SamplingRate - }, - }; - } - } - ``` - -### Anonymous types - -!!! note - - While we support anonymous type serialization by converting to a `Dictionary`, this is **not** a best practice and is **not recommended** when using native AOT. - - We recommend using concrete classes and adding them to your `JsonSerializerContext`. - -## Testing - -You can change where the `Logger` will output its logs by setting the `LogOutput` property. -We also provide a helper class for tests `TestLoggerOutput` or you can provider your own implementation of `IConsoleWrapper`. - -```csharp -// Using TestLoggerOutput -options.LogOutput = new TestLoggerOutput(); -// Custom console output for testing -options.LogOutput = new TestConsoleWrapper(); - -// Example implementation for testing: -public class TestConsoleWrapper : IConsoleWrapper -{ - public List CapturedOutput { get; } = new(); - - public void WriteLine(string message) - { - CapturedOutput.Add(message); - } -} -``` -### ILogger - -If you are using ILogger interface you can inject the logger in a dedicated constructor for your Lambda function and thus you can mock your ILogger instance. - -```csharp -public class Function -{ - private readonly ILogger _logger; - - public Function() - { - _logger = oggerFactory.Create(builder => - { - builder.AddPowertoolsLogger(config => - { - config.Service = "TestService"; - config.LoggerOutputCase = LoggerOutputCase.PascalCase; - }); - }).CreatePowertoolsLogger(); - } - - // constructor used for tests - pass the mock ILogger - public Function(ILogger logger) - { - _logger = logger ?? loggerFactory.Create(builder => - { - builder.AddPowertoolsLogger(config => - { - config.Service = "TestService"; - config.LoggerOutputCase = LoggerOutputCase.PascalCase; - }); - }).CreatePowertoolsLogger(); - } - - public async Task FunctionHandler - (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - _logger.LogInformation("Collecting payment"); - ... - } -} -``` - - diff --git a/docs/core/logging.md b/docs/core/logging.md index a8a2cfce7..9ea38fb40 100644 --- a/docs/core/logging.md +++ b/docs/core/logging.md @@ -11,154 +11,442 @@ The logging utility provides a Lambda optimized logger with output structured as * Log Lambda event when instructed (disabled by default) * Log sampling enables DEBUG log level for a percentage of requests (disabled by default) * Append additional keys to structured log at any point in time -* Ahead-of-Time compilation to native code support [AOT](https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html) from version 1.6.0 +* Ahead-of-Time compilation to native code + support [AOT](https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html) +* Custom log formatter to override default log structure +* Support + for [AWS Lambda Advanced Logging Controls (ALC)](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs-advanced.html) + {target="_blank"} +* Support for Microsoft.Extensions.Logging + and [ILogger](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.logging.ilogger?view=dotnet-plat-ext-7.0) + interface +* Support + for [ILoggerFactory](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.logging.iloggerfactory?view=dotnet-plat-ext-7.0) + interface +* Support for message templates `{}` and `{@}` for structured logging + +## Breaking changes from v1 (dependency updates) + +!!! info + + Loooking for V1 specific documentation please go to [Logging v1](/lambda/dotnet/core/logging-v1) + +| Change | Before (v1.x) | After (v2.0) | Migration Action | +|--------|---------------|--------------|-----------------| +| Amazon.Lambda.Core | 2.2.0|2.5.0 | dotnet add package Amazon.Lambda.Core | +| Amazon.Lambda.Serialization.SystemTextJson | 2.4.3 | 2.4.4 | dotnet add package Amazon.Lambda.Serialization.SystemTextJson | +| Microsoft.Extensions.DependencyInjection | 8.0.0 | 8.0.1 | dotnet add package Microsoft.Extensions.DependencyInjection | + +#### Extra keys - Breaking change + +In v1.x, the extra keys were added to the log entry as a dictionary. In v2.x, the extra keys are added to the log entry as +a JSON object. + +There is no longer a method that accepts extra keys as first argument. + +=== "Before (v1)" + + ```csharp + public class User + { + public string Name { get; set; } + public int Age { get; set; } + } + + Logger.LogInformation(user, "{Name} is {Age} years old", + new object[]{user.Name, user.Age}); + + var scopeKeys = new + { + PropOne = "Value 1", + PropTwo = "Value 2" + }; + Logger.LogInformation(scopeKeys, "message"); + + ``` + +=== "After (v2)" + + ```csharp + public class User + { + public string Name { get; set; } + public int Age { get; set; } + + public override string ToString() + { + return $"{Name} is {Age} years old"; + } + } + + // It uses the ToString() method of the object to log the message + // the extra keys are added because of the {@} in the message template + Logger.LogInformation("{@user}", user); + + var scopeKeys = new + { + PropOne = "Value 1", + PropTwo = "Value 2" + }; + + // there is no longer a method that accepts extra keys as first argument. + Logger.LogInformation("{@keys}", scopeKeys); + ``` + +This change was made to improve the performance of the logger and to make it easier to work with the extra keys. + ## Installation -Powertools for AWS Lambda (.NET) are available as NuGet packages. You can install the packages from [NuGet Gallery](https://www.nuget.org/packages?q=AWS+Lambda+Powertools*){target="_blank"} or from Visual Studio editor by searching `AWS.Lambda.Powertools*` to see various utilities available. +Powertools for AWS Lambda (.NET) are available as NuGet packages. You can install the packages +from [NuGet Gallery](https://www.nuget.org/packages?q=AWS+Lambda+Powertools*){target="_blank"} or from Visual Studio +editor by searching `AWS.Lambda.Powertools*` to see various utilities available. * [AWS.Lambda.Powertools.Logging](https://www.nuget.org/packages?q=AWS.Lambda.Powertools.Logging): - `dotnet add package AWS.Lambda.Powertools.Logging --version 1.6.5` + `dotnet add package AWS.Lambda.Powertools.Logging` ## Getting started !!! info - + AOT Support If loooking for AOT specific configurations navigate to the [AOT section](#aot-support) - Logging requires two settings: -Setting | Description | Environment variable | Attribute parameter -------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- -**Service** | Sets **Service** key that will be present across all log statements | `POWERTOOLS_SERVICE_NAME` | `Service` -**Logging level** | Sets how verbose Logger should be (Information, by default) | `POWERTOOLS_LOG_LEVEL` | `LogLevel` + Setting | Description | Environment variable | Attribute parameter +-------------------|---------------------------------------------------------------------|---------------------------|--------------------- + **Service** | Sets **Service** key that will be present across all log statements | `POWERTOOLS_SERVICE_NAME` | `Service` + **Logging level** | Sets how verbose Logger should be (Information, by default) | `POWERTOOLS_LOG_LEVEL` | `LogLevel` -### Service Property Priority Resolution - -The root level Service property now correctly follows this priority order: +### Full list of environment variables -1. LoggingAttribute.Service (property value set in the decorator) -2. POWERTOOLS_SERVICE_NAME (environment variable) +| Environment variable | Description | Default | +|-----------------------------------|----------------------------------------------------------------------------------------|-----------------------| +| **POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | `"service_undefined"` | +| **POWERTOOLS_LOG_LEVEL** | Sets logging level | `Information` | +| **POWERTOOLS_LOGGER_CASE** | Override the default casing for log keys | `SnakeCase` | +| **POWERTOOLS_LOGGER_LOG_EVENT** | Logs incoming event | `false` | +| **POWERTOOLS_LOGGER_SAMPLE_RATE** | Debug log sampling | `0` | +### Setting up the logger -### Example using AWS Serverless Application Model (AWS SAM) +You can set up the logger in different ways. The most common way is to use the `Logging` attribute on your Lambda. +You can also use the `ILogger` interface to log messages. This interface is part of the Microsoft.Extensions.Logging. -You can override log level by setting **`POWERTOOLS_LOG_LEVEL`** environment variable in the AWS SAM template. +=== "Using decorator" -You can also explicitly set a service name via **`POWERTOOLS_SERVICE_NAME`** environment variable. This sets **Service** key that will be present across all log statements. + ```c# hl_lines="6 10" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + [Logging(Service = "payment", LogLevel = LogLevel.Debug)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Logger.LogInformation("Collecting payment"); + ... + } + } + ``` -Here is an example using the AWS SAM [Globals section](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-specification-template-anatomy-globals.html). +=== "Logger Factory" -=== "template.yaml" + ```c# hl_lines="6 10-17 23" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + private readonly ILogger _logger; + + public Function(ILoggerFactory loggerFactory) + { + _logger = loggerFactory.Create(builder => + { + builder.AddPowertoolsLogger(config => + { + config.Service = "TestService"; + config.LoggerOutputCase = LoggerOutputCase.PascalCase; + }); + }).CreatePowertoolsLogger(); + } + + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + _logger.LogInformation("Collecting payment"); + ... + } + } + ``` - ```yaml hl_lines="13 14" - # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. - # SPDX-License-Identifier: MIT-0 - AWSTemplateFormatVersion: "2010-09-09" - Transform: AWS::Serverless-2016-10-31 - Description: > - Example project for Powertools for AWS Lambda (.NET) Logging utility +=== "With Builder" - Globals: - Function: - Timeout: 10 - Environment: - Variables: - POWERTOOLS_SERVICE_NAME: powertools-dotnet-logging-sample - POWERTOOLS_LOG_LEVEL: Debug - POWERTOOLS_LOGGER_LOG_EVENT: true - POWERTOOLS_LOGGER_CASE: PascalCase # Allowed values are: CamelCase, PascalCase and SnakeCase - POWERTOOLS_LOGGER_SAMPLE_RATE: 0 + ```c# hl_lines="6 10-13 19" + /** + * Handler for requests to Lambda function. + */ + public class Function + { + private readonly ILogger _logger; + + public Function(ILogger logger) + { + _logger = logger ?? new PowertoolsLoggerBuilder() + .WithService("TestService") + .WithOutputCase(LoggerOutputCase.PascalCase) + .Build(); + } + + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + _logger.LogInformation("Collecting payment"); + ... + } + } ``` -### Full list of environment variables +### Customizing the logger -| Environment variable | Description | Default | -| ------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- | -| **POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | `"service_undefined"` | -| **POWERTOOLS_LOG_LEVEL** | Sets logging level | `Information` | -| **POWERTOOLS_LOGGER_CASE** | Override the default casing for log keys | `SnakeCase` | -| **POWERTOOLS_LOGGER_LOG_EVENT** | Logs incoming event | `false` | -| **POWERTOOLS_LOGGER_SAMPLE_RATE** | Debug log sampling | `0` | +You can customize the logger by setting the following properties in the `Logger.Configure` method: +| Property | Description | +|:----------------------|--------------------------------------------------------------------------------------------------| +| `Service` | The name of the service. This is used to identify the service in the logs. | +| `MinimumLogLevel` | The minimum log level to log. This is used to filter out logs below the specified level. | +| `LogFormatter` | The log formatter to use. This is used to customize the structure of the log entries. | +| `JsonOptions` | The JSON options to use. This is used to customize the serialization of logs.| +| `LogBuffering` | The log buffering options. This is used to configure log buffering. | +| `TimestampFormat` | The format of the timestamp. This is used to customize the format of the timestamp in the logs.| +| `SamplingRate` | Sets a percentage (0.0 to 1.0) of logs that will be dynamically elevated to DEBUG level | +| `LoggerOutputCase` | The output casing of the logger. This is used to customize the casing of the log entries. | +| `LogOutput` | Specifies the console output wrapper used for writing logs. This property allows redirecting log output for testing or specialized handling scenarios. | -### Using AWS Lambda Advanced Logging Controls (ALC) -!!! question "When is it useful?" - When you want to set a logging policy to drop informational or verbose logs for one or all AWS Lambda functions, regardless of runtime and logger used. +### Configuration -With [AWS Lambda Advanced Logging Controls (ALC)](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-advanced){target="_blank"}, you can enforce a minimum log level that Lambda will accept from your application code. +You can configure Powertools Logger using the static `Logger` class. This class is a singleton and is created when the +Lambda function is initialized. You can configure the logger using the `Logger.Configure` method. -When enabled, you should keep `Logger` and ALC log level in sync to avoid data loss. +=== "Configure static Logger" -!!! warning "When using AWS Lambda Advanced Logging Controls (ALC)" - - When Powertools Logger output is set to `PascalCase` **`Level`** property name will be replaced by **`LogLevel`** as a property name. - - ALC takes precedence over **`POWERTOOLS_LOG_LEVEL`** and when setting it in code using **`[Logging(LogLevel = )]`** +```c# hl_lines="5-9" + public class Function + { + public Function() + { + Logger.Configure(options => + { + options.MinimumLogLevel = LogLevel.Information; + options.LoggerOutputCase = LoggerOutputCase.CamelCase; + }); + } -Here's a sequence diagram to demonstrate how ALC will drop both `Information` and `Debug` logs emitted from `Logger`, when ALC log level is stricter than `Logger`. + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Logger.LogInformation("Collecting payment"); + ... + } + } +``` -```mermaid -sequenceDiagram - title Lambda ALC allows WARN logs only - participant Lambda service - participant Lambda function - participant Application Logger - - Note over Lambda service: AWS_LAMBDA_LOG_LEVEL="WARN" - Note over Application Logger: POWERTOOLS_LOG_LEVEL="DEBUG" - Lambda service->>Lambda function: Invoke (event) - Lambda function->>Lambda function: Calls handler - Lambda function->>Application Logger: Logger.Warning("Something happened") - Lambda function-->>Application Logger: Logger.Debug("Something happened") - Lambda function-->>Application Logger: Logger.Information("Something happened") +### ILogger +You can also use the `ILogger` interface to log messages. This interface is part of the Microsoft.Extensions.Logging. +With this approach you get more flexibility and testability using dependency injection (DI). + +=== "Configure with LoggerFactory or Builder" + + ```c# hl_lines="5-12" + public class Function + { + public Function(ILogger logger) + { + _logger = logger ?? LoggerFactory.Create(builder => + { + builder.AddPowertoolsLogger(config => + { + config.Service = "TestService"; + config.LoggerOutputCase = LoggerOutputCase.PascalCase; + }); + }).CreatePowertoolsLogger(); + } - Lambda service->>Lambda service: DROP INFO and DEBUG logs + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Logger.LogInformation("Collecting payment"); + ... + } + } + ``` - Lambda service->>CloudWatch Logs: Ingest error logs -``` +## Standard structured keys -**Priority of log level settings in Powertools for AWS Lambda** +Your logs will always include the following keys to your structured logging: -We prioritise log level settings in this order: + Key | Type | Example | Description +------------------------|--------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------ + **Level** | string | "Information" | Logging level + **Message** | string | "Collecting payment" | Log statement value. Unserializable JSON values will be cast to string + **Timestamp** | string | "2020-05-24 18:17:33,774" | Timestamp of actual log statement + **Service** | string | "payment" | Service name defined. "service_undefined" will be used if unknown + **ColdStart** | bool | true | ColdStart value. + **FunctionName** | string | "example-powertools-HelloWorldFunction-1P1Z6B39FLU73" + **FunctionMemorySize** | string | "128" + **FunctionArn** | string | "arn:aws:lambda:eu-west-1:012345678910:function:example-powertools-HelloWorldFunction-1P1Z6B39FLU73" + **FunctionRequestId** | string | "899856cb-83d1-40d7-8611-9e78f15f32f4" | AWS Request ID from lambda context + **FunctionVersion** | string | "12" + **XRayTraceId** | string | "1-5759e988-bd862e3fe1be46a994272793" | X-Ray Trace ID when Lambda function has enabled Tracing + **Name** | string | "Powertools for AWS Lambda (.NET) Logger" | Logger name + **SamplingRate** | int | 0.1 | Debug logging sampling rate in percentage e.g. 10% in this case + **Customer Keys** | | | + +!!! Warning + If you emit a log message with a key that matches one of `level`, `message`, `name`, `service`, or `timestamp`, the Logger will ignore the key. + +## Message templates + +You can use message templates to extract properties from your objects and log them as structured data. -1. AWS_LAMBDA_LOG_LEVEL environment variable -2. Setting the log level in code using `[Logging(LogLevel = )]` -3. POWERTOOLS_LOG_LEVEL environment variable +!!! info -If you set `Logger` level lower than ALC, we will emit a warning informing you that your messages will be discarded by Lambda. + Override the `ToString()` method of your object to return a meaningful string representation of the object. -> **NOTE** -> With ALC enabled, we are unable to increase the minimum log level below the `AWS_LAMBDA_LOG_LEVEL` environment variable value, see [AWS Lambda service documentation](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-log-level){target="_blank"} for more details. + This is especially important when using `{}` to log the object as a string. -## Standard structured keys + ```csharp + public class User + { + public string FirstName { get; set; } + public string LastName { get; set; } + public int Age { get; set; } -Your logs will always include the following keys to your structured logging: + public override string ToString() + { + return $"{LastName}, {FirstName} ({Age})"; + } + } + ``` + +If you want to log the object as a JSON object, use `{@}`. This will serialize the object and log it as a JSON object. + +=== "Message template {@}" + + ```c# hl_lines="7-14" + public class Function + { + [Logging(Service = "user-service", LogLevel = LogLevel.Information)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + var user = new User + { + FirstName = "John", + LastName = "Doe", + Age = 42 + }; + + logger.LogInformation("User object: {@user}", user); + ... + } + } + ``` + +=== "{@} Output" + + ```json hl_lines="3 8-12" + { + "level": "Information", + "message": "User object: Doe, John (42)", + "timestamp": "2025-04-07 09:06:30.708", + "service": "user-service", + "coldStart": true, + "name": "AWS.Lambda.Powertools.Logging.Logger", + "user": { + "firstName": "John", + "lastName": "Doe", + "age": 42 + }, + ... + } + ``` + +If you want to log the object as a string, use `{}`. This will call the `ToString()` method of the object and log it as +a string. + +=== "Message template {} ToString" + + ```c# hl_lines="7-12 14 18 19" + public class Function + { + [Logging(Service = "user", LogLevel = LogLevel.Information)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + var user = new User + { + FirstName = "John", + LastName = "Doe", + Age = 42 + }; + + logger.LogInformation("User data: {user}", user); + + // Also works with numbers, dates, etc. + + logger.LogInformation("Price: {price:0.00}", 123.4567); // will respect decimal places + logger.LogInformation("Percentage: {percent:0.0%}", 0.1234); + ... + } + } + ``` + +=== "Output {} ToString" + + ```json hl_lines="3 8 12 17 21 26" + { + "level": "Information", + "message": "User data: Doe, John (42)", + "timestamp": "2025-04-07 09:06:30.689", + "service": "user-servoice", + "coldStart": true, + "name": "AWS.Lambda.Powertools.Logging.Logger", + "user": "Doe, John (42)" + } + { + "level": "Information", + "message": "Price: 123.46", + "timestamp": "2025-04-07 09:23:01.235", + "service": "user-servoice", + "cold_start": true, + "name": "AWS.Lambda.Powertools.Logging.Logger", + "price": 123.46 + } + { + "level": "Information", + "message": "Percentage: 12.3%", + "timestamp": "2025-04-07 09:23:01.260", + "service": "user-servoice", + "cold_start": true, + "name": "AWS.Lambda.Powertools.Logging.Logger", + "percent": "12.3%" + } + ``` -Key | Type | Example | Description -------------------------------------------------- | ------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- -**Timestamp** | string | "2020-05-24 18:17:33,774" | Timestamp of actual log statement -**Level** | string | "Information" | Logging level -**Name** | string | "Powertools for AWS Lambda (.NET) Logger" | Logger name -**ColdStart** | bool | true| ColdStart value. -**Service** | string | "payment" | Service name defined. "service_undefined" will be used if unknown -**SamplingRate** | int | 0.1 | Debug logging sampling rate in percentage e.g. 10% in this case -**Message** | string | "Collecting payment" | Log statement value. Unserializable JSON values will be cast to string -**FunctionName**| string | "example-powertools-HelloWorldFunction-1P1Z6B39FLU73" -**FunctionVersion**| string | "12" -**FunctionMemorySize**| string | "128" -**FunctionArn**| string | "arn:aws:lambda:eu-west-1:012345678910:function:example-powertools-HelloWorldFunction-1P1Z6B39FLU73" -**XRayTraceId**| string | "1-5759e988-bd862e3fe1be46a994272793" | X-Ray Trace ID when Lambda function has enabled Tracing -**FunctionRequestId**| string | "899856cb-83d1-40d7-8611-9e78f15f32f4" | AWS Request ID from lambda context ## Logging incoming event -When debugging in non-production environments, you can instruct Logger to log the incoming event with `LogEvent` parameter or via `POWERTOOLS_LOGGER_LOG_EVENT` environment variable. +When debugging in non-production environments, you can instruct Logger to log the incoming event with `LogEvent` +parameter or via `POWERTOOLS_LOGGER_LOG_EVENT` environment variable. !!! warning - Log event is disabled by default to prevent sensitive info being logged. +Log event is disabled by default to prevent sensitive info being logged. === "Function.cs" @@ -179,11 +467,12 @@ When debugging in non-production environments, you can instruct Logger to log th ## Setting a Correlation ID -You can set a Correlation ID using `CorrelationIdPath` parameter by passing a [JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03){target="_blank"}. +You can set a Correlation ID using `CorrelationIdPath` parameter by passing +a [JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03){target="_blank"}. !!! Attention - The JSON Pointer expression is `case sensitive`. In the bellow example `/headers/my_request_id_header` would work but `/Headers/my_request_id_header` would not find the element. - +The JSON Pointer expression is `case sensitive`. In the bellow example `/headers/my_request_id_header` would work but +`/Headers/my_request_id_header` would not find the element. === "Function.cs" @@ -201,6 +490,7 @@ You can set a Correlation ID using `CorrelationIdPath` parameter by passing a [J } } ``` + === "Example Event" ```json hl_lines="3" @@ -215,23 +505,25 @@ You can set a Correlation ID using `CorrelationIdPath` parameter by passing a [J ```json hl_lines="15" { + "level": "Information", + "message": "Collecting payment", + "timestamp": "2021-12-13T20:32:22.5774262Z", + "service": "lambda-example", "cold_start": true, - "xray_trace_id": "1-61b7add4-66532bb81441e1b060389429", "function_name": "test", - "function_version": "$LATEST", "function_memory_size": 128, "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72", - "timestamp": "2021-12-13T20:32:22.5774262Z", - "level": "Information", - "service": "lambda-example", + "function_version": "$LATEST", + "xray_trace_id": "1-61b7add4-66532bb81441e1b060389429", "name": "AWS.Lambda.Powertools.Logging.Logger", - "message": "Collecting payment", "sampling_rate": 0.7, "correlation_id": "correlation_id_value", } ``` -We provide [built-in JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03){target="_blank"} + +We provide [built-in JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03) +{target="_blank"} for known event sources, where either a request ID or X-Ray Trace ID are present. === "Function.cs" @@ -265,18 +557,18 @@ for known event sources, where either a request ID or X-Ray Trace ID are present ```json hl_lines="15" { + "level": "Information", + "message": "Collecting payment", + "timestamp": "2021-12-13T20:32:22.5774262Z", + "service": "lambda-example", "cold_start": true, - "xray_trace_id": "1-61b7add4-66532bb81441e1b060389429", "function_name": "test", - "function_version": "$LATEST", "function_memory_size": 128, "function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", "function_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72", - "timestamp": "2021-12-13T20:32:22.5774262Z", - "level": "Information", - "service": "lambda-example", + "function_version": "$LATEST", + "xray_trace_id": "1-61b7add4-66532bb81441e1b060389429", "name": "AWS.Lambda.Powertools.Logging.Logger", - "message": "Collecting payment", "sampling_rate": 0.7, "correlation_id": "correlation_id_value", } @@ -285,9 +577,13 @@ for known event sources, where either a request ID or X-Ray Trace ID are present ## Appending additional keys !!! info "Custom keys are persisted across warm invocations" - Always set additional keys as part of your handler to ensure they have the latest value, or explicitly clear them with [`ClearState=true`](#clearing-all-state). +Always set additional keys as part of your handler to ensure they have the latest value, or explicitly clear them with [ +`ClearState=true`](#clearing-all-state). -You can append your own keys to your existing logs via `AppendKey`. Typically this value would be passed into the function via the event. Appended keys are added to all subsequent log entries in the current execution from the point the logger method is called. To ensure the key is added to all log entries, call this method as early as possible in the Lambda handler. +You can append your own keys to your existing logs via `AppendKey`. Typically this value would be passed into the +function via the event. Appended keys are added to all subsequent log entries in the current execution from the point +the logger method is called. To ensure the key is added to all log entries, call this method as early as possible in the +Lambda handler. === "Function.cs" @@ -318,25 +614,26 @@ You can append your own keys to your existing logs via `AppendKey`. Typically th } ``` + === "Example CloudWatch Logs excerpt" ```json hl_lines="4 5 6" { + "level": "Information", + "message": "Getting ip address from external service" + "timestamp": "2022-03-14T07:25:20.9418065Z", + "service": "powertools-dotnet-logging-sample", "cold_start": false, - "xray_trace_id": "1-622eede0-647960c56a91f3b071a9fff1", - "lookup_info": { - "lookup_id": "4c50eace-8b1e-43d3-92ba-0efacf5d1625" - }, "function_name": "PowertoolsLoggingSample-HelloWorldFunction-hm1r10VT3lCy", - "function_version": "$LATEST", "function_memory_size": 256, - "function_arn": "arn:aws:lambda:ap-southeast-2:538510314095:function:PowertoolsLoggingSample-HelloWorldFunction-hm1r10VT3lCy", + "function_arn": "arn:aws:lambda:function:PowertoolsLoggingSample-HelloWorldFunction-hm1r10VT3lCy", "function_request_id": "96570b2c-f00e-471c-94ad-b25e95ba7347", - "timestamp": "2022-03-14T07:25:20.9418065Z", - "level": "Information", - "service": "powertools-dotnet-logging-sample", + "function_version": "$LATEST", + "xray_trace_id": "1-622eede0-647960c56a91f3b071a9fff1", "name": "AWS.Lambda.Powertools.Logging.Logger", - "message": "Getting ip address from external service" + "lookup_info": { + "lookup_id": "4c50eace-8b1e-43d3-92ba-0efacf5d1625" + }, } ``` @@ -376,14 +673,17 @@ You can remove any additional key from entry using `Logger.RemoveKeys()`. ## Extra Keys -Extra keys allow you to append additional keys to a log entry. Unlike `AppendKey`, extra keys will only apply to the current log entry. +Extra keys allow you to append additional keys to a log entry. Unlike `AppendKey`, extra keys will only apply to the +current log entry. -Extra keys argument is available for all log levels' methods, as implemented in the standard logging library - e.g. Logger.Information, Logger.Warning. +Extra keys argument is available for all log levels' methods, as implemented in the standard logging library - e.g. +Logger.Information, Logger.Warning. -It accepts any dictionary, and all keyword arguments will be added as part of the root structure of the logs for that log statement. +It accepts any dictionary, and all keyword arguments will be added as part of the root structure of the logs for that +log statement. !!! info - Any keyword argument added using extra keys will not be persisted for subsequent messages. +Any keyword argument added using extra keys will not be persisted for subsequent messages. === "Function.cs" @@ -414,7 +714,10 @@ It accepts any dictionary, and all keyword arguments will be added as part of th ### Clearing all state -Logger is commonly initialized in the global scope. Due to [Lambda Execution Context reuse](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html), this means that custom keys can be persisted across invocations. If you want all custom keys to be deleted, you can use `ClearState=true` attribute on `[Logging]` attribute. +Logger is commonly initialized in the global scope. Due +to [Lambda Execution Context reuse](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html), this means that +custom keys can be persisted across invocations. If you want all custom keys to be deleted, you can use +`ClearState=true` attribute on `[Logging]` attribute. === "Function.cs" @@ -439,6 +742,7 @@ Logger is commonly initialized in the global scope. Due to [Lambda Execution Con } } ``` + === "#1 Request" ```json hl_lines="11" @@ -478,7 +782,8 @@ You can dynamically set a percentage of your logs to **DEBUG** level via env var via `SamplingRate` parameter on attribute. !!! info - Configuration on environment variable is given precedence over sampling rate configuration on attribute, provided it's in valid value range. +Configuration on environment variable is given precedence over sampling rate configuration on attribute, provided it's +in valid value range. === "Sampling via attribute parameter" @@ -513,9 +818,13 @@ via `SamplingRate` parameter on attribute. ## Configure Log Output Casing -By definition Powertools for AWS Lambda (.NET) outputs logging keys using **snake case** (e.g. *"function_memory_size": 128*). This allows developers using different Powertools for AWS Lambda (.NET) runtimes, to search logs across services written in languages such as Python or TypeScript. +By definition Powertools for AWS Lambda (.NET) outputs logging keys using **snake case** (e.g. *"function_memory_size": +128*). This allows developers using different Powertools for AWS Lambda (.NET) runtimes, to search logs across services +written in languages such as Python or TypeScript. -If you want to override the default behavior you can either set the desired casing through attributes, as described in the example below, or by setting the `POWERTOOLS_LOGGER_CASE` environment variable on your AWS Lambda function. Allowed values are: `CamelCase`, `PascalCase` and `SnakeCase`. +If you want to override the default behavior you can either set the desired casing through attributes, as described in +the example below, or by setting the `POWERTOOLS_LOGGER_CASE` environment variable on your AWS Lambda function. Allowed +values are: `CamelCase`, `PascalCase` and `SnakeCase`. === "Output casing via attribute parameter" @@ -584,9 +893,119 @@ Below are some output examples for different casing. } ``` -## Custom Log formatter (Bring Your Own Formatter) -You can customize the structure (keys and values) of your log entries by implementing a custom log formatter and override default log formatter using ``Logger.UseFormatter`` method. You can implement a custom log formatter by inheriting the ``ILogFormatter`` class and implementing the ``object FormatLogEntry(LogEntry logEntry)`` method. +## Advanced + +### Log Levels + +The default log level is `Information` and can be set using the `MinimumLogLevel` property option or by using the `POWERTOOLS_LOG_LEVEL` environment variable. + +We support the following log levels: + +| Level | Numeric value | Lambda Level | +|---------------|---------------|--------------| +| `Trace` | 0 | `trace` | +| `Debug` | 1 | `debug` | +| `Information` | 2 | `info` | +| `Warning` | 3 | `warn` | +| `Error` | 4 | `error` | +| `Critical` | 5 | `fatal` | +| `None` | 6 | | + +### Using AWS Lambda Advanced Logging Controls (ALC) + +!!! question "When is it useful?" +When you want to set a logging policy to drop informational or verbose logs for one or all AWS Lambda functions, +regardless of runtime and logger used. + +With [AWS Lambda Advanced Logging Controls (ALC)](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-advanced) +{target="_blank"}, you can enforce a minimum log level that Lambda will accept from your application code. + +When enabled, you should keep `Logger` and ALC log level in sync to avoid data loss. + +!!! warning "When using AWS Lambda Advanced Logging Controls (ALC)" +- When Powertools Logger output is set to `PascalCase` **`Level`** property name will be replaced by **`LogLevel`** as + a property name. +- ALC takes precedence over **`POWERTOOLS_LOG_LEVEL`** and when setting it in code using **`[Logging(LogLevel = )]`** + +Here's a sequence diagram to demonstrate how ALC will drop both `Information` and `Debug` logs emitted from `Logger`, +when ALC log level is stricter than `Logger`. + +```mermaid +sequenceDiagram + title Lambda ALC allows WARN logs only + participant Lambda service + participant Lambda function + participant Application Logger + + Note over Lambda service: AWS_LAMBDA_LOG_LEVEL="WARN" + Note over Application Logger: POWERTOOLS_LOG_LEVEL="DEBUG" + Lambda service->>Lambda function: Invoke (event) + Lambda function->>Lambda function: Calls handler + Lambda function->>Application Logger: Logger.Warning("Something happened") + Lambda function-->>Application Logger: Logger.Debug("Something happened") + Lambda function-->>Application Logger: Logger.Information("Something happened") + + Lambda service->>Lambda service: DROP INFO and DEBUG logs + + Lambda service->>CloudWatch Logs: Ingest error logs +``` + +**Priority of log level settings in Powertools for AWS Lambda** + +We prioritise log level settings in this order: + +1. AWS_LAMBDA_LOG_LEVEL environment variable +2. Setting the log level in code using `[Logging(LogLevel = )]` +3. POWERTOOLS_LOG_LEVEL environment variable + +If you set `Logger` level lower than ALC, we will emit a warning informing you that your messages will be discarded by +Lambda. + +> **NOTE** +> With ALC enabled, we are unable to increase the minimum log level below the `AWS_LAMBDA_LOG_LEVEL` environment +> variable value, +> see [AWS Lambda service documentation](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-log-level) +> {target="_blank"} for more details. + +### Using JsonSerializerOptions + +Powertools supports customizing the serialization and deserialization of Lambda JSON events and your own types using +`JsonSerializerOptions`. +You can do this by creating a custom `JsonSerializerOptions` and passing it to the `JsonOptions` of the Powertools +Logger. + +Supports `TypeInfoResolver` and `DictionaryKeyPolicy` options. These two options are the most common ones used to +customize the serialization of Powertools Logger. + +- `TypeInfoResolver`: This option allows you to specify a custom `JsonSerializerContext` that contains the types you + want to serialize and deserialize. This is especially useful when using AOT compilation, as it allows you to specify + the types that should be included in the generated assembly. +- `DictionaryKeyPolicy`: This option allows you to specify a custom naming policy for the properties in the JSON output. + This is useful when you want to change the casing of the property names or use a different naming convention. + +!!! info +If you want to preserve the original casing of the property names (keys), you can set the `DictionaryKeyPolicy` to +`null`. + +```csharp +builder.Logging.AddPowertoolsLogger(options => +{ + options.JsonOptions = new JsonSerializerOptions + { + DictionaryKeyPolicy = JsonNamingPolicy.CamelCase, // Override output casing + TypeInfoResolver = MyCustomJsonSerializerContext.Default // Your custom JsonSerializerContext + }; +}); +``` + +### Custom Log formatter (Bring Your Own Formatter) + +You can customize the structure (keys and values) of your log entries by implementing a custom log formatter and +override default log formatter using ``LogFormatter`` property in the `configure` options. + +You can implement a custom log formatter by +inheriting the ``ILogFormatter`` class and implementing the ``object FormatLogEntry(LogEntry logEntry)`` method. === "Function.cs" @@ -601,7 +1020,10 @@ You can customize the structure (keys and values) of your log entries by impleme /// public Function() { - Logger.UseFormatter(new CustomLogFormatter()); + Logger.Configure(options => + { + options.LogFormatter = new CustomLogFormatter(); + }); } [Logging(CorrelationIdPath = "/headers/my_request_id_header", SamplingRate = 0.7)] @@ -612,6 +1034,7 @@ You can customize the structure (keys and values) of your log entries by impleme } } ``` + === "CustomLogFormatter.cs" ```c# @@ -676,21 +1099,332 @@ You can customize the structure (keys and values) of your log entries by impleme } ``` +### Buffering logs + +Log buffering enables you to buffer logs for a specific request or invocation. Enable log buffering by passing `LogBufferingOptions` when configuring a Logger instance. You can buffer logs at the `Warning`, `Information`, `Debug` or `Trace` level, and flush them automatically on error or manually as needed. + +!!! tip "This is useful when you want to reduce the number of log messages emitted while still having detailed logs when needed, such as when troubleshooting issues." + +=== "LogBufferingOptions" + + ```csharp hl_lines="5-14" + public class Function + { + public Function() + { + Logger.Configure(logger => + { + logger.Service = "MyServiceName"; + logger.LogBuffering = new LogBufferingOptions + { + BufferAtLogLevel = LogLevel.Debug, + MaxBytes = 20480, // Default is 20KB (20480 bytes) + FlushOnErrorLog = true // default true + }; + }); + + Logger.LogDebug('This is a debug message'); // This is NOT buffered + } + + [Logging] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Logger.LogDebug('This is a debug message'); // This is buffered + Logger.LogInformation('This is an info message'); + + // your business logic here + + Logger.LogError('This is an error message'); // This also flushes the buffer + } + } + + ``` + +#### Configuring the buffer + +When configuring the buffer, you can set the following options to fine-tune how logs are captured, stored, and emitted. You can configure the following options in the `logBufferOptions` constructor parameter: + +| Parameter | Description | Configuration | Default | +|---------------------|------------------------------------------------- |--------------------------------------------|---------| +| `MaxBytes` | Maximum size of the log buffer in bytes | `number` | `20480` | +| `BufferAtLogLevel` | Minimum log level to buffer | `Trace`, `Debug`, `Information`, `Warning` | `Debug` | +| `FlushOnErrorLog` | Automatically flush buffer when logging an error | `True`, `False` | `True` | + +=== "BufferAtLogLevel" + + ```csharp hl_lines="10" + public class Function + { + public Function() + { + Logger.Configure(logger => + { + logger.Service = "MyServiceName"; + logger.LogBuffering = new LogBufferingOptions + { + BufferAtLogLevel = LogLevel.Warning + }; + }); + } + + [Logging] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + // All logs below are buffered + Logger.LogDebug('This is a debug message'); + Logger.LogInformation('This is an info message'); + Logger.LogWarning('This is a warn message'); + + Logger.ClearBuffer(); // This will clear the buffer without emitting the logs + } + } + ``` + + 1. Setting `BufferAtLogLevel: 'Warning'` configures log buffering for `Warning` and all lower severity levels like `Information`, `Debug`, and `Trace`. + 2. Calling `Logger.ClearBuffer()` will clear the buffer without emitting the logs. + +=== "FlushOnErrorLog" + + ```csharp hl_lines="10" + public class Function + { + public Function() + { + Logger.Configure(logger => + { + logger.Service = "MyServiceName"; + logger.LogBuffering = new LogBufferingOptions + { + FlushOnErrorLog = false + }; + }); + } + + [Logging] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Logger.LogDebug('This is a debug message'); // this is buffered + + try + { + throw new Exception(); + } + catch (Exception e) + { + Logger.LogError(e.Message); // this does NOT flush the buffer + } + + Logger.LogDebug("Debug!!"); // this is buffered + + try + { + throw new Exception(); + } + catch (Exception e) + { + Logger.LogError(e.Message); // this does NOT flush the buffer + Logger.FlushBuffer(); // Manually flush + } + } + } + ``` + + 1. Disabling `FlushOnErrorLog` will not flush the buffer when logging an error. This is useful when you want to control when the buffer is flushed by calling the `Logger.FlushBuffer()` method. + +#### Flushing on errors + +When using the `Logger` decorator, you can configure the logger to automatically flush the buffer when an error occurs. This is done by setting the `FlushBufferOnUncaughtError` option to `true` in the decorator. + +=== "FlushBufferOnUncaughtError" + + ```csharp hl_lines="15" + public class Function + { + public Function() + { + Logger.Configure(logger => + { + logger.Service = "MyServiceName"; + logger.LogBuffering = new LogBufferingOptions + { + BufferAtLogLevel = LogLevel.Debug + }; + }); + } + + [Logging(FlushBufferOnUncaughtError = true)] + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Logger.LogDebug('This is a debug message'); + + throw new Exception(); // This causes the buffer to be flushed + } + } + ``` + +#### Buffering workflows + +##### Manual flush + +
+```mermaid +sequenceDiagram + participant Client + participant Lambda + participant Logger + participant CloudWatch + Client->>Lambda: Invoke Lambda + Lambda->>Logger: Initialize with DEBUG level buffering + Logger-->>Lambda: Logger buffer ready + Lambda->>Logger: Logger.LogDebug("First debug log") + Logger-->>Logger: Buffer first debug log + Lambda->>Logger: Logger.LogInformation("Info log") + Logger->>CloudWatch: Directly log info message + Lambda->>Logger: Logger.LogDebug("Second debug log") + Logger-->>Logger: Buffer second debug log + Lambda->>Logger: Logger.FlushBuffer() + Logger->>CloudWatch: Emit buffered logs to stdout + Lambda->>Client: Return execution result +``` +Flushing buffer manually +
+ +##### Flushing when logging an error + +
+```mermaid +sequenceDiagram + participant Client + participant Lambda + participant Logger + participant CloudWatch + Client->>Lambda: Invoke Lambda + Lambda->>Logger: Initialize with DEBUG level buffering + Logger-->>Lambda: Logger buffer ready + Lambda->>Logger: Logger.LogDebug("First log") + Logger-->>Logger: Buffer first debug log + Lambda->>Logger: Logger.LogDebug("Second log") + Logger-->>Logger: Buffer second debug log + Lambda->>Logger: Logger.LogDebug("Third log") + Logger-->>Logger: Buffer third debug log + Lambda->>Lambda: Exception occurs + Lambda->>Logger: Logger.LogError("Error details") + Logger->>CloudWatch: Emit buffered debug logs + Logger->>CloudWatch: Emit error log + Lambda->>Client: Raise exception +``` +Flushing buffer when an error happens +
+ +##### Flushing on error + +This works only when using the `Logger` decorator. You can configure the logger to automatically flush the buffer when an error occurs by setting the `FlushBufferOnUncaughtError` option to `true` in the decorator. + +
+```mermaid +sequenceDiagram + participant Client + participant Lambda + participant Logger + participant CloudWatch + Client->>Lambda: Invoke Lambda + Lambda->>Logger: Using decorator + Logger-->>Lambda: Logger context injected + Lambda->>Logger: Logger.LogDebug("First log") + Logger-->>Logger: Buffer first debug log + Lambda->>Logger: Logger.LogDebug("Second log") + Logger-->>Logger: Buffer second debug log + Lambda->>Lambda: Uncaught Exception + Lambda->>CloudWatch: Automatically emit buffered debug logs + Lambda->>Client: Raise uncaught exception +``` +Flushing buffer when an uncaught exception happens +
+ +#### Buffering FAQs + +1. **Does the buffer persist across Lambda invocations?** + No, each Lambda invocation has its own buffer. The buffer is initialized when the Lambda function is invoked and is cleared after the function execution completes or when flushed manually. + +2. **Are my logs buffered during cold starts?** + No, we never buffer logs during cold starts. This is because we want to ensure that logs emitted during this phase are always available for debugging and monitoring purposes. The buffer is only used during the execution of the Lambda function. + +3. **How can I prevent log buffering from consuming excessive memory?** + You can limit the size of the buffer by setting the `MaxBytes` option in the `LogBufferingOptions` constructor parameter. This will ensure that the buffer does not grow indefinitely and consume excessive memory. + +4. **What happens if the log buffer reaches its maximum size?** + Older logs are removed from the buffer to make room for new logs. This means that if the buffer is full, you may lose some logs if they are not flushed before the buffer reaches its maximum size. When this happens, we emit a warning when flushing the buffer to indicate that some logs have been dropped. + +5. **How is the log size of a log line calculated?** + The log size is calculated based on the size of the serialized log line in bytes. This includes the size of the log message, the size of any additional keys, and the size of the timestamp. + +6. **What timestamp is used when I flush the logs?** + The timestamp preserves the original time when the log record was created. If you create a log record at 11:00:10 and flush it at 11:00:25, the log line will retain its original timestamp of 11:00:10. + +7. **What happens if I try to add a log line that is bigger than max buffer size?** + The log will be emitted directly to standard output and not buffered. When this happens, we emit a warning to indicate that the log line was too big to be buffered. + +8. **What happens if Lambda times out without flushing the buffer?** + Logs that are still in the buffer will be lost. If you are using the log buffer to log asynchronously, you should ensure that the buffer is flushed before the Lambda function times out. You can do this by calling the `Logger.FlushBuffer()` method at the end of your Lambda function. + +### Timestamp formatting + +You can customize the timestamp format by setting the `TimestampFormat` property in the `Logger.Configure` method. The default format is `o`, which is the ISO 8601 format. +You can use any valid [DateTime format string](https://docs.microsoft.com/en-us/dotnet/standard/base-types/custom-date-and-time-format-strings) to customize the timestamp format. +For example, to use the `yyyy-MM-dd HH:mm:ss` format, you can do the following: + +```csharp +Logger.Configure(logger => +{ + logger.TimestampFormat = "yyyy-MM-dd HH:mm:ss"; +}); +``` +This will output the timestamp in the following format: + +```json +{ + "level": "Information", + "message": "Test Message", + "timestamp": "2021-12-13 20:32:22", + "service": "lambda-example", + ... +} +``` + ## AOT Support !!! info - - If you want to use the `LogEvent`, `Custom Log Formatter` features, or serialize your own types when Logging events, you need to make changes in your Lambda `Main` method. + + If you want to use the `LogEvent`, `Custom Log Formatter` features, or serialize your own types when Logging events, you need to either pass `JsonSerializerContext` or make changes in your Lambda `Main` method. !!! info Starting from version 1.6.0, it is required to update the Amazon.Lambda.Serialization.SystemTextJson NuGet package to version 2.4.3 in your csproj. -### Configure +### Using JsonSerializerOptions + +To be able to serializer your own types, you need to pass your `JsonSerializerContext` to the `TypeInfoResolver` of the `Logger.Configure` method. + +```csharp +Logger.Configure(logger => +{ + logger.JsonOptions = new JsonSerializerOptions + { + TypeInfoResolver = YourJsonSerializerContext.Default + }; +}); +``` + +### Using PowertoolsSourceGeneratorSerializer Replace `SourceGeneratorLambdaJsonSerializer` with `PowertoolsSourceGeneratorSerializer`. -This change enables Powertools to construct an instance of `JsonSerializerOptions` used to customize the serialization and deserialization of Lambda JSON events and your own types. +This change enables Powertools to construct an instance of `JsonSerializerOptions` used to customize the serialization +and deserialization of Lambda JSON events and your own types. === "Before" @@ -710,7 +1444,7 @@ This change enables Powertools to construct an instance of `JsonSerializerOption .RunAsync(); ``` -For example when you have your own Demo type +For example when you have your own Demo type ```csharp public class Demo @@ -731,11 +1465,14 @@ public partial class MyCustomJsonSerializerContext : JsonSerializerContext } ``` -When you update your code to use `PowertoolsSourceGeneratorSerializer`, we combine your `JsonSerializerContext` with Powertools' `JsonSerializerContext`. This allows Powertools to serialize your types and Lambda events. +When you update your code to use `PowertoolsSourceGeneratorSerializer`, we combine your +`JsonSerializerContext` with Powertools' `JsonSerializerContext`. This allows Powertools to serialize your types and +Lambda events. ### Custom Log Formatter -To use a custom log formatter with AOT, pass an instance of `ILogFormatter` to `PowertoolsSourceGeneratorSerializer` instead of using the static `Logger.UseFormatter` in the Function constructor as you do in non-AOT Lambdas. +To use a custom log formatter with AOT, pass an instance of `ILogFormatter` to `PowertoolsSourceGeneratorSerializer` +instead of using the static `Logger.UseFormatter` in the Function constructor as you do in non-AOT Lambdas. === "Function Main method" @@ -797,3 +1534,99 @@ To use a custom log formatter with AOT, pass an instance of `ILogFormatter` to ` While we support anonymous type serialization by converting to a `Dictionary`, this is **not** a best practice and is **not recommended** when using native AOT. We recommend using concrete classes and adding them to your `JsonSerializerContext`. + +## Testing + +You can change where the `Logger` will output its logs by setting the `LogOutput` property. +We also provide a helper class for tests `TestLoggerOutput` or you can provider your own implementation of `IConsoleWrapper`. + +```csharp +Logger.Configure(options => +{ + // Using TestLoggerOutput + options.LogOutput = new TestLoggerOutput(); + // Custom console output for testing + options.LogOutput = new TestConsoleWrapper(); +}); + +// Example implementation for testing: +public class TestConsoleWrapper : IConsoleWrapper +{ + public List CapturedOutput { get; } = new(); + + public void WriteLine(string message) + { + CapturedOutput.Add(message); + } +} +``` +```csharp +// Test example +[Fact] +public void When_Setting_Service_Should_Update_Key() +{ + // Arrange + var consoleOut = new TestLoggerOutput(); + Logger.Configure(options => + { + options.LogOutput = consoleOut; + }); + + // Act + _testHandlers.HandlerService(); + + // Assert + + var st = consoleOut.ToString(); + + Assert.Contains("\"level\":\"Information\"", st); + Assert.Contains("\"service\":\"test\"", st); + Assert.Contains("\"name\":\"AWS.Lambda.Powertools.Logging.Logger\"", st); + Assert.Contains("\"message\":\"test\"", st); +} +``` + +### ILogger + +If you are using ILogger interface you can inject the logger in a dedicated constructor for your Lambda function and thus you can mock your ILogger instance. + +```csharp +public class Function +{ + private readonly ILogger _logger; + + public Function() + { + _logger = oggerFactory.Create(builder => + { + builder.AddPowertoolsLogger(config => + { + config.Service = "TestService"; + config.LoggerOutputCase = LoggerOutputCase.PascalCase; + }); + }).CreatePowertoolsLogger(); + } + + // constructor used for tests - pass the mock ILogger + public Function(ILogger logger) + { + _logger = logger ?? loggerFactory.Create(builder => + { + builder.AddPowertoolsLogger(config => + { + config.Service = "TestService"; + config.LoggerOutputCase = LoggerOutputCase.PascalCase; + }); + }).CreatePowertoolsLogger(); + } + + public async Task FunctionHandler + (APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + _logger.LogInformation("Collecting payment"); + ... + } +} +``` + + diff --git a/docs/core/metrics-v1.md b/docs/core/metrics-v1.md new file mode 100644 index 000000000..7ed992637 --- /dev/null +++ b/docs/core/metrics-v1.md @@ -0,0 +1,416 @@ +--- +title: Metrics v1 - Legacy +description: Core utility +--- + +!!! warning + Version 1.x.x will continue to be supported **until end of October 2025** for bug fixes and security updates, but no new features will be added to this version. We recommend you upgrade to the latest version. + + The latest version is available at [Metrics v2](https://docs.powertools.aws.dev/lambda/dotnet/core/metrics-v2/). + + +Metrics creates custom metrics asynchronously by logging metrics to standard output following [Amazon CloudWatch Embedded Metric Format (EMF)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html). + +These metrics can be visualized through [Amazon CloudWatch Console](https://aws.amazon.com/cloudwatch/). + +## Key features + +* Aggregate up to 100 metrics using a single [CloudWatch EMF](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html){target="_blank"} object (large JSON blob) +* Validating your metrics against common metric definitions mistakes (for example, metric unit, values, max dimensions, max metrics) +* Metrics are created asynchronously by the CloudWatch service. You do not need any custom stacks, and there is no impact to Lambda function latency +* Context manager to create a one off metric with a different dimension +* Ahead-of-Time compilation to native code support [AOT](https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html) from version 1.7.0 + +
+ +
+ Screenshot of the Amazon CloudWatch Console showing an example of business metrics in the Metrics Explorer +
Metrics showcase - Metrics Explorer
+
+ +## Installation + +Powertools for AWS Lambda (.NET) are available as NuGet packages. You can install the packages from [NuGet Gallery](https://www.nuget.org/packages?q=AWS+Lambda+Powertools*){target="_blank"} or from Visual Studio editor by searching `AWS.Lambda.Powertools*` to see various utilities available. + +* [AWS.Lambda.Powertools.Metrics](https://www.nuget.org/packages?q=AWS.Lambda.Powertools.Metrics): + + `dotnet add package AWS.Lambda.Powertools.Metrics -v 1.7.1` + +## Terminologies + +If you're new to Amazon CloudWatch, there are two terminologies you must be aware of before using this utility: + +* **Namespace**. It's the highest level container that will group multiple metrics from multiple services for a given application, for example `ServerlessEcommerce`. +* **Dimensions**. Metrics metadata in key-value format. They help you slice and dice metrics visualization, for example `ColdStart` metric by Payment `service`. +* **Metric**. It's the name of the metric, for example: SuccessfulBooking or UpdatedBooking. +* **Unit**. It's a value representing the unit of measure for the corresponding metric, for example: Count or Seconds. +* **Resolution**. It's a value representing the storage resolution for the corresponding metric. Metrics can be either Standard or High resolution. Read more [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Resolution_definition). + +Visit the AWS documentation for a complete explanation for [Amazon CloudWatch concepts](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html). + +
+ +
Metric terminology, visually explained
+
+ +## Getting started + +**`Metrics`** is implemented as a Singleton to keep track of your aggregate metrics in memory and make them accessible anywhere in your code. To guarantee that metrics are flushed properly the **`MetricsAttribute`** must be added on the lambda handler. + +Metrics has two global settings that will be used across all metrics emitted. Use your application or main service as the metric namespace to easily group all metrics: + +Setting | Description | Environment variable | Constructor parameter +------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- +**Service** | Optionally, sets **service** metric dimension across all metrics e.g. `payment` | `POWERTOOLS_SERVICE_NAME` | `Service` +**Metric namespace** | Logical container where all metrics will be placed e.g. `MyCompanyEcommerce` | `POWERTOOLS_METRICS_NAMESPACE` | `Namespace` + +!!! info "Autocomplete Metric Units" + All parameters in **`Metrics Attribute`** are optional. Following rules apply: + + - **Namespace:** **`Empty`** string by default. You can either specify it in code or environment variable. If not present before flushing metrics, a **`SchemaValidationException`** will be thrown. + - **Service:** **`service_undefined`** by default. You can either specify it in code or environment variable. + - **CaptureColdStart:** **`false`** by default. + - **RaiseOnEmptyMetrics:** **`false`** by default. + +### Example using AWS Serverless Application Model (AWS SAM) + +=== "template.yml" + + ```yaml hl_lines="9 10" + Resources: + HelloWorldFunction: + Type: AWS::Serverless::Function + Properties: + ... + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: ShoppingCartService + POWERTOOLS_METRICS_NAMESPACE: MyCompanyEcommerce + ``` + +=== "Function.cs" + + ```csharp hl_lines="4" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + [Metrics(Namespace = "MyCompanyEcommerce", Service = "ShoppingCartService", CaptureColdStart = true, RaiseOnEmptyMetrics = true)] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + } + } + ``` + +### Full list of environment variables + +| Environment variable | Description | Default | +| ------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- | +| **POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | `"service_undefined"` | +| **POWERTOOLS_METRICS_NAMESPACE** | Sets namespace used for metrics | `None` | + +### Creating metrics + +You can create metrics using **`AddMetric`**, and you can create dimensions for all your aggregate metrics using **`AddDimension`** method. + +=== "Metrics" + + ```csharp hl_lines="5 8" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(Namespace = "ExampleApplication", Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + } + } + ``` +=== "Metrics with custom dimensions" + + ```csharp hl_lines="8-9" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(Namespace = "ExampleApplication", Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.AddDimension("Environment","Prod"); + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + } + } + ``` + +!!! tip "Autocomplete Metric Units" + `MetricUnit` enum facilitates finding a supported metric unit by CloudWatch. + +!!! note "Metrics overflow" + CloudWatch EMF supports a max of 100 metrics per batch. Metrics utility will flush all metrics when adding the 100th metric. Subsequent metrics, e.g. 101th, will be aggregated into a new EMF object, for your convenience. + +!!! warning "Metric value must be a positive number" + Metric values must be a positive number otherwise an `ArgumentException` will be thrown. + +!!! warning "Do not create metrics or dimensions outside the handler" + Metrics or dimensions added in the global scope will only be added during cold start. Disregard if that's the intended behavior. + +### Adding high-resolution metrics + +You can create [high-resolution metrics](https://aws.amazon.com/about-aws/whats-new/2023/02/amazon-cloudwatch-high-resolution-metric-extraction-structured-logs/) passing `MetricResolution` as parameter to `AddMetric`. + +!!! tip "When is it useful?" + High-resolution metrics are data with a granularity of one second and are very useful in several situations such as telemetry, time series, real-time incident management, and others. + +=== "Metrics with high resolution" + + ```csharp hl_lines="9 12 15" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(Namespace = "ExampleApplication", Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + // Publish a metric with standard resolution i.e. StorageResolution = 60 + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count, MetricResolution.Standard); + + // Publish a metric with high resolution i.e. StorageResolution = 1 + Metrics.AddMetric("FailedBooking", 1, MetricUnit.Count, MetricResolution.High); + + // The last parameter (storage resolution) is optional + Metrics.AddMetric("SuccessfulUpgrade", 1, MetricUnit.Count); + } + } + ``` + +!!! tip "Autocomplete Metric Resolutions" + Use the `MetricResolution` enum to easily find a supported metric resolution by CloudWatch. + +### Adding default dimensions + +You can use **`SetDefaultDimensions`** method to persist dimensions across Lambda invocations. + +=== "SetDefaultDimensions method" + + ```csharp hl_lines="4 5 6 7 12" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + private Dictionary _defaultDimensions = new Dictionary{ + {"Environment", "Prod"}, + {"Another", "One"} + }; + + [Metrics(Namespace = "ExampleApplication", Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.SetDefaultDimensions(_defaultDimensions); + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + } + } + ``` + +### Flushing metrics + +With **`MetricsAttribute`** all your metrics are validated, serialized and flushed to standard output when lambda handler completes execution or when you had the 100th metric to memory. + +During metrics validation, if no metrics are provided then a warning will be logged, but no exception will be raised. + +=== "Function.cs" + + ```csharp hl_lines="8" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(Namespace = "ExampleApplication", Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + } + } + ``` +=== "Example CloudWatch Logs excerpt" + + ```json hl_lines="2 7 10 15 22" + { + "BookingConfirmation": 1.0, + "_aws": { + "Timestamp": 1592234975665, + "CloudWatchMetrics": [ + { + "Namespace": "ExampleApplication", + "Dimensions": [ + [ + "service" + ] + ], + "Metrics": [ + { + "Name": "BookingConfirmation", + "Unit": "Count" + } + ] + } + ] + }, + "service": "ExampleService" + } + ``` + +!!! tip "Metric validation" + If metrics are provided, and any of the following criteria are not met, **`SchemaValidationException`** will be raised: + + * Maximum of 9 dimensions + * Namespace is set + * Metric units must be [supported by CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html) + +!!! info "We do not emit 0 as a value for ColdStart metric for cost reasons. [Let us know](https://github.com/aws-powertools/powertools-lambda-dotnet/issues/new?assignees=&labels=feature-request%2Ctriage&template=feature_request.yml&title=Feature+request%3A+TITLE) if you'd prefer a flag to override it" + +#### Raising SchemaValidationException on empty metrics + +If you want to ensure that at least one metric is emitted, you can pass **`RaiseOnEmptyMetrics`** to the Metrics attribute: + +=== "Function.cs" + + ```python hl_lines="5" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(RaiseOnEmptyMetrics = true)] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + ``` + +### Capturing cold start metric + +You can optionally capture cold start metrics by setting **`CaptureColdStart`** parameter to `true`. + +=== "Function.cs" + + ```csharp hl_lines="5" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(CaptureColdStart = true)] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + ... + ``` + +If it's a cold start invocation, this feature will: + +* Create a separate EMF blob solely containing a metric named `ColdStart` +* Add `function_name` and `service` dimensions + +This has the advantage of keeping cold start metric separate from your application metrics, where you might have unrelated dimensions. + +## Advanced + +### Adding metadata + +You can add high-cardinality data as part of your Metrics log with `AddMetadata` method. This is useful when you want to search highly contextual information along with your metrics in your logs. + +!!! info + **This will not be available during metrics visualization** - Use **dimensions** for this purpose + +!!! info + Adding metadata with a key that is the same as an existing metric will be ignored + +=== "Function.cs" + + ```csharp hl_lines="9" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(Namespace = ExampleApplication, Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + Metrics.AddMetadata("BookingId", "683EEB2D-B2F3-4075-96EE-788E6E2EED45"); + ... + ``` + +=== "Example CloudWatch Logs excerpt" + + ```json hl_lines="23" + { + "SuccessfulBooking": 1.0, + "_aws": { + "Timestamp": 1592234975665, + "CloudWatchMetrics": [ + { + "Namespace": "ExampleApplication", + "Dimensions": [ + [ + "service" + ] + ], + "Metrics": [ + { + "Name": "SuccessfulBooking", + "Unit": "Count" + } + ] + } + ] + }, + "Service": "Booking", + "BookingId": "683EEB2D-B2F3-4075-96EE-788E6E2EED45" + } + ``` + +### Single metric with a different dimension + +CloudWatch EMF uses the same dimensions across all your metrics. Use **`PushSingleMetric`** if you have a metric that should have different dimensions. + +!!! info + Generally, this would be an edge case since you [pay for unique metric](https://aws.amazon.com/cloudwatch/pricing). Keep the following formula in mind: + + **unique metric = (metric_name + dimension_name + dimension_value)** + +=== "Function.cs" + + ```csharp hl_lines="8-17" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(Namespace = ExampleApplication, Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.PushSingleMetric( + metricName: "ColdStart", + value: 1, + unit: MetricUnit.Count, + nameSpace: "ExampleApplication", + service: "Booking", + defaultDimensions: new Dictionary + { + {"FunctionContext", "$LATEST"} + }); + ... + ``` + +## Testing your code + +### Environment variables + +???+ tip + Ignore this section, if: + + * You are explicitly setting namespace/default dimension via `namespace` and `service` parameters + * You're not instantiating `Metrics` in the global namespace + + For example, `Metrics(namespace="ExampleApplication", service="booking")` + +Make sure to set `POWERTOOLS_METRICS_NAMESPACE` and `POWERTOOLS_SERVICE_NAME` before running your tests to prevent failing on `SchemaValidation` exception. You can set it before you run tests by adding the environment variable. + +```csharp title="Injecting Metric Namespace before running tests" +Environment.SetEnvironmentVariable("POWERTOOLS_METRICS_NAMESPACE","AWSLambdaPowertools"); +``` diff --git a/docs/core/metrics-v2.md b/docs/core/metrics-v2.md deleted file mode 100644 index c3d216c3e..000000000 --- a/docs/core/metrics-v2.md +++ /dev/null @@ -1,1000 +0,0 @@ ---- -title: Metrics V2 -description: Core utility ---- - -Metrics creates custom metrics asynchronously by logging metrics to standard output following [Amazon CloudWatch Embedded Metric Format (EMF)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html). - -These metrics can be visualized through [Amazon CloudWatch Console](https://aws.amazon.com/cloudwatch/). - -## Key features - -* Aggregate up to 100 metrics using a single [CloudWatch EMF](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html){target="_blank"} object (large JSON blob) -* Validating your metrics against common metric definitions mistakes (for example, metric unit, values, max dimensions, max metrics) -* Metrics are created asynchronously by the CloudWatch service. You do not need any custom stacks, and there is no impact to Lambda function latency -* Context manager to create a one off metric with a different dimension -* Ahead-of-Time compilation to native code support [AOT](https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html) from version 1.7.0 -* Support for AspNetCore middleware and filters to capture metrics for HTTP requests - -## Breaking changes from V1 - -* **`Dimensions`** outputs as an array of arrays instead of an array of objects. Example: `Dimensions: [["service", "Environment"]]` instead of `Dimensions: ["service", "Environment"]` -* **`FunctionName`** is not added as default dimension and only to cold start metric. -* **`Default Dimensions`** can now be included in Cold Start metrics, this is a potential breaking change if you were relying on the absence of default dimensions in Cold Start metrics when searching. - -
- -
- Screenshot of the Amazon CloudWatch Console showing an example of business metrics in the Metrics Explorer -
Metrics showcase - Metrics Explorer
-
- -## Installation - -Powertools for AWS Lambda (.NET) are available as NuGet packages. You can install the packages from [NuGet Gallery](https://www.nuget.org/packages?q=AWS+Lambda+Powertools*){target="_blank"} or from Visual Studio editor by searching `AWS.Lambda.Powertools*` to see various utilities available. - -* [AWS.Lambda.Powertools.Metrics](https://www.nuget.org/packages?q=AWS.Lambda.Powertools.Metrics): - - `dotnet add package AWS.Lambda.Powertools.Metrics` - -## Terminologies - -If you're new to Amazon CloudWatch, there are two terminologies you must be aware of before using this utility: - -* **Namespace**. It's the highest level container that will group multiple metrics from multiple services for a given application, for example `ServerlessEcommerce`. -* **Dimensions**. Metrics metadata in key-value format. They help you slice and dice metrics visualization, for example `ColdStart` metric by Payment `service`. -* **Metric**. It's the name of the metric, for example: SuccessfulBooking or UpdatedBooking. -* **Unit**. It's a value representing the unit of measure for the corresponding metric, for example: Count or Seconds. -* **Resolution**. It's a value representing the storage resolution for the corresponding metric. Metrics can be either Standard or High resolution. Read more [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Resolution_definition). - -Visit the AWS documentation for a complete explanation for [Amazon CloudWatch concepts](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html). - -
- -
Metric terminology, visually explained
-
- -## Getting started - -**`Metrics`** is implemented as a Singleton to keep track of your aggregate metrics in memory and make them accessible anywhere in your code. To guarantee that metrics are flushed properly the **`MetricsAttribute`** must be added on the lambda handler. - -Metrics has three global settings that will be used across all metrics emitted. Use your application or main service as the metric namespace to easily group all metrics: - - Setting | Description | Environment variable | Decorator parameter --------------------------------|---------------------------------------------------------------------------------| ------------------------------------------------- |----------------------- - **Metric namespace** | Logical container where all metrics will be placed e.g. `MyCompanyEcommerce` | `POWERTOOLS_METRICS_NAMESPACE` | `Namespace` - **Service** | Optionally, sets **Service** metric dimension across all metrics e.g. `payment` | `POWERTOOLS_SERVICE_NAME` | `Service` -**Disable Powertools Metrics** | Optionally, disables all Powertools metrics |`POWERTOOLS_METRICS_DISABLED` | N/A | - -???+ info - `POWERTOOLS_METRICS_DISABLED` will not disable default metrics created by AWS services. - -!!! info "Autocomplete Metric Units" - All parameters in **`Metrics Attribute`** are optional. Following rules apply: - - - **Namespace:** **`Empty`** string by default. You can either specify it in code or environment variable. If not present before flushing metrics, a **`SchemaValidationException`** will be thrown. - - **Service:** **`service_undefined`** by default. You can either specify it in code or environment variable. - - **CaptureColdStart:** **`false`** by default. - - **RaiseOnEmptyMetrics:** **`false`** by default. - -### Metrics object - -#### Attribute - -The **`MetricsAttribute`** is a class-level attribute that can be used to set the namespace and service for all metrics emitted by the lambda handler. - -```csharp hl_lines="3" -using AWS.Lambda.Powertools.Metrics; - -[Metrics(Namespace = "ExampleApplication", Service = "Booking")] -public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) -{ - ... -} -``` - -#### Methods - -The **`Metrics`** class provides methods to add metrics, dimensions, and metadata to the metrics object. - -```csharp hl_lines="5-7" -using AWS.Lambda.Powertools.Metrics; - -public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) -{ - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - Metrics.AddDimension("Environment", "Prod"); - Metrics.AddMetadata("BookingId", "683EEB2D-B2F3-4075-96EE-788E6E2EED45"); - ... -} -``` - -#### Initialization - -The **`Metrics`** object is initialized as a Singleton and can be accessed anywhere in your code. - -But can also be initialize with `Configure` or `Builder` patterns in your Lambda constructor, this the best option for testing. - -Configure: - -```csharp -using AWS.Lambda.Powertools.Metrics; - -public Function() -{ - Metrics.Configure(options => - { - options.Namespace = "dotnet-powertools-test"; - options.Service = "testService"; - options.CaptureColdStart = true; - options.DefaultDimensions = new Dictionary - { - { "Environment", "Prod" }, - { "Another", "One" } - }; - }); -} - -[Metrics] -public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) -{ - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - ... -} -``` - -Builder: - -```csharp -using AWS.Lambda.Powertools.Metrics; - -private readonly IMetrics _metrics; - -public Function() -{ - _metrics = new MetricsBuilder() - .WithCaptureColdStart(true) - .WithService("testService") - .WithNamespace("dotnet-powertools-test") - .WithDefaultDimensions(new Dictionary - { - { "Environment", "Prod1" }, - { "Another", "One" } - }).Build(); -} - -[Metrics] -public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) -{ - _metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - ... -} -``` - - -### Creating metrics - -You can create metrics using **`AddMetric`**, and you can create dimensions for all your aggregate metrics using **`AddDimension`** method. - -=== "Metrics" - - ```csharp hl_lines="5 8" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(Namespace = "ExampleApplication", Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - } - } - ``` -=== "Metrics with custom dimensions" - - ```csharp hl_lines="8-9" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(Namespace = "ExampleApplication", Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.AddDimension("Environment","Prod"); - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - } - } - ``` - -!!! tip "Autocomplete Metric Units" - `MetricUnit` enum facilitates finding a supported metric unit by CloudWatch. - -!!! note "Metrics overflow" - CloudWatch EMF supports a max of 100 metrics per batch. Metrics utility will flush all metrics when adding the 100th metric. Subsequent metrics, e.g. 101th, will be aggregated into a new EMF object, for your convenience. - -!!! warning "Metric value must be a positive number" - Metric values must be a positive number otherwise an `ArgumentException` will be thrown. - -!!! warning "Do not create metrics or dimensions outside the handler" - Metrics or dimensions added in the global scope will only be added during cold start. Disregard if that's the intended behavior. - -### Adding high-resolution metrics - -You can create [high-resolution metrics](https://aws.amazon.com/about-aws/whats-new/2023/02/amazon-cloudwatch-high-resolution-metric-extraction-structured-logs/) passing `MetricResolution` as parameter to `AddMetric`. - -!!! tip "When is it useful?" - High-resolution metrics are data with a granularity of one second and are very useful in several situations such as telemetry, time series, real-time incident management, and others. - -=== "Metrics with high resolution" - - ```csharp hl_lines="9 12 15" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(Namespace = "ExampleApplication", Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - // Publish a metric with standard resolution i.e. StorageResolution = 60 - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count, MetricResolution.Standard); - - // Publish a metric with high resolution i.e. StorageResolution = 1 - Metrics.AddMetric("FailedBooking", 1, MetricUnit.Count, MetricResolution.High); - - // The last parameter (storage resolution) is optional - Metrics.AddMetric("SuccessfulUpgrade", 1, MetricUnit.Count); - } - } - ``` - -!!! tip "Autocomplete Metric Resolutions" - Use the `MetricResolution` enum to easily find a supported metric resolution by CloudWatch. - -### Adding default dimensions - -You can use **`SetDefaultDimensions`** method to persist dimensions across Lambda invocations. - -=== "SetDefaultDimensions method" - - ```csharp hl_lines="4 5 6 7 12" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - private Dictionary _defaultDimensions = new Dictionary{ - {"Environment", "Prod"}, - {"Another", "One"} - }; - - [Metrics(Namespace = "ExampleApplication", Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.SetDefaultDimensions(_defaultDimensions); - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - } - } - ``` - -### Adding default dimensions with cold start metric - -You can use the Builder or Configure patterns in your Lambda class constructor to set default dimensions. - -=== "Builder pattern" - - ```csharp hl_lines="12-16" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - private readonly IMetrics _metrics; - - public Function() - { - _metrics = new MetricsBuilder() - .WithCaptureColdStart(true) - .WithService("testService") - .WithNamespace("dotnet-powertools-test") - .WithDefaultDimensions(new Dictionary - { - { "Environment", "Prod1" }, - { "Another", "One" } - }).Build(); - } - - [Metrics] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - _metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - ... - } - ``` -=== "Configure pattern" - - ```csharp hl_lines="12-16" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - public Function() - { - Metrics.Configure(options => - { - options.Namespace = "dotnet-powertools-test"; - options.Service = "testService"; - options.CaptureColdStart = true; - options.DefaultDimensions = new Dictionary - { - { "Environment", "Prod" }, - { "Another", "One" } - }; - }); - } - - [Metrics] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - ... - } - ``` -### Adding dimensions - -You can add dimensions to your metrics using **`AddDimension`** method. - -=== "Function.cs" - - ```csharp hl_lines="8" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(Namespace = "ExampleApplication", Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.AddDimension("Environment","Prod"); - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - } - } - ``` -=== "Example CloudWatch Logs excerpt" - - ```json hl_lines="11 24" - { - "SuccessfulBooking": 1.0, - "_aws": { - "Timestamp": 1592234975665, - "CloudWatchMetrics": [ - { - "Namespace": "ExampleApplication", - "Dimensions": [ - [ - "service", - "Environment" - ] - ], - "Metrics": [ - { - "Name": "SuccessfulBooking", - "Unit": "Count" - } - ] - } - ] - }, - "service": "ExampleService", - "Environment": "Prod" - } - ``` - -### Flushing metrics - -With **`MetricsAttribute`** all your metrics are validated, serialized and flushed to standard output when lambda handler completes execution or when you had the 100th metric to memory. - -You can also flush metrics manually by calling **`Flush`** method. - -During metrics validation, if no metrics are provided then a warning will be logged, but no exception will be raised. - -=== "Function.cs" - - ```csharp hl_lines="9" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(Namespace = "ExampleApplication", Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - Metrics.Flush(); - } - } - ``` -=== "Example CloudWatch Logs excerpt" - - ```json hl_lines="2 7 10 15 22" - { - "BookingConfirmation": 1.0, - "_aws": { - "Timestamp": 1592234975665, - "CloudWatchMetrics": [ - { - "Namespace": "ExampleApplication", - "Dimensions": [ - [ - "service" - ] - ], - "Metrics": [ - { - "Name": "BookingConfirmation", - "Unit": "Count" - } - ] - } - ] - }, - "service": "ExampleService" - } - ``` - -!!! tip "Metric validation" - If metrics are provided, and any of the following criteria are not met, **`SchemaValidationException`** will be raised: - - * Maximum of 30 dimensions - * Namespace is set - * Metric units must be [supported by CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html) - -!!! info "We do not emit 0 as a value for ColdStart metric for cost reasons. [Let us know](https://github.com/aws-powertools/powertools-lambda-dotnet/issues/new?assignees=&labels=feature-request%2Ctriage&template=feature_request.yml&title=Feature+request%3A+TITLE) if you'd prefer a flag to override it" - -### Raising SchemaValidationException on empty metrics - -If you want to ensure that at least one metric is emitted, you can pass **`RaiseOnEmptyMetrics`** to the Metrics attribute: - -=== "Function.cs" - - ```python hl_lines="5" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(RaiseOnEmptyMetrics = true)] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - ``` - -### Capturing cold start metric - -You can optionally capture cold start metrics by setting **`CaptureColdStart`** parameter to `true`. - -=== "Function.cs" - - ```csharp hl_lines="5" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(CaptureColdStart = true)] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - ``` -=== "Builder pattern" - - ```csharp hl_lines="9" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - private readonly IMetrics _metrics; - - public Function() - { - _metrics = new MetricsBuilder() - .WithCaptureColdStart(true) - .WithService("testService") - .WithNamespace("dotnet-powertools-test") - } - - [Metrics] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - _metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - ... - } - ``` -=== "Configure pattern" - - ```csharp hl_lines="11" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - public Function() - { - Metrics.Configure(options => - { - options.Namespace = "dotnet-powertools-test"; - options.Service = "testService"; - options.CaptureColdStart = true; - }); - } - - [Metrics] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - ... - } - ``` - -If it's a cold start invocation, this feature will: - -* Create a separate EMF blob solely containing a metric named `ColdStart` -* Add `FunctionName` and `Service` dimensions - -This has the advantage of keeping cold start metric separate from your application metrics, where you might have unrelated dimensions. - -## Advanced - -### Adding metadata - -You can add high-cardinality data as part of your Metrics log with `AddMetadata` method. This is useful when you want to search highly contextual information along with your metrics in your logs. - -!!! info - **This will not be available during metrics visualization** - Use **dimensions** for this purpose - -!!! info - Adding metadata with a key that is the same as an existing metric will be ignored - -=== "Function.cs" - - ```csharp hl_lines="9" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(Namespace = ExampleApplication, Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - Metrics.AddMetadata("BookingId", "683EEB2D-B2F3-4075-96EE-788E6E2EED45"); - ... - ``` - -=== "Example CloudWatch Logs excerpt" - - ```json hl_lines="23" - { - "SuccessfulBooking": 1.0, - "_aws": { - "Timestamp": 1592234975665, - "CloudWatchMetrics": [ - { - "Namespace": "ExampleApplication", - "Dimensions": [ - [ - "service" - ] - ], - "Metrics": [ - { - "Name": "SuccessfulBooking", - "Unit": "Count" - } - ] - } - ] - }, - "Service": "Booking", - "BookingId": "683EEB2D-B2F3-4075-96EE-788E6E2EED45" - } - ``` - -### Single metric with a different dimension - -CloudWatch EMF uses the same dimensions across all your metrics. Use **`PushSingleMetric`** if you have a metric that should have different dimensions. - -!!! info - Generally, this would be an edge case since you [pay for unique metric](https://aws.amazon.com/cloudwatch/pricing). Keep the following formula in mind: - - **unique metric = (metric_name + dimension_name + dimension_value)** - -=== "Function.cs" - - ```csharp hl_lines="8-13" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(Namespace = ExampleApplication, Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.PushSingleMetric( - name: "ColdStart", - value: 1, - unit: MetricUnit.Count, - nameSpace: "ExampleApplication", - service: "Booking"); - ... - ``` - -By default it will skip all previously defined dimensions including default dimensions. Use `dimensions` argument if you want to reuse default dimensions or specify custom dimensions from a dictionary. - -- `Metrics.DefaultDimensions`: Reuse default dimensions when using static Metrics -- `Options.DefaultDimensions`: Reuse default dimensions when using Builder or Configure patterns - -=== "New Default Dimensions.cs" - - ```csharp hl_lines="8-17" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(Namespace = ExampleApplication, Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.PushSingleMetric( - name: "ColdStart", - value: 1, - unit: MetricUnit.Count, - nameSpace: "ExampleApplication", - service: "Booking", - dimensions: new Dictionary - { - {"FunctionContext", "$LATEST"} - }); - ... - ``` -=== "Default Dimensions static.cs" - - ```csharp hl_lines="8-12" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(Namespace = ExampleApplication, Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.SetDefaultDimensions(new Dictionary - { - { "Default", "SingleMetric" } - }); - Metrics.PushSingleMetric("SingleMetric", 1, MetricUnit.Count, dimensions: Metrics.DefaultDimensions ); - ... - ``` -=== "Default Dimensions Options / Builder patterns" - - ```csharp hl_lines="9-13 18" - using AWS.Lambda.Powertools.Metrics; - - public MetricsnBuilderHandler(IMetrics metrics = null) - { - _metrics = metrics ?? new MetricsBuilder() - .WithCaptureColdStart(true) - .WithService("testService") - .WithNamespace("dotnet-powertools-test") - .WithDefaultDimensions(new Dictionary - { - { "Environment", "Prod1" }, - { "Another", "One" } - }).Build(); - } - - public void HandlerSingleMetricDimensions() - { - _metrics.PushSingleMetric("SuccessfulBooking", 1, MetricUnit.Count, dimensions: _metrics.Options.DefaultDimensions); - } - ... - ``` - -### Cold start Function Name dimension - -In cases where you want to customize the `FunctionName` dimension in Cold Start metrics. - -This is useful where you want to maintain the same name in case of auto generated handler names (cdk, top-level statement functions, etc.) - -Example: - -=== "In decorator" - - ```csharp hl_lines="5" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - [Metrics(FunctionName = "MyFunctionName", Namespace = "ExampleApplication", Service = "Booking")] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - ... - } - ``` -=== "Configure / Builder patterns" - - ```csharp hl_lines="12" - using AWS.Lambda.Powertools.Metrics; - - public class Function { - - public Function() - { - Metrics.Configure(options => - { - options.Namespace = "dotnet-powertools-test"; - options.Service = "testService"; - options.CaptureColdStart = true; - options.FunctionName = "MyFunctionName"; - }); - } - - [Metrics] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - ... - } - ``` - -## AspNetCore - -### Installation - -To use the Metrics middleware in an ASP.NET Core application, you need to install the `AWS.Lambda.Powertools.Metrics.AspNetCore` NuGet package. - -```bash -dotnet add package AWS.Lambda.Powertools.Metrics.AspNetCore -``` - -### UseMetrics() Middleware - -The `UseMetrics` middleware is an extension method for the `IApplicationBuilder` interface. - -It adds a metrics middleware to the specified application builder, which captures cold start metrics (if enabled) and flushes metrics on function exit. - -#### Example - -```csharp hl_lines="21" - -using AWS.Lambda.Powertools.Metrics.AspNetCore.Http; - -var builder = WebApplication.CreateBuilder(args); - -// Configure metrics -builder.Services.AddSingleton(_ => new MetricsBuilder() - .WithNamespace("MyApi") // Namespace for the metrics - .WithService("WeatherService") // Service name for the metrics - .WithCaptureColdStart(true) // Capture cold start metrics - .WithDefaultDimensions(new Dictionary // Default dimensions for the metrics - { - {"Environment", "Prod"}, - {"Another", "One"} - }) - .Build()); // Build the metrics - -builder.Services.AddAWSLambdaHosting(LambdaEventSource.RestApi); - -var app = builder.Build(); - -app.UseMetrics(); // Add the metrics middleware - -app.MapGet("/powertools", (IMetrics metrics) => - { - // add custom metrics - metrics.AddMetric("MyCustomMetric", 1, MetricUnit.Count); - // flush metrics - this is required to ensure metrics are sent to CloudWatch - metrics.Flush(); - }); - -app.Run(); - -``` - -Here is the highlighted `UseMetrics` method: - -```csharp -/// -/// Adds a metrics middleware to the specified application builder. -/// This will capture cold start (if CaptureColdStart is enabled) metrics and flush metrics on function exit. -/// -/// The application builder to add the metrics middleware to. -/// The application builder with the metrics middleware added. -public static IApplicationBuilder UseMetrics(this IApplicationBuilder app) -{ - app.UseMiddleware(); - return app; -} -``` - -Explanation: - -- The method is defined as an extension method for the `IApplicationBuilder` interface. -- It adds a `MetricsMiddleware` to the application builder using the `UseMiddleware` method. -- The `MetricsMiddleware` captures and records metrics for HTTP requests, including cold start metrics if the `CaptureColdStart` option is enabled. - -### WithMetrics() filter - -The `WithMetrics` method is an extension method for the `RouteHandlerBuilder` class. - -It adds a metrics filter to the specified route handler builder, which captures cold start metrics (if enabled) and flushes metrics on function exit. - -#### Example - -```csharp hl_lines="31" - -using AWS.Lambda.Powertools.Metrics; -using AWS.Lambda.Powertools.Metrics.AspNetCore.Http; - -var builder = WebApplication.CreateBuilder(args); - -// Configure metrics -builder.Services.AddSingleton(_ => new MetricsBuilder() - .WithNamespace("MyApi") // Namespace for the metrics - .WithService("WeatherService") // Service name for the metrics - .WithCaptureColdStart(true) // Capture cold start metrics - .WithDefaultDimensions(new Dictionary // Default dimensions for the metrics - { - {"Environment", "Prod"}, - {"Another", "One"} - }) - .Build()); // Build the metrics - -// Add AWS Lambda support. When the application is run in Lambda, Kestrel is swapped out as the web server with Amazon.Lambda.AspNetCoreServer. This -// package will act as the web server translating requests and responses between the Lambda event source and ASP.NET Core. -builder.Services.AddAWSLambdaHosting(LambdaEventSource.RestApi); - -var app = builder.Build(); - -app.MapGet("/powertools", (IMetrics metrics) => - { - // add custom metrics - metrics.AddMetric("MyCustomMetric", 1, MetricUnit.Count); - // flush metrics - this is required to ensure metrics are sent to CloudWatch - metrics.Flush(); - }) - .WithMetrics(); - -app.Run(); - -``` - -Here is the highlighted `WithMetrics` method: - -```csharp -/// -/// Adds a metrics filter to the specified route handler builder. -/// This will capture cold start (if CaptureColdStart is enabled) metrics and flush metrics on function exit. -/// -/// The route handler builder to add the metrics filter to. -/// The route handler builder with the metrics filter added. -public static RouteHandlerBuilder WithMetrics(this RouteHandlerBuilder builder) -{ - builder.AddEndpointFilter(); - return builder; -} -``` - -Explanation: - -- The method is defined as an extension method for the `RouteHandlerBuilder` class. -- It adds a `MetricsFilter` to the route handler builder using the `AddEndpointFilter` method. -- The `MetricsFilter` captures and records metrics for HTTP endpoints, including cold start metrics if the `CaptureColdStart` option is enabled. -- The method returns the modified `RouteHandlerBuilder` instance with the metrics filter added. - - -## Testing your code - -### Unit testing - -To test your code that uses the Metrics utility, you can use the `TestLambdaContext` class from the `Amazon.Lambda.TestUtilities` package. - -You can also use the `IMetrics` interface to mock the Metrics utility in your tests. - -Here is an example of how you can test a Lambda function that uses the Metrics utility: - -#### Lambda Function - -```csharp -using System.Collections.Generic; -using Amazon.Lambda.Core; - -public class MetricsnBuilderHandler -{ - private readonly IMetrics _metrics; - - // Allow injection of IMetrics for testing - public MetricsnBuilderHandler(IMetrics metrics = null) - { - _metrics = metrics ?? new MetricsBuilder() - .WithCaptureColdStart(true) - .WithService("testService") - .WithNamespace("dotnet-powertools-test") - .WithDefaultDimensions(new Dictionary - { - { "Environment", "Prod1" }, - { "Another", "One" } - }).Build(); - } - - [Metrics] - public void Handler(ILambdaContext context) - { - _metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - } -} - -``` -#### Unit Tests - - -```csharp -[Fact] - public void Handler_With_Builder_Should_Configure_In_Constructor() - { - // Arrange - var handler = new MetricsnBuilderHandler(); - - // Act - handler.Handler(new TestLambdaContext - { - FunctionName = "My_Function_Name" - }); - - // Get the output and parse it - var metricsOutput = _consoleOut.ToString(); - - // Assert cold start - Assert.Contains( - "\"CloudWatchMetrics\":[{\"Namespace\":\"dotnet-powertools-test\",\"Metrics\":[{\"Name\":\"ColdStart\",\"Unit\":\"Count\"}],\"Dimensions\":[[\"Service\",\"Environment\",\"Another\",\"FunctionName\"]]}]},\"Service\":\"testService\",\"Environment\":\"Prod1\",\"Another\":\"One\",\"FunctionName\":\"My_Function_Name\",\"ColdStart\":1}", - metricsOutput); - // Assert successful Memory metrics - Assert.Contains( - "\"CloudWatchMetrics\":[{\"Namespace\":\"dotnet-powertools-test\",\"Metrics\":[{\"Name\":\"SuccessfulBooking\",\"Unit\":\"Count\"}],\"Dimensions\":[[\"Service\",\"Environment\",\"Another\",\"FunctionName\"]]}]},\"Service\":\"testService\",\"Environment\":\"Prod1\",\"Another\":\"One\",\"FunctionName\":\"My_Function_Name\",\"SuccessfulBooking\":1}", - metricsOutput); - } - - [Fact] - public void Handler_With_Builder_Should_Configure_In_Constructor_Mock() - { - var metricsMock = Substitute.For(); - - metricsMock.Options.Returns(new MetricsOptions - { - CaptureColdStart = true, - Namespace = "dotnet-powertools-test", - Service = "testService", - DefaultDimensions = new Dictionary - { - { "Environment", "Prod" }, - { "Another", "One" } - } - }); - - Metrics.UseMetricsForTests(metricsMock); - - var sut = new MetricsnBuilderHandler(metricsMock); - - // Act - sut.Handler(new TestLambdaContext - { - FunctionName = "My_Function_Name" - }); - - metricsMock.Received(1).PushSingleMetric("ColdStart", 1, MetricUnit.Count, "dotnet-powertools-test", - service: "testService", Arg.Any>()); - metricsMock.Received(1).AddMetric("SuccessfulBooking", 1, MetricUnit.Count); - } -``` - -### Environment variables - -???+ tip - Ignore this section, if: - - * You are explicitly setting namespace/default dimension via `namespace` and `service` parameters - * You're not instantiating `Metrics` in the global namespace - - For example, `Metrics(namespace="ExampleApplication", service="booking")` - -Make sure to set `POWERTOOLS_METRICS_NAMESPACE` and `POWERTOOLS_SERVICE_NAME` before running your tests to prevent failing on `SchemaValidation` exception. You can set it before you run tests by adding the environment variable. - -```csharp title="Injecting Metric Namespace before running tests" -Environment.SetEnvironmentVariable("POWERTOOLS_METRICS_NAMESPACE","AWSLambdaPowertools"); -``` diff --git a/docs/core/metrics.md b/docs/core/metrics.md index 03f7d6fa8..e941600c4 100644 --- a/docs/core/metrics.md +++ b/docs/core/metrics.md @@ -14,6 +14,17 @@ These metrics can be visualized through [Amazon CloudWatch Console](https://aws. * Metrics are created asynchronously by the CloudWatch service. You do not need any custom stacks, and there is no impact to Lambda function latency * Context manager to create a one off metric with a different dimension * Ahead-of-Time compilation to native code support [AOT](https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html) from version 1.7.0 +* Support for AspNetCore middleware and filters to capture metrics for HTTP requests + +## Breaking changes from V1 + +!!! info + + Loooking for v1 specific documentation please go to [Metrics v1](/lambda/dotnet/core/metrics-v1) + +* **`Dimensions`** outputs as an array of arrays instead of an array of objects. Example: `Dimensions: [["service", "Environment"]]` instead of `Dimensions: ["service", "Environment"]` +* **`FunctionName`** is not added as default dimension and only to cold start metric. +* **`Default Dimensions`** can now be included in Cold Start metrics, this is a potential breaking change if you were relying on the absence of default dimensions in Cold Start metrics when searching.
@@ -28,7 +39,7 @@ Powertools for AWS Lambda (.NET) are available as NuGet packages. You can instal * [AWS.Lambda.Powertools.Metrics](https://www.nuget.org/packages?q=AWS.Lambda.Powertools.Metrics): - `dotnet add package AWS.Lambda.Powertools.Metrics -v 1.7.1` + `dotnet add package AWS.Lambda.Powertools.Metrics` ## Terminologies @@ -51,12 +62,16 @@ Visit the AWS documentation for a complete explanation for [Amazon CloudWatch co **`Metrics`** is implemented as a Singleton to keep track of your aggregate metrics in memory and make them accessible anywhere in your code. To guarantee that metrics are flushed properly the **`MetricsAttribute`** must be added on the lambda handler. -Metrics has two global settings that will be used across all metrics emitted. Use your application or main service as the metric namespace to easily group all metrics: +Metrics has three global settings that will be used across all metrics emitted. Use your application or main service as the metric namespace to easily group all metrics: -Setting | Description | Environment variable | Constructor parameter -------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- -**Service** | Optionally, sets **service** metric dimension across all metrics e.g. `payment` | `POWERTOOLS_SERVICE_NAME` | `Service` -**Metric namespace** | Logical container where all metrics will be placed e.g. `MyCompanyEcommerce` | `POWERTOOLS_METRICS_NAMESPACE` | `Namespace` + Setting | Description | Environment variable | Decorator parameter +-------------------------------|---------------------------------------------------------------------------------| ------------------------------------------------- |----------------------- + **Metric namespace** | Logical container where all metrics will be placed e.g. `MyCompanyEcommerce` | `POWERTOOLS_METRICS_NAMESPACE` | `Namespace` + **Service** | Optionally, sets **Service** metric dimension across all metrics e.g. `payment` | `POWERTOOLS_SERVICE_NAME` | `Service` +**Disable Powertools Metrics** | Optionally, disables all Powertools metrics |`POWERTOOLS_METRICS_DISABLED` | N/A | + +???+ info + `POWERTOOLS_METRICS_DISABLED` will not disable default metrics created by AWS services. !!! info "Autocomplete Metric Units" All parameters in **`Metrics Attribute`** are optional. Following rules apply: @@ -66,42 +81,100 @@ Setting | Description | Environment variable | Constructor parameter - **CaptureColdStart:** **`false`** by default. - **RaiseOnEmptyMetrics:** **`false`** by default. -### Example using AWS Serverless Application Model (AWS SAM) +### Metrics object -=== "template.yml" +#### Attribute - ```yaml hl_lines="9 10" - Resources: - HelloWorldFunction: - Type: AWS::Serverless::Function - Properties: - ... - Environment: - Variables: - POWERTOOLS_SERVICE_NAME: ShoppingCartService - POWERTOOLS_METRICS_NAMESPACE: MyCompanyEcommerce - ``` +The **`MetricsAttribute`** is a class-level attribute that can be used to set the namespace and service for all metrics emitted by the lambda handler. -=== "Function.cs" +```csharp hl_lines="3" +using AWS.Lambda.Powertools.Metrics; + +[Metrics(Namespace = "ExampleApplication", Service = "Booking")] +public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) +{ + ... +} +``` - ```csharp hl_lines="4" - using AWS.Lambda.Powertools.Metrics; +#### Methods - public class Function { - [Metrics(Namespace = "MyCompanyEcommerce", Service = "ShoppingCartService", CaptureColdStart = true, RaiseOnEmptyMetrics = true)] - public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) - { - ... - } - } - ``` +The **`Metrics`** class provides methods to add metrics, dimensions, and metadata to the metrics object. + +```csharp hl_lines="5-7" +using AWS.Lambda.Powertools.Metrics; + +public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) +{ + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + Metrics.AddDimension("Environment", "Prod"); + Metrics.AddMetadata("BookingId", "683EEB2D-B2F3-4075-96EE-788E6E2EED45"); + ... +} +``` + +#### Initialization -### Full list of environment variables +The **`Metrics`** object is initialized as a Singleton and can be accessed anywhere in your code. + +But can also be initialize with `Configure` or `Builder` patterns in your Lambda constructor, this the best option for testing. + +Configure: + +```csharp +using AWS.Lambda.Powertools.Metrics; + +public Function() +{ + Metrics.Configure(options => + { + options.Namespace = "dotnet-powertools-test"; + options.Service = "testService"; + options.CaptureColdStart = true; + options.DefaultDimensions = new Dictionary + { + { "Environment", "Prod" }, + { "Another", "One" } + }; + }); +} + +[Metrics] +public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) +{ + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + ... +} +``` + +Builder: + +```csharp +using AWS.Lambda.Powertools.Metrics; + +private readonly IMetrics _metrics; + +public Function() +{ + _metrics = new MetricsBuilder() + .WithCaptureColdStart(true) + .WithService("testService") + .WithNamespace("dotnet-powertools-test") + .WithDefaultDimensions(new Dictionary + { + { "Environment", "Prod1" }, + { "Another", "One" } + }).Build(); +} + +[Metrics] +public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) +{ + _metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + ... +} +``` -| Environment variable | Description | Default | -| ------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- | -| **POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | `"service_undefined"` | -| **POWERTOOLS_METRICS_NAMESPACE** | Sets namespace used for metrics | `None` | ### Creating metrics @@ -205,15 +278,127 @@ You can use **`SetDefaultDimensions`** method to persist dimensions across Lambd } ``` +### Adding default dimensions with cold start metric + +You can use the Builder or Configure patterns in your Lambda class constructor to set default dimensions. + +=== "Builder pattern" + + ```csharp hl_lines="12-16" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + private readonly IMetrics _metrics; + + public Function() + { + _metrics = new MetricsBuilder() + .WithCaptureColdStart(true) + .WithService("testService") + .WithNamespace("dotnet-powertools-test") + .WithDefaultDimensions(new Dictionary + { + { "Environment", "Prod1" }, + { "Another", "One" } + }).Build(); + } + + [Metrics] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + _metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + ... + } + ``` +=== "Configure pattern" + + ```csharp hl_lines="12-16" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + public Function() + { + Metrics.Configure(options => + { + options.Namespace = "dotnet-powertools-test"; + options.Service = "testService"; + options.CaptureColdStart = true; + options.DefaultDimensions = new Dictionary + { + { "Environment", "Prod" }, + { "Another", "One" } + }; + }); + } + + [Metrics] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + ... + } + ``` +### Adding dimensions + +You can add dimensions to your metrics using **`AddDimension`** method. + +=== "Function.cs" + + ```csharp hl_lines="8" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(Namespace = "ExampleApplication", Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.AddDimension("Environment","Prod"); + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + } + } + ``` +=== "Example CloudWatch Logs excerpt" + + ```json hl_lines="11 24" + { + "SuccessfulBooking": 1.0, + "_aws": { + "Timestamp": 1592234975665, + "CloudWatchMetrics": [ + { + "Namespace": "ExampleApplication", + "Dimensions": [ + [ + "service", + "Environment" + ] + ], + "Metrics": [ + { + "Name": "SuccessfulBooking", + "Unit": "Count" + } + ] + } + ] + }, + "service": "ExampleService", + "Environment": "Prod" + } + ``` + ### Flushing metrics With **`MetricsAttribute`** all your metrics are validated, serialized and flushed to standard output when lambda handler completes execution or when you had the 100th metric to memory. +You can also flush metrics manually by calling **`Flush`** method. + During metrics validation, if no metrics are provided then a warning will be logged, but no exception will be raised. === "Function.cs" - ```csharp hl_lines="8" + ```csharp hl_lines="9" using AWS.Lambda.Powertools.Metrics; public class Function { @@ -222,6 +407,7 @@ During metrics validation, if no metrics are provided then a warning will be log public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) { Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + Metrics.Flush(); } } ``` @@ -256,13 +442,13 @@ During metrics validation, if no metrics are provided then a warning will be log !!! tip "Metric validation" If metrics are provided, and any of the following criteria are not met, **`SchemaValidationException`** will be raised: - * Maximum of 9 dimensions + * Maximum of 30 dimensions * Namespace is set * Metric units must be [supported by CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html) !!! info "We do not emit 0 as a value for ColdStart metric for cost reasons. [Let us know](https://github.com/aws-powertools/powertools-lambda-dotnet/issues/new?assignees=&labels=feature-request%2Ctriage&template=feature_request.yml&title=Feature+request%3A+TITLE) if you'd prefer a flag to override it" -#### Raising SchemaValidationException on empty metrics +### Raising SchemaValidationException on empty metrics If you want to ensure that at least one metric is emitted, you can pass **`RaiseOnEmptyMetrics`** to the Metrics attribute: @@ -295,11 +481,58 @@ You can optionally capture cold start metrics by setting **`CaptureColdStart`** { ... ``` +=== "Builder pattern" + + ```csharp hl_lines="9" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + private readonly IMetrics _metrics; + + public Function() + { + _metrics = new MetricsBuilder() + .WithCaptureColdStart(true) + .WithService("testService") + .WithNamespace("dotnet-powertools-test") + } + + [Metrics] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + _metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + ... + } + ``` +=== "Configure pattern" + + ```csharp hl_lines="11" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + public Function() + { + Metrics.Configure(options => + { + options.Namespace = "dotnet-powertools-test"; + options.Service = "testService"; + options.CaptureColdStart = true; + }); + } + + [Metrics] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + ... + } + ``` If it's a cold start invocation, this feature will: * Create a separate EMF blob solely containing a metric named `ColdStart` -* Add `function_name` and `service` dimensions +* Add `FunctionName` and `Service` dimensions This has the advantage of keeping cold start metric separate from your application metrics, where you might have unrelated dimensions. @@ -370,6 +603,30 @@ CloudWatch EMF uses the same dimensions across all your metrics. Use **`PushSing === "Function.cs" + ```csharp hl_lines="8-13" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(Namespace = ExampleApplication, Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.PushSingleMetric( + name: "ColdStart", + value: 1, + unit: MetricUnit.Count, + nameSpace: "ExampleApplication", + service: "Booking"); + ... + ``` + +By default it will skip all previously defined dimensions including default dimensions. Use `dimensions` argument if you want to reuse default dimensions or specify custom dimensions from a dictionary. + +- `Metrics.DefaultDimensions`: Reuse default dimensions when using static Metrics +- `Options.DefaultDimensions`: Reuse default dimensions when using Builder or Configure patterns + +=== "New Default Dimensions.cs" + ```csharp hl_lines="8-17" using AWS.Lambda.Powertools.Metrics; @@ -379,20 +636,357 @@ CloudWatch EMF uses the same dimensions across all your metrics. Use **`PushSing public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) { Metrics.PushSingleMetric( - metricName: "ColdStart", + name: "ColdStart", value: 1, unit: MetricUnit.Count, nameSpace: "ExampleApplication", service: "Booking", - defaultDimensions: new Dictionary + dimensions: new Dictionary { {"FunctionContext", "$LATEST"} }); ... ``` +=== "Default Dimensions static.cs" + + ```csharp hl_lines="8-12" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(Namespace = ExampleApplication, Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.SetDefaultDimensions(new Dictionary + { + { "Default", "SingleMetric" } + }); + Metrics.PushSingleMetric("SingleMetric", 1, MetricUnit.Count, dimensions: Metrics.DefaultDimensions ); + ... + ``` +=== "Default Dimensions Options / Builder patterns" + + ```csharp hl_lines="9-13 18" + using AWS.Lambda.Powertools.Metrics; + + public MetricsnBuilderHandler(IMetrics metrics = null) + { + _metrics = metrics ?? new MetricsBuilder() + .WithCaptureColdStart(true) + .WithService("testService") + .WithNamespace("dotnet-powertools-test") + .WithDefaultDimensions(new Dictionary + { + { "Environment", "Prod1" }, + { "Another", "One" } + }).Build(); + } + + public void HandlerSingleMetricDimensions() + { + _metrics.PushSingleMetric("SuccessfulBooking", 1, MetricUnit.Count, dimensions: _metrics.Options.DefaultDimensions); + } + ... + ``` + +### Cold start Function Name dimension + +In cases where you want to customize the `FunctionName` dimension in Cold Start metrics. + +This is useful where you want to maintain the same name in case of auto generated handler names (cdk, top-level statement functions, etc.) + +Example: + +=== "In decorator" + + ```csharp hl_lines="5" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + [Metrics(FunctionName = "MyFunctionName", Namespace = "ExampleApplication", Service = "Booking")] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + ... + } + ``` +=== "Configure / Builder patterns" + + ```csharp hl_lines="12" + using AWS.Lambda.Powertools.Metrics; + + public class Function { + + public Function() + { + Metrics.Configure(options => + { + options.Namespace = "dotnet-powertools-test"; + options.Service = "testService"; + options.CaptureColdStart = true; + options.FunctionName = "MyFunctionName"; + }); + } + + [Metrics] + public async Task FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context) + { + Metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + ... + } + ``` + +## AspNetCore + +### Installation + +To use the Metrics middleware in an ASP.NET Core application, you need to install the `AWS.Lambda.Powertools.Metrics.AspNetCore` NuGet package. + +```bash +dotnet add package AWS.Lambda.Powertools.Metrics.AspNetCore +``` + +### UseMetrics() Middleware + +The `UseMetrics` middleware is an extension method for the `IApplicationBuilder` interface. + +It adds a metrics middleware to the specified application builder, which captures cold start metrics (if enabled) and flushes metrics on function exit. + +#### Example + +```csharp hl_lines="21" + +using AWS.Lambda.Powertools.Metrics.AspNetCore.Http; + +var builder = WebApplication.CreateBuilder(args); + +// Configure metrics +builder.Services.AddSingleton(_ => new MetricsBuilder() + .WithNamespace("MyApi") // Namespace for the metrics + .WithService("WeatherService") // Service name for the metrics + .WithCaptureColdStart(true) // Capture cold start metrics + .WithDefaultDimensions(new Dictionary // Default dimensions for the metrics + { + {"Environment", "Prod"}, + {"Another", "One"} + }) + .Build()); // Build the metrics + +builder.Services.AddAWSLambdaHosting(LambdaEventSource.RestApi); + +var app = builder.Build(); + +app.UseMetrics(); // Add the metrics middleware + +app.MapGet("/powertools", (IMetrics metrics) => + { + // add custom metrics + metrics.AddMetric("MyCustomMetric", 1, MetricUnit.Count); + // flush metrics - this is required to ensure metrics are sent to CloudWatch + metrics.Flush(); + }); + +app.Run(); + +``` + +Here is the highlighted `UseMetrics` method: + +```csharp +/// +/// Adds a metrics middleware to the specified application builder. +/// This will capture cold start (if CaptureColdStart is enabled) metrics and flush metrics on function exit. +/// +/// The application builder to add the metrics middleware to. +/// The application builder with the metrics middleware added. +public static IApplicationBuilder UseMetrics(this IApplicationBuilder app) +{ + app.UseMiddleware(); + return app; +} +``` + +Explanation: + +- The method is defined as an extension method for the `IApplicationBuilder` interface. +- It adds a `MetricsMiddleware` to the application builder using the `UseMiddleware` method. +- The `MetricsMiddleware` captures and records metrics for HTTP requests, including cold start metrics if the `CaptureColdStart` option is enabled. + +### WithMetrics() filter + +The `WithMetrics` method is an extension method for the `RouteHandlerBuilder` class. + +It adds a metrics filter to the specified route handler builder, which captures cold start metrics (if enabled) and flushes metrics on function exit. + +#### Example + +```csharp hl_lines="31" + +using AWS.Lambda.Powertools.Metrics; +using AWS.Lambda.Powertools.Metrics.AspNetCore.Http; + +var builder = WebApplication.CreateBuilder(args); + +// Configure metrics +builder.Services.AddSingleton(_ => new MetricsBuilder() + .WithNamespace("MyApi") // Namespace for the metrics + .WithService("WeatherService") // Service name for the metrics + .WithCaptureColdStart(true) // Capture cold start metrics + .WithDefaultDimensions(new Dictionary // Default dimensions for the metrics + { + {"Environment", "Prod"}, + {"Another", "One"} + }) + .Build()); // Build the metrics + +// Add AWS Lambda support. When the application is run in Lambda, Kestrel is swapped out as the web server with Amazon.Lambda.AspNetCoreServer. This +// package will act as the web server translating requests and responses between the Lambda event source and ASP.NET Core. +builder.Services.AddAWSLambdaHosting(LambdaEventSource.RestApi); + +var app = builder.Build(); + +app.MapGet("/powertools", (IMetrics metrics) => + { + // add custom metrics + metrics.AddMetric("MyCustomMetric", 1, MetricUnit.Count); + // flush metrics - this is required to ensure metrics are sent to CloudWatch + metrics.Flush(); + }) + .WithMetrics(); + +app.Run(); + +``` + +Here is the highlighted `WithMetrics` method: + +```csharp +/// +/// Adds a metrics filter to the specified route handler builder. +/// This will capture cold start (if CaptureColdStart is enabled) metrics and flush metrics on function exit. +/// +/// The route handler builder to add the metrics filter to. +/// The route handler builder with the metrics filter added. +public static RouteHandlerBuilder WithMetrics(this RouteHandlerBuilder builder) +{ + builder.AddEndpointFilter(); + return builder; +} +``` + +Explanation: + +- The method is defined as an extension method for the `RouteHandlerBuilder` class. +- It adds a `MetricsFilter` to the route handler builder using the `AddEndpointFilter` method. +- The `MetricsFilter` captures and records metrics for HTTP endpoints, including cold start metrics if the `CaptureColdStart` option is enabled. +- The method returns the modified `RouteHandlerBuilder` instance with the metrics filter added. + ## Testing your code +### Unit testing + +To test your code that uses the Metrics utility, you can use the `TestLambdaContext` class from the `Amazon.Lambda.TestUtilities` package. + +You can also use the `IMetrics` interface to mock the Metrics utility in your tests. + +Here is an example of how you can test a Lambda function that uses the Metrics utility: + +#### Lambda Function + +```csharp +using System.Collections.Generic; +using Amazon.Lambda.Core; + +public class MetricsnBuilderHandler +{ + private readonly IMetrics _metrics; + + // Allow injection of IMetrics for testing + public MetricsnBuilderHandler(IMetrics metrics = null) + { + _metrics = metrics ?? new MetricsBuilder() + .WithCaptureColdStart(true) + .WithService("testService") + .WithNamespace("dotnet-powertools-test") + .WithDefaultDimensions(new Dictionary + { + { "Environment", "Prod1" }, + { "Another", "One" } + }).Build(); + } + + [Metrics] + public void Handler(ILambdaContext context) + { + _metrics.AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + } +} + +``` +#### Unit Tests + + +```csharp +[Fact] + public void Handler_With_Builder_Should_Configure_In_Constructor() + { + // Arrange + var handler = new MetricsnBuilderHandler(); + + // Act + handler.Handler(new TestLambdaContext + { + FunctionName = "My_Function_Name" + }); + + // Get the output and parse it + var metricsOutput = _consoleOut.ToString(); + + // Assert cold start + Assert.Contains( + "\"CloudWatchMetrics\":[{\"Namespace\":\"dotnet-powertools-test\",\"Metrics\":[{\"Name\":\"ColdStart\",\"Unit\":\"Count\"}],\"Dimensions\":[[\"Service\",\"Environment\",\"Another\",\"FunctionName\"]]}]},\"Service\":\"testService\",\"Environment\":\"Prod1\",\"Another\":\"One\",\"FunctionName\":\"My_Function_Name\",\"ColdStart\":1}", + metricsOutput); + // Assert successful Memory metrics + Assert.Contains( + "\"CloudWatchMetrics\":[{\"Namespace\":\"dotnet-powertools-test\",\"Metrics\":[{\"Name\":\"SuccessfulBooking\",\"Unit\":\"Count\"}],\"Dimensions\":[[\"Service\",\"Environment\",\"Another\",\"FunctionName\"]]}]},\"Service\":\"testService\",\"Environment\":\"Prod1\",\"Another\":\"One\",\"FunctionName\":\"My_Function_Name\",\"SuccessfulBooking\":1}", + metricsOutput); + } + + [Fact] + public void Handler_With_Builder_Should_Configure_In_Constructor_Mock() + { + var metricsMock = Substitute.For(); + + metricsMock.Options.Returns(new MetricsOptions + { + CaptureColdStart = true, + Namespace = "dotnet-powertools-test", + Service = "testService", + DefaultDimensions = new Dictionary + { + { "Environment", "Prod" }, + { "Another", "One" } + } + }); + + Metrics.UseMetricsForTests(metricsMock); + + var sut = new MetricsnBuilderHandler(metricsMock); + + // Act + sut.Handler(new TestLambdaContext + { + FunctionName = "My_Function_Name" + }); + + metricsMock.Received(1).PushSingleMetric("ColdStart", 1, MetricUnit.Count, "dotnet-powertools-test", + service: "testService", Arg.Any>()); + metricsMock.Received(1).AddMetric("SuccessfulBooking", 1, MetricUnit.Count); + } +``` + ### Environment variables ???+ tip diff --git a/docs/getting-started/idempotency/aot.md b/docs/getting-started/idempotency/aot.md new file mode 100644 index 000000000..e69de29bb diff --git a/docs/getting-started/idempotency/simple.md b/docs/getting-started/idempotency/simple.md new file mode 100644 index 000000000..51536a470 --- /dev/null +++ b/docs/getting-started/idempotency/simple.md @@ -0,0 +1,4 @@ +--- +title: Simple Logging +description: Getting started with Logging +--- \ No newline at end of file diff --git a/docs/getting-started/logger/aot.md b/docs/getting-started/logger/aot.md new file mode 100644 index 000000000..f42610893 --- /dev/null +++ b/docs/getting-started/logger/aot.md @@ -0,0 +1,426 @@ +--- +title: Native AOT with Logger +description: Getting started with Logging in Native AOT applications +--- + +# Getting Started with AWS Lambda Powertools for .NET Logger in Native AOT + +This tutorial shows you how to set up an AWS Lambda project using Native AOT compilation with Powertools for .NET +Logger, addressing performance, trimming, and deployment considerations. + +## Prerequisites + +- An AWS account with appropriate permissions +- A code editor (we'll use Visual Studio Code in this tutorial) +- .NET 8 SDK or later +- Docker (required for cross-platform AOT compilation) + +## 1. Understanding Native AOT + +Native AOT (Ahead-of-Time) compilation converts your .NET application directly to native code during build time rather +than compiling to IL (Intermediate Language) code that gets JIT-compiled at runtime. Benefits for AWS Lambda include: + +- Faster cold start times (typically 50-70% reduction) +- Lower memory footprint +- No runtime JIT compilation overhead +- No need for the full .NET runtime to be packaged with your Lambda + +## 2. Installing Required Tools + +First, ensure you have the .NET 8 SDK installed: + +```bash +dotnet --version +``` + +Install the AWS Lambda .NET CLI tools: + +```bash +dotnet tool install -g Amazon.Lambda.Tools +dotnet new install Amazon.Lambda.Templates +``` + +Verify installation: + +```bash +dotnet lambda --help +``` + +## 3. Creating a Native AOT Lambda Project + +Create a directory for your project: + +```bash +mkdir powertools-aot-logger-demo +cd powertools-aot-logger-demo +``` + +Create a new Lambda project using the Native AOT template: + +```bash +dotnet new lambda.NativeAOT -n PowertoolsAotLoggerDemo +cd PowertoolsAotLoggerDemo +``` + +## 4. Adding the Powertools Logger Package + +Add the AWS.Lambda.Powertools.Logging package: + +```bash +cd src/PowertoolsAotLoggerDemo +dotnet add package AWS.Lambda.Powertools.Logging +``` + +## 5. Implementing the Lambda Function with AOT-compatible Logger + +Let's modify the Function.cs file to implement our function with Powertools Logger in an AOT-compatible way: + +```csharp +using Amazon.Lambda.Core; +using Amazon.Lambda.RuntimeSupport; +using Amazon.Lambda.Serialization.SystemTextJson; +using System.Text.Json.Serialization; +using System.Text.Json; +using AWS.Lambda.Powertools.Logging; +using Microsoft.Extensions.Logging; + + +namespace PowertoolsAotLoggerDemo; + +public class Function +{ + private static ILogger _logger; + + private static async Task Main() + { + _logger = LoggerFactory.Create(builder => + { + builder.AddPowertoolsLogger(config => + { + config.Service = "TestService"; + config.LoggerOutputCase = LoggerOutputCase.PascalCase; + config.JsonOptions = new JsonSerializerOptions + { + TypeInfoResolver = LambdaFunctionJsonSerializerContext.Default + }; + }); + }).CreatePowertoolsLogger(); + + Func handler = FunctionHandler; + await LambdaBootstrapBuilder.Create(handler, new SourceGeneratorLambdaJsonSerializer()) + .Build() + .RunAsync(); + } + + public static string FunctionHandler(string input, ILambdaContext context) + { + _logger.LogInformation("Processing input: {Input}", input); + _logger.LogInformation("Processing context: {@Context}", context); + + return input.ToUpper(); + } +} + + +[JsonSerializable(typeof(string))] +[JsonSerializable(typeof(ILambdaContext))] // make sure to include ILambdaContext for serialization +public partial class LambdaFunctionJsonSerializerContext : JsonSerializerContext +{ +} +``` + +## 6. Updating the Project File for AOT Compatibility + +```xml + + + + net8.0 + enable + enable + + + true + + + true + + + full + + + 0 + + + + + + true + Size + true + + + true + Lambda + + + Exe + + + false + Size + + + + + + + + + +``` + +## 8. Cross-Platform Deployment Considerations + +Native AOT compilation must target the same OS and architecture as the deployment environment. AWS Lambda runs on Amazon +Linux 2023 (AL2023) with x64 architecture. + +### Building for AL2023 on Different Platforms + +#### Option A: Using the AWS Lambda .NET Tool with Docker + +The simplest approach is to use the AWS Lambda .NET tool, which handles the cross-platform compilation: + +```bash +dotnet lambda deploy-function --function-name powertools-aot-logger-demo --function-role your-lambda-role-arn +``` + +This will: + +1. Detect your project is using Native AOT +2. Use Docker behind the scenes to compile for Amazon Linux +3. Deploy the resulting function + +#### Option B: Using Docker Directly + +Alternatively, you can use Docker directly for more control: + +##### On macOS/Linux: + +```bash +# Create a build container using Amazon's provided image +docker run --rm -v $(pwd):/workspace -w /workspace public.ecr.aws/sam/build-dotnet8:latest-x86_64 \ + bash -c "cd src/PowertoolsAotLoggerDemo && dotnet publish -c Release -r linux-x64 -o publish" + +# Deploy using the AWS CLI +cd src/PowertoolsAotLoggerDemo/publish +zip -r function.zip * +aws lambda create-function \ + --function-name powertools-aot-logger-demo \ + --runtime provided.al2023 \ + --handler bootstrap \ + --role arn:aws:iam::123456789012:role/your-lambda-role \ + --zip-file fileb://function.zip +``` + +##### On Windows: + +```powershell +# Create a build container using Amazon's provided image +docker run --rm -v ${PWD}:/workspace -w /workspace public.ecr.aws/sam/build-dotnet8:latest-x86_64 ` + bash -c "cd src/PowertoolsAotLoggerDemo && dotnet publish -c Release -r linux-x64 -o publish" + +# Deploy using the AWS CLI +cd src\PowertoolsAotLoggerDemo\publish +Compress-Archive -Path * -DestinationPath function.zip -Force +aws lambda create-function ` + --function-name powertools-aot-logger-demo ` + --runtime provided.al2023 ` + --handler bootstrap ` + --role arn:aws:iam::123456789012:role/your-lambda-role ` + --zip-file fileb://function.zip +``` + +## 9. Testing the Function + +Test your Lambda function using the AWS CLI: + +```bash +aws lambda invoke --function-name powertools-aot-logger-demo --payload '{"name":"PowertoolsAOT"}' response.json +cat response.json +``` + +You should see a response like: + +```json +{ + "Level": "Information", + "Message": "test", + "Timestamp": "2025-05-06T09:52:19.8222787Z", + "Service": "TestService", + "ColdStart": true, + "XrayTraceId": "1-6819dbd3-0de6dc4b6cc712b020ee8ae7", + "Name": "AWS.Lambda.Powertools.Logging.Logger" +} +{ + "Level": "Information", + "Message": "Processing context: Amazon.Lambda.RuntimeSupport.LambdaContext", + "Timestamp": "2025-05-06T09:52:19.8232664Z", + "Service": "TestService", + "ColdStart": true, + "XrayTraceId": "1-6819dbd3-0de6dc4b6cc712b020ee8ae7", + "Name": "AWS.Lambda.Powertools.Logging.Logger", + "Context": { + "AwsRequestId": "20f8da57-002b-426d-84c2-c295e4797e23", + "ClientContext": { + "Environment": null, + "Client": null, + "Custom": null + }, + "FunctionName": "powertools-aot-logger-demo", + "FunctionVersion": "$LATEST", + "Identity": { + "IdentityId": null, + "IdentityPoolId": null + }, + "InvokedFunctionArn": "your arn", + "Logger": {}, + "LogGroupName": "/aws/lambda/powertools-aot-logger-demo", + "LogStreamName": "2025/05/06/[$LATEST]71249d02013b42b9b044b42dd4c7c37a", + "MemoryLimitInMB": 512, + "RemainingTime": "00:00:29.9972216" + } +} +``` + +Check the logs in CloudWatch Logs to see the structured logs created by Powertools Logger. + +## 10. Performance Considerations and Best Practices + +### Trimming Considerations + +Native AOT uses aggressive trimming, which can cause issues with reflection-based code. Here are tips to avoid common +problems: + +1. **Using DynamicJsonSerializer**: If you're encountering trimming issues with JSON serialization, add a trimming hint: + +```csharp +[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.PublicFields | DynamicallyAccessedMemberTypes.PublicProperties)] +public class MyRequestType +{ + // Properties that will be preserved during trimming +} +``` + +2. **Logging Objects**: When logging objects with structural logging, consider creating simple DTOs instead of complex + types: + +```csharp +// Instead of logging complex domain objects: +Logger.LogInformation("User: {@user}", complexUserWithCircularReferences); + +// Create a simple loggable DTO: +var userInfo = new { Id = user.Id, Name = user.Name, Status = user.Status }; +Logger.LogInformation("User: {@userInfo}", userInfo); +``` + +3. **Handling Reflection**: If you need reflection, explicitly preserve types: + +```xml + + + + +``` + +And in TrimmerRoots.xml: + +```xml + + + + + + +``` + +### Lambda Configuration Best Practices + +1. **Memory Settings**: Native AOT functions typically need less memory: + +```bash +aws lambda update-function-configuration \ + --function-name powertools-aot-logger-demo \ + --memory-size 512 +``` + +2. **Environment Variables**: Set the AWS_LAMBDA_DOTNET_PREJIT environment variable to 0 (it's not needed for AOT): + +```bash +aws lambda update-function-configuration \ + --function-name powertools-aot-logger-demo \ + --environment Variables={AWS_LAMBDA_DOTNET_PREJIT=0} +``` + +3. **ARM64 Support**: For even better performance, consider using ARM64 architecture: + +When creating your project: + +```bash +dotnet new lambda.NativeAOT -n PowertoolsAotLoggerDemo --architecture arm64 +``` + +Or modify your deployment: + +```bash +aws lambda update-function-configuration \ + --function-name powertools-aot-logger-demo \ + --architectures arm64 +``` + +### Monitoring Cold Start Performance + +The Powertools Logger automatically logs cold start information. Use CloudWatch Logs Insights to analyze performance: + +``` +fields @timestamp, coldStart, billedDurationMs, maxMemoryUsedMB +| filter functionName = "powertools-aot-logger-demo" +| sort @timestamp desc +| limit 100 +``` + +## 11. Troubleshooting Common AOT Issues + +### Missing Type Metadata + +If you see errors about missing metadata, you may need to add more types to your trimmer roots: + +```xml + + + + + + +``` + +### Build Failures on macOS/Windows + +If you're building directly on macOS/Windows without Docker and encountering errors, remember that Native AOT is +platform-specific. Always use the cross-platform build options mentioned earlier. + +## Summary + +In this tutorial, you've learned: + +1. How to set up a .NET Native AOT Lambda project with Powertools Logger +2. How to handle trimming concerns and ensure compatibility +3. Cross-platform build and deployment strategies for Amazon Linux 2023 +4. Performance optimization techniques specific to AOT lambdas + +Native AOT combined with Powertools Logger gives you the best of both worlds: high-performance, low-latency Lambda +functions with rich, structured logging capabilities. + +!!! tip "Next Steps" +Explore using the Embedded Metrics Format (EMF) with your Native AOT Lambda functions for enhanced observability, or try +implementing Powertools Tracing in your Native AOT functions. diff --git a/docs/getting-started/logger/aspnet.md b/docs/getting-started/logger/aspnet.md new file mode 100644 index 000000000..991bfc399 --- /dev/null +++ b/docs/getting-started/logger/aspnet.md @@ -0,0 +1,500 @@ +--- +title: ASP.NET Core Minimal API Logging +description: Getting started with Logging in ASP.NET Core Minimal APIs +--- + +# Getting Started with AWS Lambda Powertools for .NET Logger in ASP.NET Core Minimal APIs + +This tutorial shows you how to set up an ASP.NET Core Minimal API project with AWS Lambda Powertools for .NET Logger - covering installation of required tools through deployment and advanced logging features. + +## Prerequisites + +- An AWS account with appropriate permissions +- A code editor (we'll use Visual Studio Code in this tutorial) +- .NET 8 SDK or later + +## 1. Installing Required Tools + +First, ensure you have the .NET SDK installed. If not, you can download it from the [.NET download page](https://dotnet.microsoft.com/download/dotnet). + +```bash +dotnet --version +``` + +You should see output like `8.0.100` or similar. + +Next, install the AWS Lambda .NET CLI tools: + +```bash +dotnet tool install -g Amazon.Lambda.Tools +dotnet new install Amazon.Lambda.Templates +``` + +Verify installation: + +```bash +dotnet lambda --help +``` + +## 2. Setting up AWS CLI credentials + +Ensure your AWS credentials are configured: + +```bash +aws configure +``` + +Enter your AWS Access Key ID, Secret Access Key, default region, and output format. + +## 3. Creating a New ASP.NET Core Minimal API Lambda Project + +Create a directory for your project: + +```bash +mkdir powertools-aspnet-logger-demo +cd powertools-aspnet-logger-demo +``` + +Create a new ASP.NET Minimal API project using the AWS Lambda template: + +```bash +dotnet new serverless.AspNetCoreMinimalAPI --name PowertoolsAspNetLoggerDemo +cd PowertoolsAspNetLoggerDemo/src/PowertoolsAspNetLoggerDemo +``` + +## 4. Adding the Powertools Logger Package + +Add the AWS.Lambda.Powertools.Logging package: + +```bash +dotnet add package AWS.Lambda.Powertools.Logging +``` + +## 5. Implementing the Minimal API with Powertools Logger + +Let's modify the Program.cs file to implement our Minimal API with Powertools Logger: + +```csharp +using Microsoft.Extensions.Logging; +using AWS.Lambda.Powertools.Logging; + +var builder = WebApplication.CreateBuilder(args); + +// Configure AWS Lambda +// This is what connects the Events from API Gateway to the ASP.NET Core pipeline +// In this case we are using HttpApi +builder.Services.AddAWSLambdaHosting(LambdaEventSource.HttpApi); + +// Add Powertools Logger +var logger = LoggerFactory.Create(builder => +{ + builder.AddPowertoolsLogger(config => + { + config.Service = "powertools-aspnet-demo"; + config.MinimumLogLevel = LogLevel.Debug; + config.LoggerOutputCase = LoggerOutputCase.CamelCase; + config.TimestampFormat = "yyyy-MM-dd HH:mm:ss.fff"; + }); +}).CreatePowertoolsLogger(); + +var app = builder.Build(); + +app.MapGet("/", () => { + logger.LogInformation("Processing root request"); + return "Hello from Powertools ASP.NET Core Minimal API!"; +}); + +app.MapGet("/users/{id}", (string id) => { + logger.LogInformation("Getting user with ID: {userId}", id); + + // Log a structured object + var user = new User { + Id = id, + Name = "John Doe", + Email = "john.doe@example.com" + }; + + logger.LogDebug("User details: {@user}", user); + + return Results.Ok(user); +}); + +app.Run(); + +// Simple user class for demonstration +public class User +{ + public string? Id { get; set; } + public string? Name { get; set; } + public string? Email { get; set; } + + public override string ToString() + { + return $"{Name} ({Id})"; + } +} +``` + +## 6. Understanding the LoggerFactory Setup + +Let's examine the key parts of how we've set up the logger: + +```csharp +var logger = LoggerFactory.Create(builder => +{ + builder.AddPowertoolsLogger(config => + { + config.Service = "powertools-aspnet-demo"; + config.MinimumLogLevel = LogLevel.Debug; + config.LoggerOutputCase = LoggerOutputCase.CamelCase; + config.TimestampFormat = "yyyy-MM-dd HH:mm:ss.fff"; + }); +}).CreatePowertoolsLogger(); +``` + +This setup: + +1. Creates a new `LoggerFactory` instance +2. Adds the Powertools Logger provider to the factory +3. Configures the logger with: + - Service name that appears in all logs + - Minimum logging level set to Information + - CamelCase output format for JSON properties +4. Creates a Powertools logger instance from the factory + +## 7. Building and Deploying the Lambda Function + +Build your function: + +```bash +dotnet build +``` + +Deploy the function using the AWS Lambda CLI tools: + +We started from a serverless template but we are just going to deploy a Lambda function not an API Gateway. + +First update the `aws-lambda-tools-defaults.json` file with your details: + +```json +{ + "Information": [ + ], + "profile": "", + "region": "", + "configuration": "Release", + "function-runtime": "dotnet8", + "function-memory-size": 512, + "function-timeout": 30, + "function-handler": "PowertoolsAspNetLoggerDemo", + "function-role": "arn:aws:iam::123456789012:role/my-role", + "function-name": "PowertoolsAspNetLoggerDemo" +} +``` +!!! Info "IAM Role" + Make sure to replace the `function-role` with the ARN of an IAM role that has permissions to write logs to CloudWatch. + +!!! Info + As you can see the function-handler is set to `PowertoolsAspNetLoggerDemo` which is the name of the project. + This example template uses [Executable assembly handlers](https://docs.aws.amazon.com/lambda/latest/dg/csharp-handler.html#csharp-executable-assembly-handlers) which use the assembly name as the handler. + +Then deploy the function: + +```bash +dotnet lambda deploy-function +``` + +Follow the prompts to complete the deployment. + +## 8. Testing the Function + +Test your Lambda function using the AWS CLI. +The following command simulates an API Gateway payload, more information can be found in the [AWS Lambda documentation](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html). + +```bash +dotnet lambda invoke-function PowertoolsAspNetLoggerDemo --payload '{ + "requestContext": { + "http": { + "method": "GET", + "path": "/" + } + } +}' +``` + +You should see a response and the logs in JSON format. + +```bash +Payload: +{ + "statusCode": 200, + "headers": { + "Content-Type": "text/plain; charset=utf-8" + }, + "body": "Hello from Powertools ASP.NET Core Minimal API!", + "isBase64Encoded": false +} + +Log Tail: +START RequestId: cf670319-d9c4-4005-aebc-3afd08ae01e0 Version: $LATEST +warn: Amazon.Lambda.AspNetCoreServer.AbstractAspNetCoreFunction[0] +Request does not contain domain name information but is derived from APIGatewayProxyFunction. +{ + "level": "Information", + "message": "Processing root request", + "timestamp": "2025-04-23T18:02:54.9014083Z", + "service": "powertools-aspnet-demo", + "coldStart": true, + "xrayTraceId": "1-68092b4e-352be5201ea5b15b23854c44", + "name": "AWS.Lambda.Powertools.Logging.Logger" +} +END RequestId: cf670319-d9c4-4005-aebc-3afd08ae01e0 +``` + +## 9. Advanced Logging Features + +Now that we have basic logging set up, let's explore some advanced features of Powertools Logger. + +### Adding Context with AppendKey + +You can add custom keys to all subsequent log messages: + +```csharp +app.MapGet("/users/{id}", (string id) => +{ + // Add context to all subsequent logs + Logger.AppendKey("userId", id); + Logger.AppendKey("source", "users-api"); + + logger.LogInformation("Getting user with ID: {id}", id); + + // Log a structured object + var user = new User + { + Id = id, + Name = "John Doe", + Email = "john.doe@example.com" + }; + + logger.LogInformation("User details: {@user}", user); + + return Results.Ok(user); +}); +``` + +This will add `userId` and `source` to all logs generated in this request context. +This will output: + +```bash hl_lines="19-20 32-36" +Payload: +{ + "statusCode": 200, + "headers": { + "Content-Type": "application/json; charset=utf-8" + }, + "body": "{\"id\":\"1\",\"name\":\"John Doe\",\"email\":\"john.doe@example.com\"}", + "isBase64Encoded": false +} +Log Tail: +{ + "level": "Information", + "message": "Getting user with ID: 1", + "timestamp": "2025-04-23T18:21:28.5314300Z", + "service": "powertools-aspnet-demo", + "coldStart": true, + "xrayTraceId": "1-68092fa7-64f070f7329650563b7501fe", + "name": "AWS.Lambda.Powertools.Logging.Logger", + "userId": "1", + "source": "users-api" +} +{ + "level": "Information", + "message": "User details: John Doe (1)", + "timestamp": "2025-04-23T18:21:28.6491316Z", + "service": "powertools-aspnet-demo", + "coldStart": true, + "xrayTraceId": "1-68092fa7-64f070f7329650563b7501fe", + "name": "AWS.Lambda.Powertools.Logging.Logger", + "userId": "1", + "source": "users-api", + "user": { // User object logged + "id": "1", + "name": "John Doe", + "email": "john.doe@example.com" + } +} +``` + +### Customizing Log Output + +You can customize the log output format: + +```csharp +builder.AddPowertoolsLogger(config => +{ + config.Service = "powertools-aspnet-demo"; + config.LoggerOutputCase = LoggerOutputCase.SnakeCase; // Change to snake_case + config.TimestampFormat = "yyyy-MM-dd HH:mm:ss"; // Custom timestamp format +}); +``` + +### Log Sampling for Debugging + +When you need more detailed logs for a percentage of requests: + +```csharp +// In your logger factory setup +builder.AddPowertoolsLogger(config => +{ + config.Service = "powertools-aspnet-demo"; + config.MinimumLogLevel = LogLevel.Information; // Normal level + config.SamplingRate = 0.1; // 10% of requests will log at Debug level +}); +``` + +### Structured Logging + +Powertools Logger provides excellent support for structured logging: + +```csharp +app.MapPost("/products", (Product product) => { + logger.LogInformation("Creating new product: {productName}", product.Name); + + // Log the entire object with all properties + logger.LogDebug("Product details: {@product}", product); + + // Log the ToString() of the object + logger.LogDebug("Product details: {product}", product); + + return Results.Created($"/products/{product.Id}", product); +}); + +public class Product +{ + public string Id { get; set; } = Guid.NewGuid().ToString(); + public string Name { get; set; } = string.Empty; + public decimal Price { get; set; } + public string Category { get; set; } = string.Empty; + public override string ToString() + { + return $"{Name} ({Id}) - {Category}: {Price:C}"; + } +} +``` + +### Using Log Buffering + +For high-throughput applications, you can buffer lower-level logs and only flush them when needed: + +```csharp +var logger = LoggerFactory.Create(builder => +{ + builder.AddPowertoolsLogger(config => + { + config.Service = "powertools-aspnet-demo"; + config.LogBuffering = new LogBufferingOptions + { + BufferAtLogLevel = LogLevel.Debug, + FlushOnErrorLog = true + }; + }); +}).CreatePowertoolsLogger(); + +// Usage example +app.MapGet("/process", () => { + logger.LogDebug("Debug log 1"); // Buffered + logger.LogDebug("Debug log 2"); // Buffered + + try { + // Business logic that might fail + throw new Exception("Something went wrong"); + } + catch (Exception ex) { + // This will also flush all buffered logs + logger.LogError(ex, "An error occurred"); + return Results.Problem("Processing failed"); + } + + // Manual flushing option + // Logger.FlushBuffer(); + + return Results.Ok("Processed successfully"); +}); +``` + +### Correlation IDs + +For tracking requests across multiple services: + +```csharp +app.Use(async (context, next) => { + // Extract correlation ID from headers + if (context.Request.Headers.TryGetValue("X-Correlation-ID", out var correlationId)) + { + Logger.AppendKey("correlationId", correlationId.ToString()); + } + + await next(); +}); +``` + +## 10. Best Practices for ASP.NET Minimal API Logging + +### Register Logger as a Singleton + +For better performance, you can register the Powertools Logger as a singleton: + +```csharp +// In Program.cs +builder.Services.AddSingleton(sp => { + return LoggerFactory.Create(builder => + { + builder.AddPowertoolsLogger(config => + { + config.Service = "powertools-aspnet-demo"; + }); + }).CreatePowertoolsLogger(); +}); + +// Then inject it in your handlers +app.MapGet("/example", (ILogger logger) => { + logger.LogInformation("Using injected logger"); + return "Example with injected logger"; +}); +``` + +## 11. Viewing and Analyzing Logs + +After deploying your Lambda function, you can view the logs in AWS CloudWatch Logs. The structured JSON format makes it easy to search and analyze logs. + +Here's an example of what your logs will look like: + +```json +{ + "level": "Information", + "message": "Getting user with ID: 123", + "timestamp": "2023-04-15 14:23:45.123", + "service": "powertools-aspnet-demo", + "coldStart": true, + "functionName": "PowertoolsAspNetLoggerDemo", + "functionMemorySize": 256, + "functionArn": "arn:aws:lambda:us-east-1:123456789012:function:PowertoolsAspNetLoggerDemo", + "functionRequestId": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "userId": "123" +} +``` + +## Summary + +In this tutorial, you've learned: + +1. How to set up ASP.NET Core Minimal API with AWS Lambda +2. How to integrate Powertools Logger using the LoggerFactory approach +3. How to configure and customize the logger +4. Advanced logging features like structured logging, correlation IDs, and log buffering +5. Best practices for using the logger in an ASP.NET Core application + +Powertools for AWS Lambda Logger provides structured logging that makes it easier to search, analyze, and monitor your Lambda functions, and integrates seamlessly with ASP.NET Core Minimal APIs. + +!!! tip "Next Steps" + Explore integrating Powertools Tracing and Metrics with your ASP.NET Core Minimal API to gain even more observability insights. diff --git a/docs/getting-started/logger/simple.md b/docs/getting-started/logger/simple.md new file mode 100644 index 000000000..b47ed3a07 --- /dev/null +++ b/docs/getting-started/logger/simple.md @@ -0,0 +1,329 @@ +--- +title: Simple Logging +description: Getting started with Logging +--- + +# Getting Started with AWS Lambda Powertools for .NET Logger + +This tutorial shows you how to set up a new AWS Lambda project with Powertools for .NET Logger from scratch - covering the installation of required tools through to deployment. + +## Prerequisites + +- An AWS account with appropriate permissions +- A code editor (we'll use Visual Studio Code in this tutorial) + +## 1. Installing .NET SDK + +First, let's download and install the .NET SDK. +You can find the latest version on the [.NET download page](https://dotnet.microsoft.com/download/dotnet). +Make sure to install the latest version of the .NET SDK (8.0 or later). + +Verify installation: + +```bash +dotnet --version +``` + +You should see output like `8.0.100` or similar (the version number may vary). + +## 2. Installing AWS Lambda Tools for .NET CLI + +Install the AWS Lambda .NET CLI tools: + +```bash +dotnet tool install -g Amazon.Lambda.Tools +dotnet new install Amazon.Lambda.Templates +``` + +Verify installation: + +```bash +dotnet lambda --help +``` + +You should see AWS Lambda CLI command help displayed. + +## 3. Setting up AWS CLI credentials + +Ensure your AWS credentials are configured: + +```bash +aws configure +``` + +Enter your AWS Access Key ID, Secret Access Key, default region, and output format. + +## 4. Creating a New Lambda Project + +Create a directory for your project: + +```bash +mkdir powertools-logger-demo +cd powertools-logger-demo +``` + +Create a new Lambda project using the AWS Lambda template: + +```bash +dotnet new lambda.EmptyFunction --name PowertoolsLoggerDemo +cd PowertoolsLoggerDemo/src/PowertoolsLoggerDemo +``` + +## 5. Adding the Powertools Logger Package + +Add the AWS.Lambda.Powertools.Logging and Amazon.Lambda.APIGatewayEvents packages: + +```bash +dotnet add package AWS.Lambda.Powertools.Logging +dotnet add package Amazon.Lambda.APIGatewayEvents +``` + +## 6. Implementing the Lambda Function with Logger + +Let's modify the Function.cs file to implement our function with Powertools Logger: + +```csharp +using System.Net; +using Amazon.Lambda.APIGatewayEvents; +using Amazon.Lambda.Core; +using AWS.Lambda.Powertools.Logging; + +// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class. +[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))] + +namespace PowertoolsLoggerDemo +{ + public class Function + { + /// + /// A simple function that returns a greeting + /// + /// API Gateway request object + /// Lambda context + /// API Gateway response object + [Logging(Service = "greeting-service", LogLevel = Microsoft.Extensions.Logging.LogLevel.Information)] + public async Task FunctionHandler(APIGatewayProxyRequest request, ILambdaContext context) + { + // you can {@} serialize objects to log them + Logger.LogInformation("Processing request {@request}", request); + + // You can append additional keys to your logs + Logger.AppendKey("QueryString", request.QueryStringParameters); + + // Simulate processing + string name = "World"; + if (request.QueryStringParameters != null && request.QueryStringParameters.ContainsKey("name")) + { + name = request.QueryStringParameters["name"]; + Logger.LogInformation("Custom name provided: {name}", name); + } + else + { + Logger.LogInformation("Using default name"); + } + + // Create response + var response = new APIGatewayProxyResponse + { + StatusCode = (int)HttpStatusCode.OK, + Body = $"Hello, {name}!", + Headers = new Dictionary { { "Content-Type", "text/plain" } } + }; + + Logger.LogInformation("Response successfully created"); + + return response; + } + } +} +``` + +## 7. Configuring the Lambda Project + +Let's update the aws-lambda-tools-defaults.json file with specific settings: + +```json +{ + "profile": "", + "region": "", + "configuration": "Release", + "function-runtime": "dotnet8", + "function-memory-size": 256, + "function-timeout": 30, + "function-handler": "PowertoolsLoggerDemo::PowertoolsLoggerDemo.Function::FunctionHandler", + "function-name": "powertools-logger-demo", + "function-role": "arn:aws:iam::123456789012:role/your_role_here" +} +``` + +## 8. Understanding Powertools Logger Features + +Let's examine some of the key features we've implemented: + +### Service Attribute + +The `[Logging]` attribute configures the logger for our Lambda function: + +```csharp +[Logging(Service = "greeting-service", LogLevel = Microsoft.Extensions.Logging.LogLevel.Information)] +``` + +This sets: +- The service name that will appear in all logs +- The minimum logging level + +### Structured Logging + +Powertools Logger supports structured logging with named placeholders: + +```csharp +Logger.LogInformation("Processing request {@request}", request); +``` + +This creates structured logs where `request` becomes a separate field in the JSON log output. + +### Additional Context + +You can add custom fields to all subsequent logs: + +```csharp +Logger.AppendKey("QueryString", request.QueryStringParameters); +``` + +This adds the QueryString field with the key and value from the QueryStringParameters property. +This can be an object like in the example or a simple value type. + +## 9. Building and Deploying the Lambda Function + +Build your function: + +```bash +dotnet build +``` + +Deploy the function using the AWS Lambda CLI tools: + +```bash +dotnet lambda deploy-function +``` + +The tool will use the settings from aws-lambda-tools-defaults.json. If prompted, confirm the deployment settings. + +## 10. Testing the Function + +Test your Lambda function using the AWS CLI: +You should see: `Hello, Powertools!` and the logs in JSON format. + +```bash +Payload: +{"statusCode":200,"headers":{"Content-Type":"text/plain"},"body":"Hello, Powertools!","isBase64Encoded":false} + +Log Tail: +{"level":"Information","message":"Processing request Amazon.Lambda.APIGatewayEvents.APIGatewayProxyRequest","timestamp":"2025-04-23T15:16:42.7473327Z","service":"greeting-service","cold_start":true,"function_name":"powertools-logger-demo","function_memory_size":512,"function_arn":"","function_request_id":"93f07a79-6146-4ed2-80d3-c0a06a5739e0","function_version":"$LATEST","xray_trace_id":"1-68090459-2c2aa3377cdaa9476348236a","name":"AWS.Lambda.Powertools.Logging.Logger","request":{"resource":null,"path":null,"http_method":null,"headers":null,"multi_value_headers":null,"query_string_parameters":{"name":"Powertools"},"multi_value_query_string_parameters":null,"path_parameters":null,"stage_variables":null,"request_context":null,"body":null,"is_base64_encoded":false}} +{"level":"Information","message":"Custom name provided: Powertools","timestamp":"2025-04-23T15:16:42.9064561Z","service":"greeting-service","cold_start":true,"function_name":"powertools-logger-demo","function_memory_size":512,"function_arn":"","function_request_id":"93f07a79-6146-4ed2-80d3-c0a06a5739e0","function_version":"$LATEST","xray_trace_id":"1-68090459-2c2aa3377cdaa9476348236a","name":"AWS.Lambda.Powertools.Logging.Logger","query_string":{"name":"Powertools"}} +{"level":"Information","message":"Response successfully created","timestamp":"2025-04-23T15:16:42.9082709Z","service":"greeting-service","cold_start":true,"function_name":"powertools-logger-demo","function_memory_size":512,"function_arn":"","function_request_id":"93f07a79-6146-4ed2-80d3-c0a06a5739e0","function_version":"$LATEST","xray_trace_id":"1-68090459-2c2aa3377cdaa9476348236a","name":"AWS.Lambda.Powertools.Logging.Logger","query_string":{"name":"Powertools"}} +END RequestId: 98e69b78-f544-4928-914f-6c0902ac8678 +REPORT RequestId: 98e69b78-f544-4928-914f-6c0902ac8678 Duration: 547.66 ms Billed Duration: 548 ms Memory Size: 512 MB Max Memory Used: 81 MB Init Duration: 278.70 ms +``` + +## 11. Checking the Logs + +Visit the AWS CloudWatch console to see your structured logs. You'll notice: + +- JSON-formatted logs with consistent structure +- Service name "greeting-service" in all logs +- Additional fields like "query_string" +- Cold start information automatically included +- Lambda context information (function name, memory, etc.) + +Here's an example of what your logs will look like: + +```bash +{ + "level": "Information", + "message": "Processing request Amazon.Lambda.APIGatewayEvents.APIGatewayProxyRequest", + "timestamp": "2025-04-23T15:16:42.7473327Z", + "service": "greeting-service", + "cold_start": true, + "function_name": "powertools-logger-demo", + "function_memory_size": 512, + "function_arn": "", + "function_request_id": "93f07a79-6146-4ed2-80d3-c0a06a5739e0", + "function_version": "$LATEST", + "xray_trace_id": "1-68090459-2c2aa3377cdaa9476348236a", + "name": "AWS.Lambda.Powertools.Logging.Logger", + "request": { + "resource": null, + "path": null, + "http_method": null, + "headers": null, + "multi_value_headers": null, + "query_string_parameters": { + "name": "Powertools" + }, + "multi_value_query_string_parameters": null, + "path_parameters": null, + "stage_variables": null, + "request_context": null, + "body": null, + "is_base64_encoded": false + } +} +{ + "level": "Information", + "message": "Response successfully created", + "timestamp": "2025-04-23T15:16:42.9082709Z", + "service": "greeting-service", + "cold_start": true, + "function_name": "powertools-logger-demo", + "function_memory_size": 512, + "function_arn": "", + "function_request_id": "93f07a79-6146-4ed2-80d3-c0a06a5739e0", + "function_version": "$LATEST", + "xray_trace_id": "1-68090459-2c2aa3377cdaa9476348236a", + "name": "AWS.Lambda.Powertools.Logging.Logger", + "query_string": { + "name": "Powertools" + } +} +``` + +## Advanced Logger Features + +### Correlation IDs + +Track requests across services by extracting correlation IDs: + +```csharp +[Logging(CorrelationIdPath = "/headers/x-correlation-id")] +``` + +### Customizing Log Output Format + +You can change the casing style of the logs: + +```csharp +[Logging(LoggerOutputCase = LoggerOutputCase.CamelCase)] +``` + +Options include `CamelCase`, `PascalCase`, and `SnakeCase` (default). + +## Summary + +In this tutorial, you've: + +1. Installed the .NET SDK and AWS Lambda tools +2. Created a new Lambda project +3. Added and configured Powertools Logger +4. Deployed and tested your function + +Powertools for AWS Logger provides structured logging that makes it easier to search, analyze, and monitor your Lambda functions. The key benefits are: + +- JSON-formatted logs for better machine readability +- Consistent structure across all logs +- Automatic inclusion of Lambda context information +- Ability to add custom fields for better context +- Integration with AWS CloudWatch for centralized log management + +!!! tip "Next Steps" + Explore more advanced features like custom log formatters, log buffering, and integration with other Powertools utilities like Tracing and Metrics. \ No newline at end of file diff --git a/docs/getting-started/metrics/aot.md b/docs/getting-started/metrics/aot.md new file mode 100644 index 000000000..e69de29bb diff --git a/docs/getting-started/metrics/aspnet.md b/docs/getting-started/metrics/aspnet.md new file mode 100644 index 000000000..e69de29bb diff --git a/docs/getting-started/metrics/simple.md b/docs/getting-started/metrics/simple.md new file mode 100644 index 000000000..51536a470 --- /dev/null +++ b/docs/getting-started/metrics/simple.md @@ -0,0 +1,4 @@ +--- +title: Simple Logging +description: Getting started with Logging +--- \ No newline at end of file diff --git a/docs/getting-started/tracing/aot.md b/docs/getting-started/tracing/aot.md new file mode 100644 index 000000000..e69de29bb diff --git a/docs/getting-started/tracing/simple.md b/docs/getting-started/tracing/simple.md new file mode 100644 index 000000000..51536a470 --- /dev/null +++ b/docs/getting-started/tracing/simple.md @@ -0,0 +1,4 @@ +--- +title: Simple Logging +description: Getting started with Logging +--- \ No newline at end of file diff --git a/docs/roadmap.md b/docs/roadmap.md index e88a50c15..0d537abc2 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -27,19 +27,19 @@ You can help us prioritize by [upvoting existing feature requests](https://githu Modernizing our logging capabilities to align with .NET practices and improve developer experience. -- [ ] Logger buffer implementation -- [ ] New .NET-friendly API design (Serilog-like patterns) -- [ ] Filtering and JMESPath expression support -- [ ] Documentation for SDK context.Logger vs Powertools Logger differences +- [x] Logger buffer implementation +- [x] New .NET-friendly API design ILogger and LoggerFactory support +- [x] Filtering and JMESPath expression support +- [x] Message templates #### Metrics V2 Updating metrics implementation to support latest EMF specifications and improve performance. -- [ ] Update to latest EMF specifications -- [ ] Breaking changes implementation for multiple dimensions -- [ ] Add support for default dimensions on ColdStart metric -- [ ] API updates - missing functionality that is present in Python implementation (ie: flush_metrics) +- [x] Update to latest EMF specifications +- [x] Breaking changes implementation for multiple dimensions +- [x] Add support for default dimensions on ColdStart metric +- [x] API updates - missing functionality that is present in Python implementation (ie: flush_metrics) ### Security and Production Readiness (P1) @@ -47,7 +47,7 @@ Ensuring enterprise-grade security and compatibility with latest .NET developmen - [ ] .NET 10 support from day one - [ ] Deprecation path for .NET 6 -- [ ] Scorecard implementation +- [x] Scorecard implementation - [ ] Security compliance checks on our pipeline - [ ] All utilities with end-to-end tests in our pipeline diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index f24b32faa..93b397f56 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -33,3 +33,9 @@ [data-md-color-scheme="slate"] { --md-typeset-a-color: rgb(28, 152, 152) } + +/*.md-nav__link[for] {*/ +/* font-weight: bold*/ +/*}*/ +.md-nav__link[for] { color: var(--md-default-fg-color) !important; } + diff --git a/docs/we_made_this.md b/docs/we_made_this.md index abbb7b3f7..df6299d0e 100644 --- a/docs/we_made_this.md +++ b/docs/we_made_this.md @@ -77,6 +77,19 @@ Check out the great series of videos from Rahul Nath on Powertools for AWS Lambd +## Powertools for AWS Lambda - MCP Server + +> **Author: [Michael Walmsley](https://www.linkedin.com/in/walmsles/){target="_blank"} :material-linkedin:** +This project implements an MCP server that enables Large Language Models (LLMs) to search through Powertools for AWS Lambda documentation. + +The server accesses the live documentation `search_index.json` data and re-constructs a local search index using lunr.js. This provides an identical search experience for AI Agents and returns the exact same results as a person would get on the website. + +With the index being local searches are super fast and the index is cached for the life of the server to save rebuilding used indexes. Since the MCP Server uses real search data it is capable of working for any Powertools for AWS document site so naturally supports all the runtimes. + + + +* [https://github.com/serverless-dna/powertools-mcp](https://github.com/serverless-dna/powertools-mcp){target="_blank"} + ## Workshops ### Accelerate your serverless journey with Powertools for AWS Lambda diff --git a/mkdocs.yml b/mkdocs.yml index ec2179c00..a86866a43 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -6,29 +6,30 @@ edit_uri: edit/develop/docs site_url: https://docs.powertools.aws.dev/lambda/dotnet/ nav: - - Homepage: index.md - - References: references.md - - Changelog: changelog.md - - Roadmap: roadmap.md + - Homepage: + - index.md + - References: references.md + - Changelog: changelog.md + - Roadmap: roadmap.md + - We Made This (Community): we_made_this.md + - Workshop 🆕: https://s12d.com/powertools-for-aws-lambda-workshop" target="_blank + - Getting started: + - Logging: + - getting-started/logger/simple.md + - getting-started/logger/aspnet.md + - getting-started/logger/aot.md + - Features: + - core/logging.md + - core/metrics.md + - core/tracing.md + - utilities/idempotency.md + - utilities/batch-processing.md + - Event Handler: + - core/event_handler/appsync_events.md + - utilities/parameters.md + - utilities/jmespath-functions.md - API Reference: api/" target="_blank - - We Made This (Community): we_made_this.md - - Workshop 🆕: https://s12d.com/powertools-for-aws-lambda-workshop" target="_blank - - Core utilities: - - Logging: - - core/logging.md - - core/logging-v2.md - - Metrics: - - core/metrics.md - - core/metrics-v2.md - - core/tracing.md - - Event Handler: - - core/event_handler/appsync_events.md - - Utilities: - - utilities/parameters.md - - utilities/idempotency.md - - utilities/batch-processing.md - - utilities/jmespath-functions.md - + theme: name: material font: @@ -49,14 +50,16 @@ theme: features: - header.autohide - navigation.sections - - navigation.expand - navigation.top + - navigation.tabs - navigation.instant - navigation.indexes - navigation.tracking - content.code.annotate + - content.code.copy - toc.follow - announce.dismiss + - content.tabs.link icon: repo: fontawesome/brands/github logo: media/aws-logo-light.svg