Summary
In this release we are excited to announce Logging 2.0.0 developer preview.
This release brings many features requested by customers:
- Support for
ILogger,LoggerFactoryand builder pattern - Message templating
{@}and{}for structured json outputs. - Support for
JsonSerializerOptions - Log Buffering
- Option to keep the original casing of nested objects
We also releasing version 1.7.0 of Logging which adds support for cold start on provisioned concurrency and a fix to log level display mismatch in CloudWatch Logs when using Top Level Statement Lambdas (Executable assembly handlers).
Thanks to @nCubed for raising the issue #783
Logging 2.0.0 (developer preview)
Breaking changes from v1
The new major version of the Logging utility introduces several additions without breaking Logger functionality. However, dependency updates are required before upgrading to version 2.x.x.
Breaking changes to dependencies (Action Required)
| Change | Before (v1.x) | After (v2.0) | Migration Action |
|---|---|---|---|
| Amazon.Lambda.Core | 2.2.0 | 2.5.0 | dotnet add package Amazon.Lambda.Core |
| Amazon.Lambda.Serialization.SystemTextJson | 2.4.3 | 2.4.4 | dotnet add package Amazon.Lambda.Serialization.SystemTextJson |
| Microsoft.Extensions.DependencyInjection | 8.0.0 | 8.0.1 | dotnet add package Microsoft.Extensions.DependencyInjection |
Logger setup
You can now setup the logger in different ways. Before this release the only way was to use the Logging attribute on your Lambda handler. From this release you can now also use the ILogger or LoggerFactory that are part of the Microsoft.Extensions.Logging package.
Using LoggerFactory
Using LoggerFactory gives you more flexibility and testability in case you want to test the Logger as well.
If you are using dependency injection have two constructors one for lambda initialisation and the other for testing or useLambda Annotations that enables a simpler dependency injection mechanism where you don’t require both constructors.
Using Builder pattern
Configuration
You can now configure the Logger outside of the decorator, you can find the full list of properties in our documentation.
This gives you more flexibility, more options, and testability.
Configuring the static Logger in the constructor example:
Configuring using ILogger:
Message Templates
You can now use message templates to extract properties from your objects and log them as structured data.
To take full advantage of message templates, override the ToString() method of your object to return a meaningful string representation of the object.
This is especially important when using {} to log the object as a string.
If you want to log the object as a JSON object, use {@}. This will serialize the object and log it as a JSON object. The Message property will contain the ToString value of that object
If you want to log the object as a string, use {}. This will call the ToString() method of the object and log it as a string.
The .NET composite formatting tokens can be used for formatting parameters. For example, if we have a cost variable with a value of 8.12345 that needs to be rounded to two decimal places in our logging, the logging call would be:
context.Logger.LogInformation("The cost is {cost:0.00}", cost);
This will produce the corresponding JSON document.
Using JsonSerializerOptions
Powertools supports customizing the serialization of Lambda JSON events and your own types using JsonSerializerOptions. You can do this by creating a custom JsonSerializerOptions and passing it to the JsonOptions of the Powertools Logger.
Supports TypeInfoResolver and DictionaryKeyPolicy options and others.
These two options are the most common ones used to customize the serialization of Powertools Logger.
- TypeInfoResolver: This option allows you to specify a custom JsonSerializerContext that contains the types you want to serialize and deserialize. This is especially useful when using AOT compilation, as it allows you to specify the types that should be included in the generated assembly.
- DictionaryKeyPolicy: This option allows you to specify a custom naming policy for the properties in the JSON output. This is useful when you want to change the casing of the property names or use a different naming convention.
Preserving the original casing of the properties
If you want to preserve the original casing of the property names (keys), you can set the DictionaryKeyPolicy to null.
Log Buffering
Log buffering enables you to buffer logs for a specific request or invocation.
Enable log buffering by passing LogBufferingOptions when configuring a Logger instance. You can buffer logs at the Warning, Information, Debug or Trace level, and flush them automatically on error or manually as needed.
- Buffer logs at the Warning, Information, Debug, and Trace levels
- Automatically flush logs on error or manually as needed. Flushing on error is useful if you would like to get additional context around errors while avoiding emitting verbose logs all the time.
- Reduce CloudWatch costs by decreasing the number of emitted log messages
Configuration options
| Option | Description | Default |
|---|---|---|
| MaxBytes | Maximum size of the log buffer in bytes | 20480 |
| BufferAtLogLevel | Minimum log level to buffer (more verbose levels are also buffered) | default |
| FlushOnErrorLog | Automatically flush buffer when logging an error | true |
When log buffering is enabled, you can now pass a new opt-in FlushBufferOnUncaughtError flag to the Logger decorator. When enabled, 1/ we'll intercept any error thrown, 2/ flush the buffer, and 3/ re-throw your original error. This enables you to have detailed logs from your application when you need them the most.
Buffering FAQs
- Does the buffer persist across Lambda invocations? No, each Lambda invocation has its own buffer. The buffer is initialized when the Lambda function is invoked and is cleared after the function execution completes or when flushed manually.
- Are my logs buffered during cold starts? No, we never buffer logs during cold starts. This is because we want to ensure that logs emitted during this phase are always available for debugging and monitoring purposes. The buffer is only used during the execution of the Lambda function.
- How can I prevent log buffering from consuming excessive memory? You can limit the size of the buffer by setting the MaxBytes option in the LogBufferingOptions constructor parameter. This will ensure that the buffer does not grow indefinitely and consume excessive memory.
- What happens if the log buffer reaches its maximum size? Older logs are removed from the buffer to make room for new logs. This means that if the buffer is full, you may lose some logs if they are not flushed before the buffer reaches its maximum size. When this happens, we emit a warning when flushing the buffer to indicate that some logs have been dropped.
- How is the log size of a log line calculated? The log size is calculated based on the size of the serialized log line in bytes. This includes the size of the log message, the size of any additional keys, and the size of the timestamp.
- What timestamp is used when I flush the logs? The timestamp preserves the original time when the log record was created. If you create a log record at 11:00:10 and flush it at 11:00:25, the log line will retain its original timestamp of 11:00:10.
- What happens if I try to add a log line that is bigger than max buffer size? The log will be emitted directly to standard output and not buffered. When this happens, we emit a warning to indicate that the log line was too big to be buffered.
- What happens if Lambda times out without flushing the buffer? Logs that are still in the buffer will be lost. If you are using the log buffer to log asynchronously, you should ensure that the buffer is flushed before the Lambda function times out. You can do this by calling the Logger.FlushBuffer() method at the end of your Lambda function.
Logger 1.7.0 - Bug fix
New
Cold start metric now correctly identifies if Lambda is running on-demand or provisioned-concurrency mode. If running on provisioned-concurrency mode cold start is always false.
Issue Fixed
Improved console output to override Lambda default console output specially in Top Level Statement Lambdas (Executable assembly handlers) to prevent the wrong LogLevel to be displayed in CloudWatch
Before: The Logging utility only override the console at initialization
After: The utility now properly overrides the console when first invoked and now the Log Level in the default Lambda logger is not sent to CloudWatch
🌟New features and non-breaking changes
- feat(workflows): update .NET version setup to support multiple versions and improve package handling (#814) by @hjgraca
📜 Documentation updates
🔧 Maintenance
- chore: Revert Common copy and setup (#844) by @hjgraca
- chore: update Microsoft.Extensions.DependencyInjection to version 8.0.1 (#843) by @hjgraca
- chore: update example verions and update test (#842) by @hjgraca
- chore: Remove version check from execution environment (#841) by @hjgraca
- chore: Fix execution env verion and ConsoleWrapper for test mode (#840) by @hjgraca
- chore: Powertools Logger v2 (#832) by @hjgraca
- chore(deps): bump actions/setup-node from 4.2.0 to 4.3.0 (#821) by @dependabot[bot]
- chore(deps): bump actions/setup-dotnet from 4.3.0 to 4.3.1 (#820) by @dependabot[bot]
- chore: Fix Console output when running on Top Level Statements (#835) by @hjgraca
- chore: Cold start with provisioned concurrency support (#834) by @hjgraca
- feat(workflows): update .NET version setup to support multiple versions and improve package handling (#814) by @hjgraca
- chore: Update examples for release 1.30 and add workflows (#813) by @hjgraca
This release was made possible by the following contributors:
@dependabot[bot], @hjgraca and dependabot[bot]
