Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Powertools Logger v2 #832

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
01638c9
refactor to support ILogger
hjgraca Mar 6, 2025
477ad84
refactor: change Logger class to static and enhance logging capabilities
hjgraca Mar 17, 2025
1e1716e
feat(logging): add GetLogOutput method and CompositeJsonTypeInfoResol…
hjgraca Mar 18, 2025
a494c2b
refactor(logging): enhance IsEnabled method for improved log level ha…
hjgraca Mar 18, 2025
8c03866
feat(logging): introduce custom logger output and enhance configurati…
hjgraca Mar 18, 2025
cfd2b28
add log formatting
hjgraca Mar 19, 2025
70ba886
combine context, remove json key
hjgraca Mar 19, 2025
1f5ea65
add new LoggerFactory, add builder
hjgraca Mar 19, 2025
d865fa2
refactor builder, configurations add timestamp format
hjgraca Mar 19, 2025
c635752
log formatter
hjgraca Mar 19, 2025
490a449
update to PowertoolsLoggerProvider
hjgraca Mar 20, 2025
498f3e1
log buffering 1
hjgraca Mar 20, 2025
7a28645
buffer 2
hjgraca Mar 21, 2025
9d48307
buffer 3
hjgraca Mar 21, 2025
29690e2
buffer4
hjgraca Mar 21, 2025
fc7b26b
refactor: clean up whitespace and improve logger configuration handling
hjgraca Mar 22, 2025
b4741c0
fix net8 serializer
hjgraca Mar 23, 2025
2495098
refactor: update log buffering options and improve serializer handling
hjgraca Mar 24, 2025
e497563
refactor configuration and create StringCaseExtensions
hjgraca Mar 25, 2025
a742e5d
checkpoint, some issues with buffer and loglevel
hjgraca Mar 28, 2025
e7dc704
fix buffering
hjgraca Mar 29, 2025
c43c165
refactor: enhance logger configuration and output handling. Fix tests
hjgraca Mar 31, 2025
2a8da60
tests green
hjgraca Apr 1, 2025
bb9166d
refactor: replace SystemWrapper with ConsoleWrapper in tests and upda…
hjgraca Apr 1, 2025
6b8f843
refactor: replace SystemWrapper with ConsoleWrapper in tests and upda…
hjgraca Apr 1, 2025
34f2946
loggeroutput
hjgraca Apr 1, 2025
49ed301
refactor: improve logging buffer management and configuration handling
hjgraca Apr 2, 2025
8abfa70
update and add tests. refactor logger formatter to support @ and non …
hjgraca Apr 2, 2025
30672a7
refactor: update parameter names and improve documentation in logging…
hjgraca Apr 2, 2025
4662622
refactor: update logger factory and builder to support log output con…
hjgraca Apr 2, 2025
8d5195e
refactor: enhance log buffer management to discard oversized entries …
hjgraca Apr 2, 2025
adb331a
log buffering - remove enabled property;Tracer-Id env variable;Buffer…
hjgraca Apr 3, 2025
08c50a4
override console logger level on lambda.
hjgraca Apr 3, 2025
c01408e
fix xraytraceid now from env
hjgraca Apr 3, 2025
5bad7b6
refactor aspect to use MethodWrapper.FlushBufferOnUncaughtError feature
hjgraca Apr 4, 2025
c77c8be
Logging Aspect with MethodWrapper. Log buffering with clear old invoc…
hjgraca Apr 4, 2025
8e86735
Remove duplicate keys.Add universalwrapper and attribute. Fix aspect …
hjgraca Apr 5, 2025
91195a4
Update all projects for new release target.
hjgraca Apr 7, 2025
c03a30c
Merge branch 'develop' into feature/logger-ilogger-instance
hjgraca Apr 7, 2025
6331ba7
fix(build): update ProjectReference condition to always include AWS.L…
hjgraca Apr 7, 2025
eb19473
fix(tests): update AWS_EXECUTION_ENV version in assertions to 1.0.0
hjgraca Apr 7, 2025
45a2211
feat(logger): enhance random number generation and improve regex matc…
hjgraca Apr 7, 2025
f093633
sonar fixes and doc updates
hjgraca Apr 7, 2025
738dbf5
sonar refactor
hjgraca Apr 7, 2025
fe360e1
test coverage
hjgraca Apr 7, 2025
99073ba
fix console tests
hjgraca Apr 7, 2025
98659ce
more test coverage
hjgraca Apr 7, 2025
592b931
fix sonar add coverage
hjgraca Apr 7, 2025
8e2e167
fix test
hjgraca Apr 7, 2025
2aa6cfc
merge upstream
hjgraca Apr 8, 2025
399ba36
remove duplicate
hjgraca Apr 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,529 changes: 1,529 additions & 0 deletions docs/core/logging-v2.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/core/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Powertools for AWS Lambda (.NET) are available as NuGet packages. You can instal

* [AWS.Lambda.Powertools.Logging](https://www.nuget.org/packages?q=AWS.Lambda.Powertools.Logging):

`dotnet add package AWS.Lambda.Powertools.Logging`
`dotnet add package AWS.Lambda.Powertools.Logging --version 1.6.5`

## Getting started

Expand Down
15 changes: 15 additions & 0 deletions libraries/AWS.Lambda.Powertools.sln
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "AWS.Lambda.Powertools.Metri
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Metrics", "Metrics", "{A566F2D7-F8FE-466A-8306-85F266B7E656}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "AOT-Function-ILogger", "tests\e2e\functions\core\logging\AOT-Function-ILogger\src\AOT-Function-ILogger\AOT-Function-ILogger.csproj", "{7FC6DD65-0352-4139-8D08-B25C0A0403E3}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -548,6 +550,18 @@ Global
{F8F80477-1EAD-4C5C-A329-CBC0A60C7CAB}.Release|x64.Build.0 = Release|Any CPU
{F8F80477-1EAD-4C5C-A329-CBC0A60C7CAB}.Release|x86.ActiveCfg = Release|Any CPU
{F8F80477-1EAD-4C5C-A329-CBC0A60C7CAB}.Release|x86.Build.0 = Release|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Debug|Any CPU.Build.0 = Debug|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Debug|x64.ActiveCfg = Debug|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Debug|x64.Build.0 = Debug|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Debug|x86.ActiveCfg = Debug|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Debug|x86.Build.0 = Debug|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Release|Any CPU.ActiveCfg = Release|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Release|Any CPU.Build.0 = Release|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Release|x64.ActiveCfg = Release|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Release|x64.Build.0 = Release|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Release|x86.ActiveCfg = Release|Any CPU
{7FC6DD65-0352-4139-8D08-B25C0A0403E3}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection

GlobalSection(NestedProjects) = preSolution
Expand Down Expand Up @@ -596,5 +610,6 @@ Global
{A566F2D7-F8FE-466A-8306-85F266B7E656} = {1CFF5568-8486-475F-81F6-06105C437528}
{F8F80477-1EAD-4C5C-A329-CBC0A60C7CAB} = {A566F2D7-F8FE-466A-8306-85F266B7E656}
{A422C742-2CF9-409D-BDAE-15825AB62113} = {A566F2D7-F8FE-466A-8306-85F266B7E656}
{7FC6DD65-0352-4139-8D08-B25C0A0403E3} = {4EAB66F9-C9CB-4E8A-BEE6-A14CD7FDE02F}
EndGlobalSection
EndGlobal
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@
<PackageReference Include="Amazon.Lambda.DynamoDBEvents" />
<PackageReference Include="Amazon.Lambda.KinesisEvents" />
<PackageReference Include="Amazon.Lambda.SQSEvents" />
<ProjectReference Include="..\AWS.Lambda.Powertools.Common\AWS.Lambda.Powertools.Common.csproj" Condition="'$(Configuration)'=='Debug'"/>
<ProjectReference Include="..\AWS.Lambda.Powertools.Common\AWS.Lambda.Powertools.Common.csproj"/>
</ItemGroup>
</Project>
59 changes: 54 additions & 5 deletions libraries/src/AWS.Lambda.Powertools.Common/Core/ConsoleWrapper.cs
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,67 @@
*/

using System;
using System.IO;

namespace AWS.Lambda.Powertools.Common;

/// <inheritdoc />
public class ConsoleWrapper : IConsoleWrapper
{
private static bool _override;

/// <inheritdoc />
public void WriteLine(string message) => Console.WriteLine(message);
/// <inheritdoc />
public void Debug(string message) => System.Diagnostics.Debug.WriteLine(message);
public void WriteLine(string message)
{
OverrideLambdaLogger();
Console.WriteLine(message);
}

/// <inheritdoc />
public void Error(string message) => Console.Error.WriteLine(message);
public void Debug(string message)
{
OverrideLambdaLogger();
System.Diagnostics.Debug.WriteLine(message);
}

Check warning on line 38 in libraries/src/AWS.Lambda.Powertools.Common/Core/ConsoleWrapper.cs

View check run for this annotation

Codecov / codecov/patch

libraries/src/AWS.Lambda.Powertools.Common/Core/ConsoleWrapper.cs#L35-L38

Added lines #L35 - L38 were not covered by tests

/// <inheritdoc />
public string ReadLine() => Console.ReadLine();
public void Error(string message)
{
if (!_override)
{
var errordOutput = new StreamWriter(Console.OpenStandardError());
errordOutput.AutoFlush = true;
Console.SetError(errordOutput);
}

Check warning on line 48 in libraries/src/AWS.Lambda.Powertools.Common/Core/ConsoleWrapper.cs

View check run for this annotation

Codecov / codecov/patch

libraries/src/AWS.Lambda.Powertools.Common/Core/ConsoleWrapper.cs#L44-L48

Added lines #L44 - L48 were not covered by tests

Console.Error.WriteLine(message);
}

internal static void SetOut(StringWriter consoleOut)
{
_override = true;
Console.SetOut(consoleOut);
}

private void OverrideLambdaLogger()
{
if (_override)
{
return;
}
// Force override of LambdaLogger
var standardOutput = new StreamWriter(Console.OpenStandardOutput());
standardOutput.AutoFlush = true;
Console.SetOut(standardOutput);
}

internal static void WriteLine(string logLevel, string message)
{
Console.WriteLine($"{DateTime.UtcNow:yyyy-MM-ddTHH:mm:ss.fffZ}\t{logLevel}\t{message}");
}

public static void ResetForTest()

Check warning on line 76 in libraries/src/AWS.Lambda.Powertools.Common/Core/ConsoleWrapper.cs

View workflow job for this annotation

GitHub Actions / build

Missing XML comment for publicly visible type or member 'ConsoleWrapper.ResetForTest()'

Check warning on line 76 in libraries/src/AWS.Lambda.Powertools.Common/Core/ConsoleWrapper.cs

View workflow job for this annotation

GitHub Actions / build

Missing XML comment for publicly visible type or member 'ConsoleWrapper.ResetForTest()'
{
_override = false;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,4 @@ public interface IConsoleWrapper
/// </summary>
/// <param name="message">The error message to write.</param>
void Error(string message);

/// <summary>
/// Reads the next line of characters from the standard input stream.
/// </summary>
/// <returns>The next line of characters from the input stream, or null if no more lines are available.</returns>
string ReadLine();
}
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,10 @@ public interface IPowertoolsEnvironment
/// <typeparam name="T"></typeparam>
/// <returns>Assembly Version in the Major.Minor.Build format</returns>
string GetAssemblyVersion<T>(T type);

/// <summary>
/// Sets the execution Environment Variable (AWS_EXECUTION_ENV)
/// </summary>
/// <param name="type"></param>
void SetExecutionEnvironment<T>(T type);
}
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
*
* Licensed under the Apache License, Version 2.0 (the "License").
* You may not use this file except in compliance with the License.
* A copy of the License is located at
*
*
* http://aws.amazon.com/apache2.0
*
*
* or in the "license" file accompanying this file. This file is distributed
* on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
* express or implied. See the License for the specific language governing
Expand All @@ -25,6 +25,8 @@ namespace AWS.Lambda.Powertools.Common;
/// <seealso cref="IPowertoolsConfigurations" />
public class PowertoolsConfigurations : IPowertoolsConfigurations
{
private readonly IPowertoolsEnvironment _powertoolsEnvironment;

/// <summary>
/// The maximum dimensions
/// </summary>
Expand All @@ -40,26 +42,21 @@ public class PowertoolsConfigurations : IPowertoolsConfigurations
/// </summary>
private static IPowertoolsConfigurations _instance;

/// <summary>
/// The system wrapper
/// </summary>
private readonly ISystemWrapper _systemWrapper;

/// <summary>
/// Initializes a new instance of the <see cref="PowertoolsConfigurations" /> class.
/// </summary>
/// <param name="systemWrapper">The system wrapper.</param>
internal PowertoolsConfigurations(ISystemWrapper systemWrapper)
/// <param name="powertoolsEnvironment"></param>
internal PowertoolsConfigurations(IPowertoolsEnvironment powertoolsEnvironment)
{
_systemWrapper = systemWrapper;
_powertoolsEnvironment = powertoolsEnvironment;
}

/// <summary>
/// Gets the instance.
/// </summary>
/// <value>The instance.</value>
public static IPowertoolsConfigurations Instance =>
_instance ??= new PowertoolsConfigurations(SystemWrapper.Instance);
_instance ??= new PowertoolsConfigurations(PowertoolsEnvironment.Instance);

/// <summary>
/// Gets the environment variable.
Expand All @@ -68,7 +65,7 @@ internal PowertoolsConfigurations(ISystemWrapper systemWrapper)
/// <returns>System.String.</returns>
public string GetEnvironmentVariable(string variable)
{
return _systemWrapper.GetEnvironmentVariable(variable);
return _powertoolsEnvironment.GetEnvironmentVariable(variable);
}

/// <summary>
Expand All @@ -79,7 +76,7 @@ public string GetEnvironmentVariable(string variable)
/// <returns>System.String.</returns>
public string GetEnvironmentVariableOrDefault(string variable, string defaultValue)
{
var result = _systemWrapper.GetEnvironmentVariable(variable);
var result = _powertoolsEnvironment.GetEnvironmentVariable(variable);
return string.IsNullOrWhiteSpace(result) ? defaultValue : result;
}

Expand All @@ -91,7 +88,7 @@ public string GetEnvironmentVariableOrDefault(string variable, string defaultVal
/// <returns>System.Int32.</returns>
public int GetEnvironmentVariableOrDefault(string variable, int defaultValue)
{
var result = _systemWrapper.GetEnvironmentVariable(variable);
var result = _powertoolsEnvironment.GetEnvironmentVariable(variable);
return int.TryParse(result, out var parsedValue) ? parsedValue : defaultValue;
}

Expand All @@ -103,7 +100,7 @@ public int GetEnvironmentVariableOrDefault(string variable, int defaultValue)
/// <returns><c>true</c> if XXXX, <c>false</c> otherwise.</returns>
public bool GetEnvironmentVariableOrDefault(string variable, bool defaultValue)
{
return bool.TryParse(_systemWrapper.GetEnvironmentVariable(variable), out var result)
return bool.TryParse(_powertoolsEnvironment.GetEnvironmentVariable(variable), out var result)
? result
: defaultValue;
}
Expand Down Expand Up @@ -161,7 +158,8 @@ public bool GetEnvironmentVariableOrDefault(string variable, bool defaultValue)
/// </summary>
/// <value>The logger sample rate.</value>
public double LoggerSampleRate =>
double.TryParse(_systemWrapper.GetEnvironmentVariable(Constants.LoggerSampleRateNameEnv), NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var result)
double.TryParse(_powertoolsEnvironment.GetEnvironmentVariable(Constants.LoggerSampleRateNameEnv),
NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var result)
? result
: 0;

Expand Down Expand Up @@ -191,7 +189,7 @@ public bool GetEnvironmentVariableOrDefault(string variable, bool defaultValue)
/// </summary>
/// <value><c>true</c> if this instance is Lambda; otherwise, <c>false</c>.</value>
public bool IsLambdaEnvironment => GetEnvironmentVariable(Constants.LambdaTaskRoot) is not null;

/// <summary>
/// Gets a value indicating whether [tracing is disabled].
/// </summary>
Expand All @@ -202,28 +200,32 @@ public bool GetEnvironmentVariableOrDefault(string variable, bool defaultValue)
/// <inheritdoc />
public void SetExecutionEnvironment<T>(T type)
{
_systemWrapper.SetExecutionEnvironment(type);
_powertoolsEnvironment.SetExecutionEnvironment(type);
}

/// <inheritdoc />
public bool IdempotencyDisabled =>
GetEnvironmentVariableOrDefault(Constants.IdempotencyDisabledEnv, false);

/// <inheritdoc />
public string BatchProcessingErrorHandlingPolicy => GetEnvironmentVariableOrDefault(Constants.BatchErrorHandlingPolicyEnv, "DeriveFromEvent");
public string BatchProcessingErrorHandlingPolicy =>
GetEnvironmentVariableOrDefault(Constants.BatchErrorHandlingPolicyEnv, "DeriveFromEvent");

/// <inheritdoc />
public bool BatchParallelProcessingEnabled => GetEnvironmentVariableOrDefault(Constants.BatchParallelProcessingEnabled, false);
public bool BatchParallelProcessingEnabled =>
GetEnvironmentVariableOrDefault(Constants.BatchParallelProcessingEnabled, false);

/// <inheritdoc />
public int BatchProcessingMaxDegreeOfParallelism => GetEnvironmentVariableOrDefault(Constants.BatchMaxDegreeOfParallelismEnv, 1);
public int BatchProcessingMaxDegreeOfParallelism =>
GetEnvironmentVariableOrDefault(Constants.BatchMaxDegreeOfParallelismEnv, 1);

/// <inheritdoc />
public bool BatchThrowOnFullBatchFailureEnabled => GetEnvironmentVariableOrDefault(Constants.BatchThrowOnFullBatchFailureEnv, true);
public bool BatchThrowOnFullBatchFailureEnabled =>
GetEnvironmentVariableOrDefault(Constants.BatchThrowOnFullBatchFailureEnv, true);

/// <inheritdoc />
public bool MetricsDisabled => GetEnvironmentVariableOrDefault(Constants.PowertoolsMetricsDisabledEnv, false);

/// <inheritdoc />
public bool IsColdStart => LambdaLifecycleTracker.IsColdStart;

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using System;
using System.Text;

namespace AWS.Lambda.Powertools.Common;

Expand Down Expand Up @@ -40,4 +41,52 @@
var version = type.GetType().Assembly.GetName().Version;
return version != null ? $"{version.Major}.{version.Minor}.{version.Build}" : string.Empty;
}

/// <inheritdoc />
public void SetExecutionEnvironment<T>(T type)
{
const string envName = Constants.AwsExecutionEnvironmentVariableName;
var envValue = new StringBuilder();
var currentEnvValue = GetEnvironmentVariable(envName);
var assemblyName = ParseAssemblyName(GetAssemblyName(type));

// If there is an existing execution environment variable add the annotations package as a suffix.
if (!string.IsNullOrEmpty(currentEnvValue))
{
// Avoid duplication - should not happen since the calling Instances are Singletons - defensive purposes
if (currentEnvValue.Contains(assemblyName))
{
return;
}

envValue.Append($"{currentEnvValue} ");
}

var assemblyVersion = GetAssemblyVersion(type);

envValue.Append($"{assemblyName}/{assemblyVersion}");

SetEnvironmentVariable(envName, envValue.ToString());
}

/// <summary>
/// Parsing the name to conform with the required naming convention for the UserAgent header (PTFeature/Name/Version)
/// Fallback to Assembly Name on exception
/// </summary>
/// <param name="assemblyName"></param>
/// <returns></returns>
private string ParseAssemblyName(string assemblyName)
{
try
{
var parsedName = assemblyName.Substring(assemblyName.LastIndexOf(".", StringComparison.Ordinal) + 1);
return $"{Constants.FeatureContextIdentifier}/{parsedName}";
}
catch
{

Check warning on line 86 in libraries/src/AWS.Lambda.Powertools.Common/Core/PowertoolsEnvironment.cs

View check run for this annotation

Codecov / codecov/patch

libraries/src/AWS.Lambda.Powertools.Common/Core/PowertoolsEnvironment.cs#L85-L86

Added lines #L85 - L86 were not covered by tests
//NOOP
}

Check warning on line 88 in libraries/src/AWS.Lambda.Powertools.Common/Core/PowertoolsEnvironment.cs

View check run for this annotation

Codecov / codecov/patch

libraries/src/AWS.Lambda.Powertools.Common/Core/PowertoolsEnvironment.cs#L88

Added line #L88 was not covered by tests

return $"{Constants.FeatureContextIdentifier}/{assemblyName}";

Check warning on line 90 in libraries/src/AWS.Lambda.Powertools.Common/Core/PowertoolsEnvironment.cs

View check run for this annotation

Codecov / codecov/patch

libraries/src/AWS.Lambda.Powertools.Common/Core/PowertoolsEnvironment.cs#L90

Added line #L90 was not covered by tests
}
}
Loading
Loading