.Net: Update GeminiChatMessageContent to support System.Text.Json deserialization when calledToolResult is null

[julionav]

Motivation and Context

I am wrapping an Agent from the new agent framework in an orleans grain, and serializing ChatContentMessage with Json directly.

When using with a Gemini model, the deserialization fails with the error:

Exception: System.ArgumentNullException: Value cannot be null. (Parameter 'calledToolResult')
         at System.ArgumentNullException.Throw(String paramName)
         at System.ArgumentNullException.ThrowIfNull(Object argument, String paramName)
         at Microsoft.SemanticKernel.Connectors.Google.GeminiChatMessageContent..ctor(GeminiFunctionToolResult calledToolResult)
         at .ctor(GeminiFunctionToolResult, Object, Object, Object)
         at System.Text.Json.Serialization.Converters.SmallObjectWithParameterizedConstructorConverter`5.CreateObject(ReadStackFrame& frame)
         at System.Text.Json.Serialization.Converters.ObjectWithParameterizedConstructorConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
         at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
         at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
         at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
         at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsObject(Utf8JsonReader& reader, ReadStack& state)
         at System.Text.Json.JsonSerializer.ReadAsObject(Utf8JsonReader& reader, JsonTypeInfo jsonTypeInfo)

The problem seems to be that the deserialization is using the only only public constructor from GeminiChatMessageContent, which takes a non-nullable GeminiFunctionToolResult argument public GeminiChatMessageContent(GeminiFunctionToolResult calledToolResult). However, there are instances where the content message has a null value for that property (both of the other 2 constructors here support a null value for that property).

So, this causes the need to create a JsonConverter that looks a bit like this:

public sealed class GeminiChatMessageContentSurrogate : ChatMessageContent
{
    /// <summary>
    /// Initializes a new instance of the <see cref="GeminiChatMessageContent"/> class.
    /// </summary>
    /// <param name="calledToolResult">The result of tool called by the kernel.</param>
    public GeminiChatMessageContentSurrogate() { }

    public GeminiChatMessageContent ToChatMessageContent()
    {
        if (Role == AuthorRole.Tool && Content == null && CalledToolResult != null)
        {
            return new GeminiChatMessageContent(CalledToolResult);
        }

        if (ToolCalls != null)
        {
            var toolCallsConstructor = typeof(GeminiChatMessageContent)
                .GetConstructors(
                    BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public
                )
                .First(c =>
                    c.GetParameters().Length == 5
                    && c.GetParameters()[0].ParameterType == typeof(AuthorRole)
                    && c.GetParameters()[1].ParameterType == typeof(string)
                    && c.GetParameters()[2].ParameterType == typeof(string)
                    && c.GetParameters()[3].ParameterType != typeof(GeminiFunctionToolResult)
                    && c.GetParameters()[4].ParameterType == typeof(GeminiMetadata)
                );
            return (GeminiChatMessageContent)
                toolCallsConstructor.Invoke([Role, Content, ModelId, ToolCalls, Metadata]);
        }

        var constructor = typeof(GeminiChatMessageContent)
            .GetConstructors(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public)
            .First(c =>
                c.GetParameters().Length == 5
                && c.GetParameters()[0].ParameterType == typeof(AuthorRole)
                && c.GetParameters()[1].ParameterType == typeof(string)
                && c.GetParameters()[2].ParameterType == typeof(string)
                && c.GetParameters()[3].ParameterType == typeof(GeminiFunctionToolResult)
                && c.GetParameters()[4].ParameterType == typeof(GeminiMetadata)
            );
        return (GeminiChatMessageContent)
            constructor.Invoke([Role, Content, ModelId, CalledToolResult, Metadata]);
    }

    public static GeminiChatMessageContentSurrogate FromGeminiChatMessageContent(
        GeminiChatMessageContent content
    )
    {
        return new GeminiChatMessageContentSurrogate
        {
            Role = content.Role,
            Content = content.Content,
            ModelId = content.ModelId,
            CalledToolResult = content.CalledToolResult,
            Metadata = content.Metadata,
            ToolCalls = content.ToolCalls
        };
    }

    /// <summary>
    /// A list of the tools returned by the model with arguments.
    /// </summary>
    public IReadOnlyList<GeminiFunctionToolCall>? ToolCalls { get; set; }

    /// <summary>
    /// The result of tool called by the kernel.
    /// </summary>
    public GeminiFunctionToolResult? CalledToolResult { get; set; }
}

public class GeminiChatMessageContentConverter : JsonConverter<GeminiChatMessageContent>
{
    public override GeminiChatMessageContent Read(
        ref Utf8JsonReader reader,
        Type typeToConvert,
        JsonSerializerOptions options
    )
    {
        var surrogate =
            JsonSerializer.Deserialize<GeminiChatMessageContentSurrogate>(ref reader, options)
            ?? throw new JsonException("Failed to deserialize GeminiChatMessageContentSurrogate");
        return surrogate.ToChatMessageContent();
    }

    public override void Write(
        Utf8JsonWriter writer,
        GeminiChatMessageContent value,
        JsonSerializerOptions options
    )
    {
        var surrogate = GeminiChatMessageContentSurrogate.FromGeminiChatMessageContent(value);
        JsonSerializer.Serialize(writer, surrogate, options);
    }
}

Description

I added a parameterless constructor with [JsonConstructor], similar to how is done on ChatMessageContent

Contribution Checklist

Working on these right now 🫡

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[julionav]

Well... I tried building this but got 900 fails which I am 98% sure I didn't cause.