Fix self-referential schema handling for collection-based types

[captainsafia]

Based on feedback from #58968 (comment) and #58968 (comment).

We need to resolve the bug for schema comparisons around the following type hierarchy:

// List<LocationContainer>
{
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "location": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "address": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "relatedLocation": {
                "type": [
                  "object",
                  "null"
                ],
                "properties": {
                  "address": {
                    "$ref": "#/items/properties/location/properties/address",
                    "x-schema-id": "AddressDto",
                    "nullable": true
                  }
                },
                "x-schema-id": "LocationDto",
                "nullable": true
              }
            },
            "x-schema-id": "AddressDto",
            "nullable": true
          }
        },
        "x-schema-id": "LocationDto",
        "nullable": true
      }
    },
    "x-schema-id": "LocationContainer"
  }
}

// LocationContainer[]
{
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "location": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "address": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "relatedLocation": {
                "type": [
                  "object",
                  "null"
                ],
                "properties": {
                  "address": {
                    "$ref": "#/items/properties/location/properties/address",
                    "x-schema-id": "AddressDto",
                    "nullable": true
                  }
                },
                "x-schema-id": "LocationDto",
                "nullable": true
              }
            },
            "x-schema-id": "AddressDto",
            "nullable": true
          }
        },
        "x-schema-id": "LocationDto",
        "nullable": true
      }
    },
    "x-schema-id": "LocationContainer"
  }
}

// Dictionary<string, LocationContainer>
{
  "type": "object",
  "additionalProperties": {
    "type": "object",
    "properties": {
      "location": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "address": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "relatedLocation": {
                "type": [
                  "object",
                  "null"
                ],
                "properties": {
                  "address": {
                    "$ref": "#/additionalProperties/properties/location/properties/address",
                    "x-schema-id": "AddressDto",
                    "nullable": true
                  }
                },
                "x-schema-id": "LocationDto",
                "nullable": true
              }
            },
            "x-schema-id": "AddressDto",
            "nullable": true
          }
        },
        "x-schema-id": "LocationDto",
        "nullable": true
      }
    },
    "x-schema-id": "LocationContainer"
  }
}

// ListContainer
{
  "type": "object",
  "properties": {
    "location": {
      "type": [
        "object",
        "null"
      ],
      "properties": {
        "address": {
          "type": [
            "object",
            "null"
          ],
          "properties": {
            "relatedLocation": {
              "type": [
                "object",
                "null"
              ],
              "properties": {
                "address": {
                  "$ref": "#/properties/location/properties/address",
                  "x-schema-id": "AddressDto",
                  "nullable": true
                }
              },
              "x-schema-id": "LocationDto",
              "nullable": true
            }
          },
          "x-schema-id": "AddressDto",
          "nullable": true
        }
      },
      "x-schema-id": "LocationDto",
      "nullable": true
    }
  },
  "x-schema-id": "LocationContainer"
}

[haverjes]

Not sure if it helps, but here's the simplest repro I have (eliminates the nullability difference from my previous one)

public class TestOpenApiResult
{
    public List<SingleItemDataResult> FirstListOfItems { get; init; }
    public List<SingleItemDataResult> SecondListOfItems {get; init;}
}

Which results in the following schema:

 "TestOpenApiResult": {
        "type": "object",
        "properties": {
          "firstListOfItems": {
            "type": "array",
            "items": {
              "$ref": "#/components/schemas/SingleItemDataResult"
            },
            "nullable": true
          },
          "secondListOfItems": {
            "type": "array",
            "items": {
              "$ref": "#/components/schemas/#/items/properties/firstListOfItems/items"
            },
            "nullable": true
          }
        },
        "nullable": true
      }
    }

[Darent98]

I am also experiencing similar issues. In the following is a minimal example how the problem occurs in my project. I don't know if this has the same cause, maybe it helps.

I created the following class in an existing project

public class ApiTest
{
     public List<ApiTest> children { get; set; }
}

Used it in a controller like

[HttpGet]
[Route("/tests")]
[ProducesResponseType(StatusCodes.Status200OK)]
public List<ApiTest> TestsGet () {
      return new List<ApiTest>() {};
}

[HttpGet]
[Route("/test")]
[ProducesResponseType(StatusCodes.Status200OK)]
public ApiTest TestGet () {
      return new ApiTest() {};
}

A part of the generated schema

"/tests": {
      "get": {
        "tags": [
          "General"
        ],
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "type": "array",
                  "items": {
                    "$ref": "#/components/schemas/ApiTest"
                  }
                }
              },
              "text/json": {
                "schema": {
                  "type": "array",
                  "items": {
                    "$ref": "#/components/schemas/ApiTest"
                  }
                }
              }
            }
          }
        },
      }
    },
    "/test": {
      "get": {
        "tags": [
          "General"
        ],
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/ApiTest2"
                }
              },
              "text/json": {
                "schema": {
                  "$ref": "#/components/schemas/ApiTest2"
                }
              }
            }
          }
        },
      }
    },

....

"ApiTest": {
        "required": [
          "children"
        ],
        "type": "object",
        "properties": {
          "children": {
            "type": "array",
            "items": {
              "$ref": "#/components/schemas/#/items"
            }
          }
        }
      },
      "ApiTest2": {
        "required": [
          "children"
        ],
        "type": "object",
        "properties": {
          "children": {
            "type": "array",
            "items": {
              "$ref": "#/components/schemas/ApiTest"
            }
          }
        }
      },

[DoubleTK]

I'm experiencing the same thing. I'm building an OTLP/HTTP receiver to handle OpenTelemetry signals. These signals have recursive, nested properties like the AnyValue, ArrayValue, and KeyValue properties. I'm happy to perform any testing or provide additional info if needed.

[zyofeng]

Same issue here we migrated from NSwag.AspNetCore, even the type names are messed up for us.

[waynebrantley]

@captainsafia I see your new V2 library works correctly already for dotnet 10 release. Is there a way we can get this fixed in 9 as this is a complete blocker. (unless you have a workaround)

[captainsafia]

OK! First of all, thanks for your patience with all these $ref issues. The bug fix for this and related issues that have been duped to it has been merged with expected release in 9.0.4. I've included test cases from the feedback items left here in our coverage for it as well. These test cases are also shared with the M.OpenApi team as they work to improve reference handling in V2.

@captainsafia I see your new V2 library works correctly already for dotnet 10 release. Is there a way we can get this fixed in 9 as this is a complete blocker. (unless you have a workaround)

The new V2 library has much more robust handling for relative references (the kind that are causing these issues) than the current implementation. There are still some remaining issues with it, as you can see from some of the test suppressions in #60822 but we're working with the M.OpenApi team to resolve those before v2 goes GA.

Since we can't consume breaking changes in .NET 9, I ended up making some tactical changes in our package to workaround some of the gaps in M.OpenApi v1.6.

[Aymeeeric]

OK! First of all, thanks for your patience with all these $ref issues. The bug fix for this and related issues that have been duped to it has been merged with expected release in 9.0.4. I've included test cases from the feedback items left here in our coverage for it as well. These test cases are also shared with the M.OpenApi team as they work to improve reference handling in V2.

@captainsafia I see your new V2 library works correctly already for dotnet 10 release. Is there a way we can get this fixed in 9 as this is a complete blocker. (unless you have a workaround)

The new V2 library has much more robust handling for relative references (the kind that are causing these issues) than the current implementation. There are still some remaining issues with it, as you can see from some of the test suppressions in #60822 but we're working with the M.OpenApi team to resolve those before v2 goes GA.

Since we can't consume breaking changes in .NET 9, I ended up making some tactical changes in our package to workaround some of the gaps in M.OpenApi v1.6.

Hi. Thx for your answer. Can we have an estimate of the release date of the 9.0.4?

[martincostello]

Tuesday 8th April is the likely date - there's typically a new release on the second Tuesday of the month for Update Tuesday.

[daniatic]

@martincostello is there a pre release I can use until then? We're not in production yet, so pre release would be fine.

[martincostello]

You could try looking in the dotnet/sdk repo for details of a daily build for 9.0.x latest.

[oPreis-Systecs]

I have now tried version 9.0.4.
This fixes a reference in my json, but unfortunately there are still duplicated schemas when arrays are used.

[martincostello]

@oPreis-Systecs Please open a new issue with a repro for your specific issue.

[oPreis-Systecs]

@oPreis-Systecs Please open a new issue with a repro for your specific issue.

Opend an issue for this:
#61408

[oskarjs]

I am now encountering a different ref issue on 9.0.4, as well as the afformentioned duplication bug detailed in #61408

Instead of creating an invalid ref it now generates nothing in the items object.

This response type:

public class Response
{
  public required List<InnerResponse> Prop1 { get; set; }
  public required List<InnerResponse> Prop2 { get; set; }
}

public class InnerResponse
{
}

generates as:

{
  "Response": {
    "required": [
      "prop1",
      "prop2"
    ],
    "type": "object",
    "properties": {
      "prop1": {
        "type": "array",
        "items": {
          "$ref": "#/components/schemas/InnerResponse"
        }
      },
      "prop2": {
        "type": "array",
        "items": {}
      }
    }
  },
  "InnerResponse": {
    "type": "object"
  }
}

[martincostello]

@oskarjs That looks the same as #61407.