CDK: Schema resolver should support nested references

[keu]

Tell us about the problem you're trying to solve

The discovery command should produce resolvable schemas, i.e. without referencing any external files.
Currently we use simple algorithm as a workaround:

1. move external files to shared folder
2. during discovery we load schemas and replace any reference to shared files with the content of these files

This of course has doesn't cover all cases:

1. reference inside shared files
2. reference to definitions sections

Describe the solution you’d like

We should process all $refs recursively and generate the definitions section as Pydantic does. Ideally, we would like to use Pydantic where it is possible

Describe the alternative you’ve considered or used

none

Additional context

Example of such schemas in the FB connector:

airbyte/airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/schemas/shared/targeting.json

Line 21 in 9e260fe

"$ref": "targeting.json#/definitions/id_name_pairs"

Are you willing to submit a PR?

Your answer