Skip to content

Change the settings page in the UI to display the union of the schema and the configured schema #9902

Closed
@cgardens

Description

@cgardens

Epic: #9895

Tell us about the problem you're trying to solve

Currently the UI gets the catalog for a source from sources/discover_schema. It is then used to create a connection connections/create.

The configured catalog can be lossy. For example, if the catalog from discover_schema has stream1 and stream2, but then the connection only configures stream1, then the user will never be able to see that stream again in the UI (unless they click force refresh schema). This is very confusing. Our goal is not forget this information. Thus, when loading the settings page, we should use the information page and call sources/discover_schema. With the new caching behavior in this endpoint (#9901), this will be cheap. We can then use that catalog in the settings page and then mask the actual configuration on top of it. If needed, we can do this on the backend first. We should discuss the right interface here.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions