Support aliased Destinations or Sources (for State Caches)

[Guilherme-B]

The current implementation of the DestinationStreamStateModel assumes a one-to-one relationship between a Source name and a Destination name. This is fine in principle, under the assumption that the Connection only has one instance of the Source type. However, this is not always the case, lets imagine we have multiple cases of a Google Sheets Source, for which we want to keep its state tracked via state_cache when writing to a destination. How can we achieve this?

In Airbyte, we achieve this by aliasing the Sources and Destinations:

By separating the ORM model to write based on the aliased Source and Destination/Cache names, we can support multiple types of Sources and Destinations and still hold cache for them.

[aaronsteers]

@Guilherme-B Yes, I think this makes a lot of sense. Thanks for logging this.

I could see get_source() and get_destination() accepting an optional 'alias' keyword arg. That would be a fairly straightforward implementation, I think. And then the cache would use connector alias when provided, otherwise connector name. Wdyt?

[aaronsteers]

@Guilherme-B - Do you by any chance have cycles to pick this up? I've added accepting_pull_requests label to indicate I think it is ready to move forward (assuming the spec above sounds approximately correct).

[Guilherme-B]

Hey @aaronsteers , I have created a bypass for this by renaming both the source and destinations on the fly, it works given the cache uses those names to write the state. However I don't feel like this is a proper solution.

Will investigate the usage of an alias property and create a PR for it!