Closed
Description
Proposed solution
Steps:
- Modify S3 config to support both:
- make
dataset
,path_pattern
andprovider
not required andairbyte_hidden=True
to all V3 fields updateThis does not seem relevantorder
so that it does not clash with V4- have
SourceFilesAbstract.spec
add allsource_s3.v4.config.Config.schema
required toconnectionSpecification
and consider those fields when reading the config
- make
- Have the
read_config
method emit an AirbyteControlMessage with the new config - Deploy
Expected result:
- There are no V3 configurations anymore
User impact:
- Once deployed, if a user wants to edit an already existing config before a new sync has been executed, he will have to re-enter all the configuration hence we should make sure the time between those steps is as little a possible
- If the user updates from <3.1.X to V4, the migration result is undetermined. Based on @flash1293 in most cases, it should be fine as the spec hasn't changed much between v1.X and now but we have only tested with clearly V3 configs
Acceptance criteria
- End state:
- migration is still in place and will migrate configs. Eventually, we could remove it
- Spec visible to the user is V4 spec. V3 specific fields are hidden
- Update the breaking change to mention that some fields might be empty at the beginning (if a sync has not been executed yet)