Source S3: no longer supports tab-delimited (TSV) configurations

[dashirov]

Connector Name

source-s3

Connector Version

4.2.0

What step the error happened?

Configuring a new connector

Relevant information

The connector no longer honors '\t' (tab) into delimiter control of the configuration user interface. This was previously addressed in a workaround pull request #9163. When configuring the connector for TSV files:

1. Documentation insists on use of a two character sequence \t
2. UI Configuration modal accepts a two character sequence \t
3. JSON-serialized configuration states the delimiter is serialized as a string "\t"
4. An attempt to test configuration fails with error "Configuration check failed Something went wrong in the connector. See the logs for more details."

check_connection_source-failure-e1256f04-27d5-4218-9ad2-3b789e07be1e.txt

5. Integration tests do not feature tab-separated files as use cases

Relevant log output

unexpected value; permitted: 'parquet' (type=value_error.const; given=csv; permitted=['parquet'])
streams -> 2 -> format -> filetype
  unexpected value; permitted: 'unstructured' (type=value_error.const; given=csv; permitted=['unstructured'])
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/airbyte/integration_code/main.py", line 42, in <module>
    launch(source, _args)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 209, in launch
    for message in source_entrypoint.run(parsed_args):
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 109, in run
    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.check(source_spec, config))
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 133, in check
    check_result = self.source.check(self.logger, config)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 82, in check
    check_succeeded, error = self.check_connection(logger, config)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/file_based_source.py", line 63, in check_connection
    streams = self.streams(config)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/file_based_source.py", line 114, in streams
    raise ConfigValidationError(FileBasedSourceError.CONFIG_VALIDATION_ERROR) from exc
airbyte_cdk.sources.file_based.exceptions.ConfigValidationError: Error creating stream config object. Contact Support if you need assistance.

Contribute

• Yes, I want to contribute

### Tasks

[dashirov]

[dashirov]

I would say a patch may look like this:

Index: airbyte-integrations/connectors/source-s3/source_s3/v4/source.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/airbyte-integrations/connectors/source-s3/source_s3/v4/source.py b/airbyte-integrations/connectors/source-s3/source_s3/v4/source.py
--- a/airbyte-integrations/connectors/source-s3/source_s3/v4/source.py	(revision e67543cb66b18cefb5c5ac3a9cee16466844f725)
+++ b/airbyte-integrations/connectors/source-s3/source_s3/v4/source.py	(date 1699673822308)
@@ -27,6 +27,8 @@
         validate the config against the new spec.
         """
         config = super().read_config(config_path)
+        if config.get("format", {}).get("delimiter") == r"\t":
+            config["format"]["delimiter"] = "\t"
         if not self._is_v4_config(config):
             parsed_legacy_config = SourceS3Spec(**config)
             converted_config = LegacyConfigTransformer.convert(parsed_legacy_config)

But I'm not too familiar with airbyte_cdk and how things are done here.