Skip to content

[source-file] parse_dates reader option not able to parse ISO 8601 date values from csv file as data type timestamp without timezone #59199

Open
@og-foobarred

Description

@og-foobarred

Connector Name

source-file

Connector Version

0.5.28

What step the error happened?

Other

Relevant information

I am importing a CSV file that has two columns that are date fields (datepublished, datepublic) in ISO 8601 date format (2020-01-10T13:20:12). I have the following option set in Reader Options:

{"parse_dates": ["datepublished", "datepublic"]}

However those fields are not being parsed as date values. They are falling back to string values in the Airbyte schema and are being imported into Postgres as varchar dtype. I have also tried forcing data type for those fields in reader options using dtype: ["field": "datetime64"] however I get an error message saying that setting datetime64 using dtype is not supported and to use the parse_date option instead.

Relevant log output

Contribute

  • Yes, I want to contribute

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions