Skip to content

Destination BigQuery: map "airbyte_type": "big_number" to BIGNUMERIC #12658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #10148
yurii-bidiuk opened this issue May 6, 2022 · 4 comments · Fixed by #19721
Closed
Tracked by #10148

Destination BigQuery: map "airbyte_type": "big_number" to BIGNUMERIC #12658

yurii-bidiuk opened this issue May 6, 2022 · 4 comments · Fixed by #19721
Assignees
Labels
normalization team/destinations Destinations team's backlog type/enhancement New feature or request

Comments

@yurii-bidiuk
Copy link
Contributor

yurii-bidiuk commented May 6, 2022

Due to new updates in the building of JSON schema, a new property "airbyte_type" was added to the field description.
One of the possible values for "airbyte_type" is "big_number". Now "airbyte_type" is ignored and all "big_number" fields are mapped as "STRING" in destination BigQuery.
According to #11090 and this comment all fields with "airbyte_type": "big_number" should be mapped as BIGNUMERIC in destination BigQuery.

Input JSON schema example:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "type_big_number": {"type": "string", "airbyte_type": "big_number"},
  }
}

Input data example:

{
  "type_big_number": "1000000000000000000000000000000000.1234"
}
@yurii-bidiuk
Copy link
Contributor Author

As datatype mapping for destiantion-bigquery should be done on the normalization side, I'm moving this ticket to the board of Phyton team
cc: @alexandr-shegeda

@grishick
Copy link
Contributor

notes from grooming: in the new protocol that supports annotations for data types, this will not be needed. Likely can close this after Destination BigQuery and normalization are updated to handle new protocol version

@etsybaev etsybaev self-assigned this Dec 5, 2022
@etsybaev
Copy link
Contributor

etsybaev commented Dec 5, 2022

Depends on https://app.zenhub.com/workspaces/destinations-6332532d6893ce001c7a56a7/issues/airbytehq/airbyte/17642

Basically should be closed along the way once this PR is merged #19721

@etsybaev etsybaev linked a pull request Dec 5, 2022 that will close this issue
37 tasks
@grishick
Copy link
Contributor

merged into the feature branch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
normalization team/destinations Destinations team's backlog type/enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants