Skip to content

BigQuery Denormalized Destination: Support for Date/Datetime types through the format annotation #6142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cptjacky opened this issue Sep 16, 2021 · 0 comments · Fixed by #6145

Comments

@cptjacky
Copy link
Contributor

cptjacky commented Sep 16, 2021

Tell us about the problem you're trying to solve

Trying to load dates columns in BigQuery using the BigQuery Denormalized Destination.

Describe the solution you’d like

When using the BigQuery Denormalized Destination, the generated schema should support the date/datetime types when possible (using the "format" key in the json spec).

Describe the alternative you’ve considered or used

I patched the destination

Additional context

Parsing dates on the fly is costly and I would like to avoid intermediate ("raw") tables in BigQuery to limit storage costs.

Are you willing to submit a PR?

Yes! I will be making a proposal shortly!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants