Skip to content

bug: array literal names are too long for bigquery #8931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 task done
cpcloud opened this issue Apr 11, 2024 · 0 comments · Fixed by #9916
Closed
1 task done

bug: array literal names are too long for bigquery #8931

cpcloud opened this issue Apr 11, 2024 · 0 comments · Fixed by #9916
Labels
bigquery The BigQuery backend bug Incorrect behavior inside of ibis
Milestone

Comments

@cpcloud
Copy link
Member

cpcloud commented Apr 11, 2024

What happened?

In [9]: from ibis.interactive import *

In [10]: con = ibis.connect("bigquery://ibis-gbq/workflows")

In [11]: con.execute(ibis.literal([1.0, 2.0, 3.0] * 100).unnest())
...
BadRequest: 400 Invalid field name "1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_...". Fields must contain the allowed characters, and be at most 300 characters long. For allowed characters, please refer to https://cloud.google.com/bigquery/docs/schemas#column_names; reason: invalid, message: Invalid field name "1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_...". Fields must contain the allowed characters, and be at most 300 characters long. For allowed characters, please refer to https://cloud.google.com/bigquery/docs/schemas#column_names

Ideally we can avoid this problem in general, but it should suffice to truncate literal value names up to some small fixed size and add a unique identifier on the end to allow disambiguation.

What version of ibis are you using?

main

What backend(s) are you using, if any?

BigQuery

Relevant log output

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@cpcloud cpcloud added the bug Incorrect behavior inside of ibis label Apr 11, 2024
@gforsyth gforsyth added the bigquery The BigQuery backend label Apr 12, 2024
@github-project-automation github-project-automation bot moved this from backlog to done in Ibis planning and roadmap Aug 26, 2024
@github-actions github-actions bot added this to the 9.4 milestone Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bigquery The BigQuery backend bug Incorrect behavior inside of ibis
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants