Skip to content

bug: array literal names are too long for bigquery #8931

Closed
@cpcloud

Description

@cpcloud

What happened?

In [9]: from ibis.interactive import *

In [10]: con = ibis.connect("bigquery://ibis-gbq/workflows")

In [11]: con.execute(ibis.literal([1.0, 2.0, 3.0] * 100).unnest())
...
BadRequest: 400 Invalid field name "1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_...". Fields must contain the allowed characters, and be at most 300 characters long. For allowed characters, please refer to https://cloud.google.com/bigquery/docs/schemas#column_names; reason: invalid, message: Invalid field name "1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_0_ 2_0_ 3_0_ 1_...". Fields must contain the allowed characters, and be at most 300 characters long. For allowed characters, please refer to https://cloud.google.com/bigquery/docs/schemas#column_names

Ideally we can avoid this problem in general, but it should suffice to truncate literal value names up to some small fixed size and add a unique identifier on the end to allow disambiguation.

What version of ibis are you using?

main

What backend(s) are you using, if any?

BigQuery

Relevant log output

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bigqueryThe BigQuery backendbugIncorrect behavior inside of ibis

    Type

    No type

    Projects

    Status

    done

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions