-
Notifications
You must be signed in to change notification settings - Fork 642
feat: add QueryJobConfig properties to bigquery backend specified at query time #11255
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Resolves #11231 . Right? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, just a quick note about naming, but overall this looks like a good change.
ibis/backends/bigquery/__init__.py
Outdated
query: str, | ||
params: dict | None = None, | ||
job_id: str | None = None, | ||
query_job_props: dict | None = None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's call it query_job_config
to match the Google API naming convention.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I updated to a different approach which is better. My rationale for using the props
naming is no longer relevant, so I also updated to query_job_config
.
We would need to do something with the |
@dlstadther I believe it does close that one. @dtran-im the read methods allow for |
@cpcloud It appears the linting is failing both with and without a blank line. Do you know how to properly format this to make ruff happy? |
@cpcloud I think I figured out the lint issue, but I'm still not understanding the MySQL backend test failures:
It doesn't feel like I touched that code. As for the SQLite failure: I don't see any test failures, but the tests are timing out when nearly complete. I expect the timeout duration will need to be extended. |
Description of changes
Add a parameter to the query level methods in the Big Query backend to specify the
QueryJobConfig
. The following methods are affected:raw_sql
_to_query
to_pyarrow
to_pyarrow_batches
execute
Each is given an optional named parameter
query_job_config
. Ifparams
is also provided, the query parameters in both will be merged whereparams
takes precedence.