Repository navigation
fix(bigquery): pass timeout to the table lookup when loading dataframes and JSON - #18937
abhishekacharya200 wants to merge 2 commits into
Conversation
…taframe and load_table_from_json Both methods call get_table before uploading, without the caller's timeout. BigQuery's default timeout is None, so that request could hang however short a timeout was set.
There was a problem hiding this comment.
Code Review
This pull request ensures that the timeout parameter is correctly passed to the get_table lookup in both load_table_from_dataframe and load_table_from_json methods of the BigQuery client. Corresponding unit tests have been added to verify this behavior. I have no feedback to provide as there are no review comments.
|
Thanks for opening a PR @abhishekacharya200, please could you take a look at the mypy failure in https://github.com/googleapis/google-cloud-python/actions/runs/41204145322/job/126019710987?pr=18937? We may need something like this |
load_table_from_dataframe and load_table_from_json accept a (connect, read) tuple, while get_table is annotated as float. get_table passes the value to requests, which accepts both, so only the annotation needs bridging.
|
Thanks @parthea, fixed. The load methods allow a (connect, read) tuple and get_table is typed as float, so I cast it there; requests accepts both. mypy passes locally now. |
Fixes #18936
load_table_from_dataframeandload_table_from_jsontake atimeout, but theget_table(destination)call they make before uploading didn't receive it. BigQuery'sDEFAULT_TIMEOUTisNone, so that request had no timeout at all and could hang however short a timeout the caller set.This passes
timeoutto both lookups. Callers who don't set one still getNone, as before. Both methods typetimeoutasResumableTimeoutType, which can be a(connect, read)tuple;get_tablehands it to the same HTTP layer, so a tuple works there too, though the tests only use a float.Added
test_load_table_from_dataframe_passes_timeout_to_table_lookupandtest_load_table_from_json_passes_timeout_to_table_lookup, which fail onmainand pass with this change.tests/unit/test_client.pypasses (285 passed, 1 skipped). black 23.7.0 and flake8 report nothing on the changed files.This is the same gap as #18767, in two other methods, and the two PRs don't touch the same lines.