Skip to content

Commit

Permalink
fix: avoid blocking in download thread when using BQ Storage API
Browse files Browse the repository at this point in the history
This prevents a deadlock between the main thead and download threads
when the threadpool is shutdown prematurely.
  • Loading branch information
kien-truong committed Sep 28, 2024
1 parent 02706e2 commit a3486d7
Showing 1 changed file with 8 additions and 3 deletions.
11 changes: 8 additions & 3 deletions google/cloud/bigquery/_pandas_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -795,10 +795,15 @@ def _download_table_bqstorage_stream(
rowstream = reader.rows(session)

for page in rowstream.pages:
if download_state.done:
return
item = page_to_item(page)
worker_queue.put(item)
while True:
if download_state.done:
return
try:
worker_queue.put(item, timeout=_PROGRESS_INTERVAL)
break
except queue.Full: # pragma: NO COVER
continue


def _nowait(futures):
Expand Down

0 comments on commit a3486d7

Please sign in to comment.