Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provided functionality and support for BigQuery Storage Write API #2052

Open
jlynchMicron opened this issue Nov 5, 2024 · 1 comment
Open
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@jlynchMicron
Copy link

Is your feature request related to a problem? Please describe.
No, just lacking support.

Describe the solution you'd like
Please provide client functions and code examples of using the BigQuery Storage Write API, especially in regards to streaming writes. The only streaming write function I see is "client.insert_rows_json()" which uses the legacy streaming API. It would be nice to have a similarly easy to use 1-line function call to use the new BigQuery Storage Write API.

This documentation only shows examples for java and nodejs: https://cloud.google.com/bigquery/docs/write-api-streaming

Describe alternatives you've considered
Just using legacy streaming API.

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Nov 5, 2024
@jlynchMicron
Copy link
Author

Additional support for Pandas DataFrame upload using streaming inserts instead of batch load inserts would be nice too, to avoid hitting quota limits on batch load jobs when trying to insert lots of small DataFrames from various clients in a real-time fashion.

@whuffman36 whuffman36 removed their assignment Nov 18, 2024
@Linchin Linchin added priority: p3 Desirable enhancement or fix. May not be included in next release. type: question Request for information or clarification. Not an issue. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed type: question Request for information or clarification. Not an issue. labels Nov 20, 2024
@Linchin Linchin self-assigned this Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

3 participants