GCSToBigQueryOperator 404 error

[koliankolin]

Hello!

We migrate from 2.2.3 to 2.5.2 version of Airflow and suddenly operator GCSToBigQueryOperator start to fail with error 404.
google.api_core.exceptions.NotFound: 404 Not found: Uris gs://example_gcs_object.
This didn't happen in 2.2.3 version.

Is there some parameter in a new GCSToBigQueryOperator to turn back functionality with skipping this not found objects?

[potiuk]

I think you should check release notes of google provider that got upgraded. Check what versions you had before and which you use now https://airflow.apache.org/docs/apache-airflow-providers-google/stable/index.html - the providers might introduce breaking changes on their own.

You can also downgrade/upgrede provider individually either manually https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#installing-upgrading-downgrading-providers-separately-from-airflow-core or via custom docker to see which/when it has changed.

[koliankolin]

I suppose that it's not the problem of this operator but Google API through which this operator is working

[koliankolin]

Maybe there is some option or parameter for gentle fail like it was in previous version