-
Notifications
You must be signed in to change notification settings - Fork 75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
redshift-connector
giving connection time-outs on Codebuild.
#212
Comments
Issue is also seen from various other members of the DBT community using Redshift; since they migrated to use redshift_connector over pyscopg2. Queries that consistently time out locally from me, run fine in SQL workbench type tooling (say 15minute query runs) and never time out. |
Hi folks, thanks for the issue report. I took a look at the associated DBT issue and see it has been closed out as a suspected network issue with Codebuild. Regardless, I understand that folks were not seeing this issue with psycopg2 and see it with redshift-connector when using the driver alone with Codebuild. What's weird to me is that issues are seen with Given the repro above is using username/password, redshift-connector is essentially just creating a socket to the host and port provided. As the timeout is occurring on connection, and it only presents on codebuild, I'm inclined to believe this issue is related to the networking settings configured on the CodeBuild machine. Here's a list of possible things to try that can help us rule out potential issues:
If you don't see resolution, please let me know and the team can investigate further. |
Sorry to add confusion with the DBT issue; it was closed before most of the comments were added. Various sources display issues like like Github Actions, Circle CI, but the majority of people commenting in the dbt slack redshift channel about time outs are using local machines to their clusters. Some generic observations:
|
Hello all, thanks a lot for your comments.
This failing macro is basically the same sql statement that we are running in our script. At this point, I am assuming dbt has something to do in relation to how the connection is being handled using |
@Brooke-white We're also seeing timeouts when running DBT in GitHub Actions. |
Driver version
2.0.918
Redshift version
PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.63282
Client Operating System
Docker container:
Debian GNU/Linux 11 (bullseye)
onpython3.9
docker imageCodebuild: Using
aws/codebuild/standard:7.0
Python version
python3.9
Table schema
Problem description
Codebuild machine connecting correctly to redshift.
Codebuild machine not connecting to redshift.
psycopg2
.dbt compile
ordbt run
we had no problems on connecting to redshift with our codebuild environment.redshift-connector
, and we have not been successful as we are getting time outs multiple times.Things we've tried:
redshift-connector
script to do a simple query (to discard that the issue is caused by dbt itself) -> Same result: connection time outpsycopg2
script to do a simple query (to discard that the issue is caused by dbt itself) -> THIS WORKS!psql -h <host> -p 5439 -U <user> -d <db>
-> Same result: connection time outIt is also important to note that we have no problem when running any of the above approaches in our local machines.
Python Driver trace logs
Reproduction code
This does not work, giving the following error:
Whereas the following snippet works:
The text was updated successfully, but these errors were encountered: