Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added support for PostgreSQL for spanner reverse replication template gcs-to-sourcedb #1378

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Added support for PostgreSQL for spanner reverse replication template gcs-to-sourcedb #1378

wants to merge 9 commits into from

Conversation

tanbirsohail
Copy link
Member

Added support for PostgreSQL dialect for spanner reverse replication template gcs-to-sourcedb.

Refer to DMLGenerator module PostgreSQLDMLGenerator for support for native PostgreSQL type.

@tanbirsohail tanbirsohail requested a review from a team as a code owner March 19, 2024 15:34
@tanbirsohail tanbirsohail requested review from Deep1998 and asthamohta and removed request for a team March 19, 2024 15:34
@tanbirsohail
Copy link
Member Author

@aksharauke Changes from PR #1381 , #1396 and #1455 merged.

@tanbirsohail tanbirsohail force-pushed the spanner_reverse_replication_gcs_templates_postgres_support branch from a41e4a1 to 1aa7c99 Compare May 10, 2024 09:40
@tanbirsohail
Copy link
Member Author

Rebased branch to b910d17 to resolve merge conflicts.

@codecov Codecov
Copy link

codecov bot commented May 10, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 260 lines in your changes missing coverage. Please review.

Project coverage is 44.03%. Comparing base (0837b98) to head (bd849a9).
Report is 530 commits behind head on main.

Files with missing lines Patch % Lines
...mplates/processing/dml/PostgreSQLDMLGenerator.java 0.00% 69 Missing ⚠️
...v2/templates/processing/dml/MySQLDMLGenerator.java 0.00% 58 Missing ⚠️
...cloud/teleport/v2/templates/dao/PostgreSQLDao.java 0.00% 35 Missing ⚠️
...ogle/cloud/teleport/v2/templates/dao/MySQLDao.java 0.00% 29 Missing ⚠️
...le/cloud/teleport/v2/templates/dao/DaoFactory.java 0.00% 22 Missing ⚠️
...oogle/cloud/teleport/v2/templates/dao/BaseDao.java 0.00% 18 Missing ⚠️
.../templates/processing/dml/DMLGeneratorFactory.java 0.00% 8 Missing ⚠️
...eleport/v2/templates/common/ProcessingContext.java 0.00% 7 Missing ⚠️
...rocessing/handler/GCSToSourceStreamingHandler.java 0.00% 6 Missing ⚠️
...lates/processing/handler/InputRecordProcessor.java 0.00% 3 Missing ⚠️
... and 3 more
Additional details and impacted files 
@@             Coverage Diff              @@
##               main    #1378      +/-   ##
============================================
+ Coverage     41.49%   44.03%   +2.54%     
+ Complexity     2920      674    -2246     
============================================
  Files           754      304     -450     
  Lines         43977    16264   -27713     
  Branches       4707     1625    -3082     
============================================
- Hits          18248     7162   -11086     
+ Misses        24190     8580   -15610     
+ Partials       1539      522    -1017
Components Coverage Δ
spanner-templates 55.07% <0.00%> (-6.20%) ⬇️
spanner-import-export ∅ <ø> (∅)
spanner-live-forward-migration 73.21% <ø> (-0.36%) ⬇️
spanner-live-reverse-replication 43.85% <0.00%> (-5.30%) ⬇️
spanner-bulk-migration 81.58% <ø> (-0.34%) ⬇️
Files with missing lines Coverage Δ
...gle/cloud/teleport/v2/templates/GCSToSourceDb.java 0.00% <0.00%> (ø)
...port/v2/templates/processing/dml/DMLGenerator.java 0.00% <0.00%> (-76.32%) ⬇️
...lates/processing/handler/InputRecordProcessor.java 0.00% <0.00%> (ø)
...v2/templates/utils/ProcessingContextGenerator.java 0.00% <0.00%> (ø)
...rocessing/handler/GCSToSourceStreamingHandler.java 0.00% <0.00%> (ø)
...eleport/v2/templates/common/ProcessingContext.java 0.00% <0.00%> (ø)
.../templates/processing/dml/DMLGeneratorFactory.java 0.00% <0.00%> (ø)
...oogle/cloud/teleport/v2/templates/dao/BaseDao.java 0.00% <0.00%> (ø)
...le/cloud/teleport/v2/templates/dao/DaoFactory.java 0.00% <0.00%> (ø)
...ogle/cloud/teleport/v2/templates/dao/MySQLDao.java 0.00% <0.00%> (ø)
... and 3 more

... and 477 files with indirect coverage changes

aksharauke
aksharauke reviewed May 16, 2024
View reviewed changes
v2/gcs-to-sourcedb/src/main/java/com/google/cloud/teleport/v2/templates/GCSToSourceDb.java Outdated Show resolved Hide resolved
v2/gcs-to-sourcedb/src/main/java/com/google/cloud/teleport/v2/templates/dao/BaseDao.java Outdated Show resolved Hide resolved
v2/gcs-to-sourcedb/src/main/java/com/google/cloud/teleport/v2/templates/dao/MySQLDao.java Outdated Show resolved Hide resolved
v2/gcs-to-sourcedb/src/main/java/com/google/cloud/teleport/v2/templates/dao/PostgreSQLDao.java Outdated
}

// frees up the pooling resources
public void cleanup() throws Exception {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to base class

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored and moved to base class.

.../main/java/com/google/cloud/teleport/v2/templates/processing/dml/PostgreSQLDMLGenerator.java Show resolved Hide resolved
v2/gcs-to-sourcedb/src/main/java/com/google/cloud/teleport/v2/templates/dao/BaseDao.java Outdated Show resolved Hide resolved
@tanbirsohail tanbirsohail force-pushed the spanner_reverse_replication_gcs_templates_postgres_support branch from 7b931d1 to 54b9cc8 Compare May 17, 2024 06:24
@tanbirsohail
Copy link
Member Author

Rebased and updated to incorporate changes in #1549.

@tanbirsohail tanbirsohail mentioned this pull request Jun 3, 2024
Closed
@tanbirsohail tanbirsohail force-pushed the spanner_reverse_replication_gcs_templates_postgres_support branch from 54b9cc8 to c233e73 Compare June 7, 2024 07:49
@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 5, 2024

This pull request has been marked as stale due to 180 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time. Thank you for your contributions.

@github-actions github-actions bot added stale and removed stale labels Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aksharauke aksharauke aksharauke left review comments

@Deep1998 Deep1998 Awaiting requested review from Deep1998 Deep1998 is a code owner automatically assigned from GoogleCloudPlatform/spanner-migrations-team

@asthamohta asthamohta Awaiting requested review from asthamohta asthamohta is a code owner automatically assigned from GoogleCloudPlatform/spanner-migrations-team

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
size/XXL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tanbirsohail @aksharauke