Skip to content

Releases: GoogleCloudPlatform/spanner-migration-tool

v3.4.0

01 Aug 09:25
4912731
Compare
Choose a tag to compare

VERSION(3.4.0)

New Features/Improvements

  • Support for FLOAT32 type in SMT.
    Support for migration of spanner supported Foreign key actions from source database(MySQL and PostgreSQL) to spanner.
  • Support for custom transformation in forward migration and reverse replication pipelines(documentation).

Bug Fixes

  • Fix for making GenerateIdSuffix thread safe
  • Updated the Makefile to include npm install
  • Updates for translating numeric primary keys to strings in the case of the PostgreSQL dialect in Spanner, as it doesn’t support numeric primary keys.

Documentation Updates

  • Documentation for custom transformation.

Dependency Updates

  • Upgraded pg_query to v5

General Changes

Please refer to the Spanner migration tool README for further details.

v3.3.0

26 Jun 06:09
c0f3ad8
Compare
Choose a tag to compare

VERSION(3.3.0)

New Features/Improvements

  • Support for automatic migration of Sequences and UUID (documentation) in SMT has been added
  • Support to configure metadata gcs bucket manually has been added

Bug Fixes

  • Integration test fix for issue with loading sqlplus
  • For orchestration and schema parallel tasks, queued tasks are now always completed even if one of the parallel task fails
  • Fix for when changing order of index columns on Spanner side, it also changes the order of columns on the MySQL side
  • Added validators for Index and Database names in UI
  • Fix for error during column name modification

Documentation Updates

  • Documentation for Autogen Keys
  • Documentation for GCS Bucket fine tuning

Dependency Updates

No dependency updates

General Changes

Please refer to the Spanner migration tool README for further details.

v3.2.5

21 May 05:32
12c2340
Compare
Choose a tag to compare

VERSION(3.2.5)

New Features/Improvements

  • JSON based resource generation. Datastream connection profiles can now be created in batch by specifying the JSON configuration via both the CLI and the UI.
  • Introduced dataflow-template flag. Gives the ability to override the version of the DatastreamToSpanner Dataflow template run by SMT for the migration.
  • Change endpoint of the DatastreamToSpanner Dataflow template to point to the latest released tag.
  • Updated version of the writer job for reverse replication in the reverse replication launcher script.

Bug Fixes

  • Fixed ID to name mapping for CSV table migration.

Documentation Updates

  • Added documentation for –dataflow-template flag added for schema, data and web commands. - TBD
  • Add documentation on Dataflow firewall configuration in reverse replication user guide.

Dependency Updates

No dependency updates

General Changes

Please refer to the Spanner migration tool README for further details.

v3.2.4

08 Apr 10:58
7681dce
Compare
Choose a tag to compare

VERSION(3.2.4)

New Features/Improvements

  • Introduced migration project id. Gives the ability to have spanner in a separate project than Dataflow. For both Live migration (documentation link) and reverse replication (documentation link)

Bug Fixes

  • No major bug fixes

Documentation Updates

  • Added documentation for –project flag added for schema, data and schema-and-data commands. - Link

Dependency Updates

  • No dependency updates

Please refer to the Spanner migration tool Documentation for further details.

v3.2.3

28 Mar 06:33
796e8d8
Compare
Choose a tag to compare

VERSION(3.2.3)

New Features/Improvements

  • Create source and target profiles based on json configuration passed in the command line.
  • The code was refactored, and accessors were added to enhance the overall code quality.
  • Improved the unit test coverage.
  • Update DatastreamToSpanner template to 2024-03-22-02_RC00 version.

Documentation Updates

  • Added documentation for migration_shard_id - Link
  • Updated documentation for Reverse Replication - Link

Dependency Updates

  • Angular update to v17.2.0

Please refer to the Spanner migration tool Documentation for further details.

v3.2.2

11 Jan 05:21
e323330
Compare
Choose a tag to compare

VERSION(3.2.2)

New Features/Improvements

  • Added basic rate limiting for DataStream and DataFlow calls to avoid quota issues during large sharded migrations
  • Added idempotent retries for data stream creation calls

Documentation Updates

  • Added documentation for Reverse Replication - Link

Please refer to the Spanner migration tool Documentation for further details.

v3.2.1

04 Jan 11:39
3fd736f
Compare
Choose a tag to compare

VERSION(3.2.1)

New Features/Improvements

  • Support cleanup command via the CLI to delete generated resources, this command is currently not available on gCloud.
  • Add IAM based authentication for CloudSQL MySQL and PostgreSQL databases
  • UI improvements to display shard level generated resources during minimal downtime migrations in schema-and-data, data modes
  • MySQL BIT(1) data type now maps to BOOLEAN instead of BYTES
  • Enhance error reporting for data-only migration

Bug Fixes

  • Fixed issues while validating existing spanner database schema in data-only mode for PostgreSQL dialect of Cloud Spanner
  • Validate only tables, columns and primary key for data-only migration

Documentation Updates

  • Added documentation for Monitoring Dashboards - Link
  • Added documentation for running SMT on Cloud shell - Link

Please refer to the Spanner migration tool Documentation for further details.

v3.2.0

06 Dec 13:25
833d141
Compare
Choose a tag to compare

VERSION(3.2.0)

New Features/Improvements

  • Add monitoring dashboard support for sharded and non-sharded migration
  • Added heartbeat to the UI to check backend service health
  • Added support for Datastream and GCS Tuning
  • Angular Version updated to v16
  • Added support for Postgresql to include objects config for datastream job
    Please refer to the updated documentation for more details.

Bug Fixes

  • During sharded migration configuration, setting shard source details after dataflow config overrides the set dataflow config
  • Fix for using correct ProjectId when shared VPC is used

Documentation Updates

  • Documentation updates on how to rerun for regular flow and for reprocessing DLQ directory flow
  • Permissions Documentation for using Monitoring Dashboards
  • Documentation for datastream and gcs tuning

Dependency Updates

cloud.google.com/go/monitoring added v1.16.1

Please refer to the Spanner migration tool Documentation for further details.

v3.1.3

18 Oct 11:07
386defc
Compare
Choose a tag to compare

VERSION(3.1.3)

New Features/Improvements

Spanner Migration Tool will switch to using pub/sub notifications from GCS buckets (to discover new objects) from the earlier poll mode for enhanced scalability.
Creation of pub/sub topics will be orchestrated by Spanner migration tool Additional permissions required for this have been documented
Please refer to the updated documentation for more details.

Bug Fixes

N/A

Documentation Updates

Documentation updates regarding the new pub/sub based dataflow jobs.

Dependency Updates

TiDB (v6.5.5)

General Changes

Please refer to the Spanner migration tool Documentation for further details.

v3.1.2

12 Oct 11:36
4d321d6
Compare
Choose a tag to compare

VERSION(3.1.2)

New Features/Improvements

Added Support for:

  1. DataFlow Configuration for Workers and Service Accounts
  2. Downloading json after migration starts for sharded migrations
  3. Displaying DataFlow gcloud cli equivalent for forward migrations
  4. Redesign of Assessment report.
  5. GCS Bucket created by SMT will be created in the same region as Spanner instance for improved data residency.

Bug Fixes

  1. Generated columns cannot be used for sharding id (#651)
  2. Process SQLServer CDC tables and UI bug fixes (#640)
  3. Dump file migration issue for PostgreSQL dialect (#645)
  4. Spanner instance length (#624)

Documentation Updates

  1. Revamp of SMT documentation
  2. Additions to troubleshooting section, reverse replication documentation.

Dependency Updates

  1. Postcss (8.4.31)
  2. Angular (16.2.5)
  3. Golang(v0.17.0)
  4. Spanner Emulator (v1.5.11)

General Changes

Please refer to the Spanner migration tool Documentation for further details.