Inclusion of tsp flag leads to incorrect DVT output results

[karcot1]

Overview: the -tsp flag was implemented as an optional flag due to a bug in teradatasql that adds extra whitespaces to CHAR data types, resulting in incorrect DVT results.

Example: ‘ABC ’ CHAR(10) in TD and “ABC‘ in BQ will become identical when DVT executes with -tsp

We understand this raises a risk of duplicate primary keys in the instance where a table contains two different CHAR or VARCHAR PKs, which only differ by trailing spaces.

Observation: -tsp flag applied to CHAR/VARCHAR data results in incorrect DVT output results, even after accounting for the expected duplicate rows. We see identical hashes flagged as "fail" and different hashes flagged as "success"