DateTime(-1420095600) should be between -1420070400 and 9904550399 inclusive of both values

[congvmit]

Hello team, I am synchronizing data from MySQL v5.6 to Clickhouse v23.4.5.22
Can you please help me debug this?

debezium-embedded    | 109221 2023-07-05 08:49:39.273 [pool-791-thread-1] ERROR com.altinity.clickhouse.sink.connector.db.DbWriter  - ******* ERROR inserting Batch *****************
debezium-embedded    | java.lang.IllegalArgumentException: DateTime(-1420095600) should be between -1420070400 and 9904550399 inclusive of both values
debezium-embedded    |  at com.clickhouse.data.ClickHouseChecker.newException(ClickHouseChecker.java:17)
debezium-embedded    |  at com.clickhouse.data.ClickHouseChecker.between(ClickHouseChecker.java:109)
debezium-embedded    |  at com.clickhouse.data.format.BinaryDataProcessor$DateTime64SerDe.serialize(BinaryDataProcessor.java:292)
debezium-embedded    |  at com.clickhouse.data.format.BinaryDataProcessor$NullableSerializer.serialize(BinaryDataProcessor.java:136)
debezium-embedded    |  at com.clickhouse.data.ClickHouseDataProcessor.write(ClickHouseDataProcessor.java:534)
debezium-embedded    |  at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.addBatch(InputBasedPreparedStatement.java:345)
debezium-embedded    |  at com.altinity.clickhouse.sink.connector.db.DbWriter.addToPreparedStatementBatch(DbWriter.java:476)
debezium-embedded    |  at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.flushRecordsToClickHouse(ClickHouseBatchRunnable.java:198)
debezium-embedded    |  at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.processRecordsByTopic(ClickHouseBatchRunnable.java:169)
debezium-embedded    |  at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.run(ClickHouseBatchRunnable.java:101)
debezium-embedded    |  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
debezium-embedded    |  at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
debezium-embedded    |  at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
debezium-embedded    |  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
debezium-embedded    |  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
debezium-embedded    |  at java.base/java.lang.Thread.run(Thread.java:829)

[subkanthi]

Hi @congvmit , there is an open bug in the clickhouse jdbc library that addresses this
ClickHouse/clickhouse-java#1386

Will update the library once this is fixed.

[congvmit]

Thank you @subkanthi

[subkanthi]

Hi @congvmit , this was already fixed as we map to DateTime64 . Can you please post the image, tag, are you trying the sink connector lightweight or sink connector?

This is the range we use and thats well within the CH server range.
1900-01-01 00:00:00, 2299-12-31 23:59:59.99999999

      DATETIME64_MAX = LocalDateTime.of(LocalDate.of(2283, 11, 11), LocalTime.MAX).toEpochSecond(ZoneOffset.UTC);
        DATETIME64_MIN = LocalDateTime.of(LocalDate.of(1925, 1, 1), LocalTime.MIN).toEpochSecond(ZoneOffset.UTC);

[subkanthi]

It looks like the issues was the wrong tag similar to #277