Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DateTime/Date limits not compatible with CH server limits #1103

Open
subkanthi opened this issue Oct 7, 2022 · 4 comments · May be fixed by #1386
Open

DateTime/Date limits not compatible with CH server limits #1103

subkanthi opened this issue Oct 7, 2022 · 4 comments · May be fixed by #1386
Labels
enhancement

Comments

@subkanthi
Copy link
Contributor

subkanthi commented Oct 7, 2022 •
edited
Loading 

        DATETIME64_MAX = LocalDateTime.of(LocalDate.of(2283, 11, 11), LocalTime.MAX).toEpochSecond(ZoneOffset.UTC);

In Java 11, the nanosecond part is lost because of converting to LocalDateTime, maybe it should be kept as Instant.

Also the limits for DATETIME64 and DATE does not align with the latest CH server limits, is there a strategy to support limits that change across server versions?

        DATETIME64_MAX = LocalDateTime.of(LocalDate.of(2283, 11, 11), LocalTime.MAX).toEpochSecond(ZoneOffset.UTC);
        DATETIME64_MIN = LocalDateTime.of(LocalDate.of(1925, 1, 1), LocalTime.MIN).toEpochSecond(ZoneOffset.UTC);

    public static final int DATE32_MAX = (int)LocalDate.of(2283, 11, 11).toEpochDay();
    public static final int DATE32_MIN = (int)LocalDate.of(1925, 1, 1).toEpochDay();

Supported range of values: [1900-01-01 00:00:00, 2299-12-31 23:59:59.99999999]
https://clickhouse.com/docs/en/sql-reference/data-types/datetime64/
@subkanthi subkanthi changed the title DateTime limits not compatible with CH server limits DateTime/Date limits not compatible with CH server limits Oct 7, 2022
@zhicwu zhicwu added this to the 0.3.3 release milestone Oct 7, 2022
@zhicwu
Copy link
Contributor

zhicwu commented Oct 7, 2022

Thanks @subkanthi. And the upper limit is different when scale is 9 🤦

As to support limits across server versions, I'm not sure if it's worthy of doing that, but let me investigate once I completed refactoring.

@subkanthi
Copy link
Contributor Author

I can take a shot at fixing the MAX value if it's OK, recently have been looking at a lot of java datetime classes

@zhicwu
Copy link
Contributor

zhicwu commented Oct 7, 2022

Thanks for helping, that'd be great! I'm refactoring *Value classes and data processors to better support unsigned types along with performance improvement of handling arrays and tuples. There's no conflict if the changes are mainly about Date & DateTime.

@zhicwu zhicwu mentioned this issue Oct 18, 2022
Merged
@zhicwu zhicwu modified the milestones: 0.4.0 release, 0.4.1 release Jan 18, 2023
@zhicwu zhicwu modified the milestones: 0.4.2 release, 0.5 release Mar 21, 2023
@zhicwu zhicwu linked a pull request Jun 26, 2023 that will close this issue
WIP - Lower the datetime limit to what CH also accepts - 1900-01-01 #1386
Open
3 tasks
@mshustov mshustov mentioned this issue Sep 6, 2023
Closed
1 task
@mshustov mshustov modified the milestones: 0.6 release, v0.5.1 Dec 4, 2023
@TimonSP
Copy link
Contributor

TimonSP commented Jan 9, 2024

There is no any issue with scale (8/9) for Date32 but DATE32_MIN and DATE32_MAX are out of supported by ClickHouse range [1900-01-01, 2299-12-31].
BTW attached pull request is for DateTime64 only and won't close Date32 issue.

TimonSP added a commit to TimonSP/clickhouse-java that referenced this issue Jan 9, 2024
@TimonSP
Update Date32 and DateTime64 range for ClickHouse, remove datetime64 …
f0ef314 
…limits check layer ClickHouse#1103 ClickHouse#1386 ClickHouse#1441
@TimonSP TimonSP mentioned this issue Jan 9, 2024
Merged
@mshustov mshustov modified the milestones: v0.6.1, v0.6.0 Jan 10, 2024
mzitnik pushed a commit that referenced this issue Jan 11, 2024
@TimonSP
Update Date32 and DateTime64 range for ClickHouse, remove datetime64 …
52634fb 
…limits check layer (#1518)

* Update Date32 and DateTime64 range for ClickHouse, remove datetime64 limits check layer #1103 #1386 #1441

* fix of byte arrays for updated dates in BinaryStreamUtilsTest
@mzitnik mzitnik modified the milestones: v0.6.0, v0.6.1 Jan 16, 2024
@chernser chernser modified the milestones: 0.6.4 Release, 0.7.0 Aug 16, 2024
@chernser chernser modified the milestones: 0.7.0, 0.7.1 Oct 1, 2024
@chernser chernser added this to the Priority Backlog milestone Oct 3, 2024
@chernser chernser modified the milestones: Priority Backlog, 0.7.2 Oct 31, 2024
@chernser chernser removed this from the 0.7.2 milestone Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
6 participants
@mzitnik @chernser @mshustov @zhicwu @subkanthi @TimonSP