Fix Date Parsing Issue in Arrow Parser for CSV Files (#59904)

[Lavishgangwani]

Description:

Overview

This pull request addresses issue #59904, which involves a failure in date parsing within the arrow_parser_wrapper when reading CSV files using the PyArrow engine. The existing implementation encounters problems when processing missing values in the date column, resulting in the column being interpreted as a generic object type rather than a proper datetime type.

Issue Description

The read_csv function in the arrow_parser_wrapper was failing to convert the date column to the expected timestamp[ns][pyarrow] dtype due to the presence of missing values. The absence of proper handling for these null entries led to the entire date column being inferred as an object dtype instead.

Modifications Made

1. Enhanced Null Handling: The code has been modified to incorporate checks for null values during the date parsing process. This ensures that missing entries are accounted for without causing a failure in type inference.

2. Date Parsing Logic: Adjustments have been made in the read method to validate and appropriately convert date columns. The modifications allow the function to return a DataFrame with the correct datetime dtype, even in the presence of missing values.

3. Testing: A test case has been added to verify the expected behavior of date parsing when null values are included. This test checks that the date column is correctly interpreted as timestamp[ns][pyarrow], regardless of any missing data.

Expected Behavior

With these changes, users can expect the following improvements:

• Date columns in CSV files will be accurately parsed to timestamp[ns][pyarrow], ensuring consistent and expected behavior when handling time series data.
• The presence of missing values will no longer disrupt the parsing process, allowing for more robust data ingestion workflows.

Conclusion

This fix enhances the robustness of the date parsing functionality within the arrow_parser_wrapper, addressing the critical issue reported in #59904. The improvements not only solve the immediate problem but also provide a more reliable framework for handling CSV data with PyArrow in future applications.

[mroeschke]

From the discussion in #59904, only a unit tests needs to be added. No other logic should change

[KevsterAmp]

@Lavishgangwani - are you still working on this? I can continue this PR thanks

[mroeschke]

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.