Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Column Function Coverage with Additional Test Cases #68

Merged
merged 18 commits into from
Aug 9, 2024

Conversation

lexara-prime-ai
Copy link
Contributor

Description

This pull request introduces additional test cases for the column functions.
The new tests specifically evaluate the behavior of the following column functions:

  • asc_nulls_first
  • asc_nulls_last
  • desc_nulls_first
  • desc_nulls_last
  • like
  • ilike
  • rlike
  • eq
  • or
  • isnotnull
  • isnan
  • alias
  • cast
  • substr

, ensuring that they correctly identify and handle these cases.

Additionally, the isnotnull function definition has been moved to the appropriate section within the code, under generate_functions! for functions that require a single column argument, as well as a typo on the subtract function. This has been updated from substract to subtract.

Fix

  • Moved isnotnull function definition to:
// functions that require a single col argument
generate_functions!(
 one_col: isnan,
 isnull,
 isnotnull,
  ...);

Related Issue(s)

Documentation

Source reference links provided by @sjrusso8

    * feat(tests): implement `test_func_asc_nulls_first`
      - Verify sorting functionality for ascending order with NULL values positioned first.
    * feat(tests): implement `test_func_asc_nulls_last`
      - Verify sorting functionality for ascending order with NULL values positioned last.
…#49)

    * feat(tests): implement `test_func_desc_nulls_first`
      - Verify sorting functionality for descending order with NULL values positioned first.

    * feat(tests): implement `test_func_desc_nulls_last`
      - Verify sorting functionality for descending order with NULL values positioned last.

    * validate robustness and accuracy of the sorting mechanism
…#17)

    * feat(tests): implement `test_func_desc_nulls_first`
      - Verify sorting functionality for descending order with NULL values positioned first.

    * feat(tests): implement `test_func_desc_nulls_last`
      - Verify sorting functionality for descending order with NULL values positioned last.

    * validate robustness and accuracy of the sorting mechanism
    * feat(tests): implement `test_func_like`
      - Verify filter functionality using the LIKE operator.
      - Ensure the filter correctly selects the name "Alice".

    * validate robustness and accuracy of the LIKE filter mechanism
    * feat(tests): implement `test_func_ilike`
      - Verify filter functionality using the ILIKE operator.
      - Ensure the filter correctly selects the name "Alice" using a case-insensitive pattern "%Ice".

    * validate robustness and accuracy of the ILIKE filter mechanism
   * feat(tests): implement
     - Implemented a test to validate the behavior of the 'rlike' filter on a DataFrame.
     - Ensured that the filter correctly identifies records with names matching the regex pattern ice.
       - Added necessary setup and assertions for expected results.
   * feat(tests): implement `test_func_rlike`
     - Implemented a test to validate the behavior of the 'rlike' filter on a DataFrame.
     - Ensured that the filter correctly identifies records with names matching the regex pattern ice.
     - Added necessary setup and assertions for expected results.
    * feat(tests): implement `test_func_eq`
      - Verify filter functionality using the EQ operator.
    * feat(tests): implement `test_func_or`
      - Verify filter functionality using the OR operator.
      - Ensure the filter correctly selects the name "Alice" using a case-insensitive pattern "%Ice".

    * validate robustness and accuracy of the ILIKE filter mechanism
 main
   * feat(tests): implement `test_func_is_null`
     - Implemented a test to validate the behavior of the 'is_null' filter on a DataFrame.
     - Ensured that the filter correctly identifies records with null values.
     - Added necessary setup and assertions for expected results.
   * feat(tests): implement `test_func_is_not_null`
     - Implemented a test to validate the behavior of the 'isnotnull' filter on a DataFrame.
     - Ensured that the filter correctly identifies records with non-null values.
     - Added necessary setup and assertions for expected results.
     - Moved `isnotnull` function definition to `
        // functions that require a single col argument
        generate_functions!(
            one_col: isnan,
            isnull,
            isnotnull,
            ...)
     `
…nality(#17)

* feat(tests): implement `test_func_over`
  - Implemented a test to validate the behavior of window functions on a DataFrame.
  - Ensured that the window functions (`rank` and `min`) correctly compute values within the specified window.
  - Added necessary setup, sorting, and assertions for expected results.

* feat(tests): implement `test_func_isnan`
  - Implemented a test to validate the behavior of the `isnan` function on a DataFrame.
  - Ensured that the `isnan` function correctly identifies NaN values.
  - Added necessary setup and assertions for expected results.
…nctions (#17)

- feat(tests): implement `test_func_cast`
  - Added a test to validate the behavior of casting an integer column to a string.
  - Ensured that the cast operation correctly transforms the data type and assigns the alias.

- feat(tests): implement `test_func_alias`
  - Added a test to verify column aliasing functionality.
  - Confirmed that the DataFrame correctly applies aliases to columns.

- feat(tests): implement `test_func_substr`
  - Added a test to validate the substring function on a string column.
  - Ensured that the substring operation works as expected and assigns the alias.
@sjrusso8
Copy link
Owner

sjrusso8 commented Aug 8, 2024

@lexara-prime-ai there was a change in the CI runner for the docker compose step. If you update your fork to the most recent change, the integration tests should work

@lexara-prime-ai
Copy link
Contributor Author

@sjrusso8 I just updated the PR with the fix you made for the CI runner.

@sjrusso8
Copy link
Owner

sjrusso8 commented Aug 9, 2024

@lexara-prime-ai LGTM! Thanks so much for adding some much needed code coverage

@sjrusso8 sjrusso8 merged commit 7ea744b into sjrusso8:main Aug 9, 2024
3 checks passed
@lexara-prime-ai
Copy link
Contributor Author

Glad to help @sjrusso8 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Write unit test(s) for Spark functions
2 participants