Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests to check for report scores #426

Open
npatki opened this issue Aug 24, 2023 · 0 comments
Open

Integration tests to check for report scores #426

npatki opened this issue Aug 24, 2023 · 0 comments
Labels
internal The issue doesn't change the API or functionality

Comments

@npatki
Copy link
Contributor

npatki commented Aug 24, 2023

Descriptions

When we change around abstractions and reports, it may be the case that we accidentally change how a metric works without intending to.

To pre-empt this, should we add end-to-end integration tests for the quality and diagnostic reports? These integration tests can verify that the score is always the same for some given dataset (eg. the demo dataset).

TBD: Is there any intended randomness in the logic that could affect the score? The test may have to account for it.

@npatki npatki added the internal The issue doesn't change the API or functionality label Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal The issue doesn't change the API or functionality
Projects
None yet
Development

No branches or pull requests

1 participant