Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add CV metrics to evalem #28

Open
NISH1001 opened this issue Jun 20, 2023 · 1 comment
Open

add CV metrics to evalem #28

NISH1001 opened this issue Jun 20, 2023 · 1 comment

Comments

@NISH1001
Copy link
Collaborator

NISH1001 commented Jun 20, 2023

cc: @rbavery @weiji14

I looked at torchmetrics. And it seems since the metric there -- just like evalem.metrics -- also uses (predictions, references) type of argument, I think we can have new wrappers around torchmetrics at evalem.cv.metrics

image

For instance, the base metrics in the evalem has evalem._base.metrics.compute(...) method to be overridden by downstream implementation:
https://github.com/NASA-IMPACT/evalem/blob/main/evalem/_base/metrics.py#L38

One evalem.nlp.metrics.ExactMatchMetric has the implementation for example:
https://github.com/NASA-IMPACT/evalem/blob/main/evalem/nlp/metrics/basics.py#L16

@NISH1001
Copy link
Collaborator Author

Related thread:
#19

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants