You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi! This is based on the original code, but we can add another metric to show alongside by adding a flexible-extract filter, as gsm8k does it. PR welcome!
Hi thanks for the library! It seems that the way how math answer is extracted, i.e.
lm-evaluation-harness/lm_eval/tasks/hendrycks_math/utils.py
Lines 20 to 24 in bcb4cbf
For example, the following answer:
... some reasoning logic ... Thus the answer is \[ \boxed{42} \]
is not extracted, because it is not a$
.The text was updated successfully, but these errors were encountered: