Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added file #18

Merged
merged 2 commits into from
Jul 19, 2024
Merged

added file #18

merged 2 commits into from
Jul 19, 2024

Conversation

mrm1001
Copy link

@mrm1001 mrm1001 commented Jul 19, 2024

No description provided.

@mrm1001 mrm1001 requested a review from davidsbatista July 19, 2024 12:59
@davidsbatista
Copy link
Collaborator

can we have instead a list of JSON objects per query/question, with the following fields:

{
'question: "....",
'relevant_document: "..."
}

@mrm1001
Copy link
Author

mrm1001 commented Jul 19, 2024

Yes, I could do that, but the reason I didn't is because I thought it would be better to keep it consistent with the first version of the labels: https://github.com/deepset-ai/haystack-evaluation/blob/add-aragog-dataset-with-filepaths/datasets/ARAGOG/eval_questions.json. Do you still prefer to have it like you suggest? I can change it no problem.

@davidsbatista
Copy link
Collaborator

I see, makes sense. Can you rename eval_questions_with_files.json to eval_questions_relevant_doc.json

@davidsbatista davidsbatista merged commit 6db15f8 into main Jul 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants