Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Confident Predictions Selection #269

Closed
alinapark opened this issue Sep 25, 2023 · 5 comments
Closed

Confident Predictions Selection #269

alinapark opened this issue Sep 25, 2023 · 5 comments

Comments

@alinapark
Copy link

alinapark commented Sep 25, 2023

💸 Confident Predictions Selection is a bounty challenge 💸

Your task

To design a model capable of selecting the top 10% of predictions that will exhibit the smallest mean distance.
You have an option to work with either or both of the text data and raw predictions. Additionally, you may perform any form of aggregation or transformation on the raw predictions as you see fit.

Validation Metrics

The primary metric for this challenge is the mean distance of the selected top 10% predictions. Your objective is to minimize this value.
As a secondary metric, we designed the Class Representation Index (CRI). In essence, CRI compares the class distribution before and after filtering, giving a higher weight to classes that were initially larger. The primary purpose of this metric is to detect cases where a class is significantly less represented after filtering compared to its original size.

Please, see the dedicated repository for instructions

@Annyvichi
Copy link
Contributor

Annyvichi commented Jul 30, 2024

Do you consider having a better solution for this issue and do i have some time to work on it?

@ingakaspar ingakaspar removed their assignment Jul 31, 2024
@Annyvichi
Copy link
Contributor

Do you consider having a better solution for this issue and do i have some time to work on it?

I'm afraid you can ruin my work by closing the issue before i get a good score, would you please keep it open?

@Annyvichi
Copy link
Contributor

@evgenydmitriev I need to ask you about this issue. Did anyone exceed the winner's score significantly or is it impossible?

I liked this task very much and I'd like to work on other tasks like this.
Very curious about this position: Junior NLP Developer - Data Scraping, can I take it?

@evgenydmitriev
Copy link
Contributor

@Annyvichi, I closed the issue because I saw no activity from the challenge organizers for almost a year, but if you email us at [email protected], I can forward your resume/profile to a few NLP teams.

@Annyvichi
Copy link
Contributor

@evgenydmitriev, I will take some time to improve my profile and then will apply at the email and ask about open positions in your team.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants