Stats card: Rating keeps dropping?

[ErencanPelin]

Hi everyone, I just had a question regarding the Stats Card:
It appears the more I commit, pull request, add errors, etc, The lower my stat becomes. At the beginning of this year, I was at an A++ before dropping straight to a B last month, and now I've dropped to C++.

I read through the documentation but I couldn't figure out why in my specific instance this is happening, and from what I could tell it hasn't happened to anyone else? Not sure if its an error, or a bug, or something I'm doing from my end - I just want to get back to an A++ :D

Any guidance is always appreciated! Thanks!

[AKABharat]

@ErencanPelin, ya i am also facing the same issue , my rating dropped to C+ in one day. Github Profile

[rickstaa]

@AKABharat thanks for your report. We improved the ranking (see #2815 (comment)). Previously, the ranking mechanism was broken and nearly everybody got a+ and A++ and obtaining a better or lower rank was near impossible. Due to @francois-rozet's efforts, the ranking mechanism is now more balanced and beginner-friendly. You should be able to rise in ranks rapidly in the beginning, but obtaining the last 2 ranks now is a bit harder 🚀.

[AKABharat]

@rickstaa thank you 👍 for clarifying the doubt.

[qwerty541]

This behavior was connected to recent changes to the ranking system. Three weeks ago #1186 and yesterday #2762 was merged into master and deployed. We use #1935 to decide which features or PRs we implement. There was a community request to improve the ranking system so these changes have been made.

You can find detailed explanation how ranking system currently works in readme

Note
Available ranks are S (top 1%), A+ (12.5%), A (25%), A- (37.5%), B+ (50%), B (62.5%), B- (75%), C+ (87.5%) and C (everyone). This ranking scheme is based on the Japanese academic grading system. The global percentile is calculated as a weighted sum of percentiles for each statistic (number of commits, pull requests, issues, stars and followers), based on the cumulative distribution function of the exponential and the log-normal distributions. The implementation can be investigated at src/calculateRank.js. The circle around the rank shows 100 minus the global percentile.

[ErencanPelin]

Thanks for the super quick reply!

That makes sense, in that case the thing drawing my rank back is the fact I have a tonne of commits and not many PRs or issues (it looks like stars and followers have the lesser weighting) am I understanding this correctly? I apologise, my understanding of the codebase is quite minimal!

[francois-rozet]

Ok! @ErencanPelin, you can actually drop count_private, as it is not in the API anymore.

[ErencanPelin]

Oh right! Thanks for that!
Something still seems a little off though, forgive me if I'm just failing to understand the deeper workings inside the calculateRank.js script, how is it that without any of the other values changing, having more commits appears to lower your rank rather than increase it?
I understand there is some statistical math going on, but my expected output was for the rank to increase the more PRs, issues, and commits you make?

[francois-rozet]

Setting include_all_commits to true makes the target number of commits go up by a factor 4 (we expect people to have more total commits than yearly commits). Because your number of total commits is marginally higher than your yearly number of commits, this results in your rank for that stat going down.

[rickstaa]

@ErencanPelin you can check out the workings of the ranking mechanism using the code below. Eventually, it would be great to be able to see this through a web app, but we didn't find the time yet to implement it (see #1761). I hope that somebody creates it during the next Hacktoberfest 🙏🏻.

Script

/*
 * Small test script that can be used to see how the rank changes if stats are changed.
 */
function exponential_cdf(x) {
    return 1 - 2 ** -x;
}

function log_normal_cdf(x) {
    // approximation
    return x / (1 + x);
}

/**
 * Calculates the users rank.
 *
 * @param {object} params Parameters on which the user's rank depends.
 * @param {boolean} params.all_commits Whether `include_all_commits` was used.
 * @param {number} params.commits Number of commits.
 * @param {number} params.prs The number of pull requests.
 * @param {number} params.issues The number of issues.
 * @param {number} params.repos Total number of repos.
 * @param {number} params.stars The number of stars.
 * @param {number} params.followers The number of followers.
 * @returns {{level: string, percentile: number}}} The users rank.
 */
function calculateRank({
    all_commits,
    commits,
    prs,
    issues,
    repos, // unused
    stars,
    followers,
}) {
    const COMMITS_MEDIAN = all_commits ? 1000 : 250,
        COMMITS_WEIGHT = 2;
    const PRS_MEDIAN = 50,
        PRS_WEIGHT = 3;
    const ISSUES_MEDIAN = 25,
        ISSUES_WEIGHT = 1;
    const STARS_MEDIAN = 50,
        STARS_WEIGHT = 4;
    const FOLLOWERS_MEDIAN = 10,
        FOLLOWERS_WEIGHT = 1;

    const TOTAL_WEIGHT =
        COMMITS_WEIGHT +
        PRS_WEIGHT +
        ISSUES_WEIGHT +
        STARS_WEIGHT +
        FOLLOWERS_WEIGHT;

    const THRESHOLDS = [1, 12.5, 25, 37.5, 50, 62.5, 75, 87.5, 100];
    const LEVELS = ["S", "A+", "A", "A-", "B+", "B", "B-", "C+", "C"];

    const rank =
        1 -
        (COMMITS_WEIGHT * exponential_cdf(commits / COMMITS_MEDIAN) +
            PRS_WEIGHT * exponential_cdf(prs / PRS_MEDIAN) +
            ISSUES_WEIGHT * exponential_cdf(issues / ISSUES_MEDIAN) +
            STARS_WEIGHT * log_normal_cdf(stars / STARS_MEDIAN) +
            FOLLOWERS_WEIGHT * log_normal_cdf(followers / FOLLOWERS_MEDIAN)) /
        TOTAL_WEIGHT;

    const level = LEVELS[THRESHOLDS.findIndex((t) => rank * 100 <= t)];

    return { level: level, percentile: rank * 100 };
}

// Setup example data
const params = {
    all_commits: false,
    commits: 0,
    prs: 0,
    issues: 0,
    repos: 0,
    stars: 0,
    followers: 0,
}

const main = () => {
    // Increase the commits from 1-10000 and see how the rank changes.
    console.log("\nInclude_all_commits: false\n")
    for (let i = 1; i < 2000; i += 100) {
        params.commits = i;
        console.log(i, calculateRank(params));
    }

    // Now with `include_all_commits` set to true.
    console.log("\nInclude_all_commits: true\n")
    for (let i = 1; i < 10000; i += 500) {
        params.commits = i;
        params.all_commits = true;
        console.log(i, calculateRank(params));
    }
}

main();

Output

Include_all_commits: false

1 { level: 'C', percentile: 99.94965911549217 }
101 { level: 'C', percentile: 95.55927207560205 }
201 { level: 'C', percentile: 92.23198089072507 }
301 { level: 'C', percentile: 89.71036570546403 }
401 { level: 'C', percentile: 87.79933875013187 }
501 { level: 'C+', percentile: 86.3510511425094 }
601 { level: 'C+', percentile: 85.25345438253687 }
701 { level: 'C+', percentile: 84.42163158631763 }
801 { level: 'C+', percentile: 83.79122779000237 }
901 { level: 'C+', percentile: 83.31347105116933 }
1001 { level: 'C+', percentile: 82.95139914926372 }
1101 { level: 'C+', percentile: 82.67699995927057 }
1201 { level: 'C+', percentile: 82.46904426021577 }
1301 { level: 'C+', percentile: 82.31144331113696 }
1401 { level: 'C+', percentile: 82.1920041264287 }
1501 { level: 'C+', percentile: 82.1014861509523 }
1601 { level: 'C+', percentile: 82.032886353454 }
1701 { level: 'C+', percentile: 81.9808974286903 }
1801 { level: 'C+', percentile: 81.94149719142061 }
1901 { level: 'C+', percentile: 81.91163739524355 }

Include_all_commits: true

1 { level: 'C', percentile: 99.9874016907355 }
501 { level: 'C', percentile: 94.66576039893391 }
1001 { level: 'C', percentile: 90.90279175445866 }
1501 { level: 'C', percentile: 88.24197110855786 }
2001 { level: 'C+', percentile: 86.36048678632025 }
2501 { level: 'C+', percentile: 85.03007646336984 }
3001 { level: 'C+', percentile: 84.08933430225103 }
3501 { level: 'C+', percentile: 83.42412914077583 }
4001 { level: 'C+', percentile: 82.95375806021642 }
4501 { level: 'C+', percentile: 82.62115547947883 }
5001 { level: 'C+', percentile: 82.38596993919913 }
5501 { level: 'C+', percentile: 82.21966864883032 }
6001 { level: 'C+', percentile: 82.10207587869047 }
6501 { level: 'C+', percentile: 82.01892523350607 }
7001 { level: 'C+', percentile: 81.96012884843614 }
7501 { level: 'C+', percentile: 81.91855352584395 }
8001 { level: 'C+', percentile: 81.88915533330898 }
8501 { level: 'C+', percentile: 81.86836767201288 }
9001 { level: 'C+', percentile: 81.8536685757454 }
9501 { level: 'C+', percentile: 81.84327474509735 }

[ErencanPelin]

Ohh! right! That makes a lot more sense 😅 Thanks for clearing that up! A web app would be awesome especially for newbies like me to understand how the system works. I appreciate everyone's help and I appreciate the hard work you're all putting in! Keep it up!

[ishqDehlvi]

What does -B means is github stats??

[rickstaa]

We added an explanation in the documentation:

Note Available ranks are S (top 1%), A+ (12.5%), A (25%), A- (37.5%), B+ (50%), B (62.5%), B- (75%), C+ (87.5%) and C (everyone). This ranking scheme is based on the Japanese academic grading system. The global percentile is calculated as a weighted sum of percentiles for each statistic (number of commits, pull requests, issues, stars and followers), based on the cumulative distribution function of the exponential and the log-normal distributions. The implementation can be investigated at src/calculateRank.js. The circle around the rank shows 100 minus the global percentile.

Hope that helps 👍🏻.

[ishqDehlvi]

Thanks..

[rickstaa]

Thanks..

No problems, if you have any questions or feedback, feel free to let us know 👍🏻.