Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build comparison table #34

Merged
merged 15 commits into from
Jul 17, 2023
Merged

Build comparison table #34

merged 15 commits into from
Jul 17, 2023

Commits on Jul 7, 2023

  1. Add Named pipeline

    @NISH1001
    NISH1001 committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    a4548a3 View commit details
    Browse the repository at this point in the history

  2. Add MetricResult as the only return type for any downstream 

    Metric.compute() method

Now `evalem._base.structures.MetricResult` dataclass is used as a dto to
any compute method return value.
    @NISH1001
    NISH1001 committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    418e58f View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2023

  1. Add utility to compare multiple evaluation 

    `evalem.misc.utils.build_comparison_table(...)` is added.

Note:
`evalem._base.evaluators.Evaluator(...)` now returns list of result, not
dict.
    @NISH1001
    NISH1001 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    aa1fc27 View commit details
    Browse the repository at this point in the history

  2. Bugfix rounding exception for NoneType scores

    @NISH1001
    NISH1001 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    387f1ed View commit details
    Browse the repository at this point in the history

  3. Add test coverage flag for pytest runner

    @NISH1001
    NISH1001 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    7464283 View commit details
    Browse the repository at this point in the history

  4. Bugfix bartscore test score range to +- infinity

    @NISH1001
    NISH1001 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    33a2143 View commit details
    Browse the repository at this point in the history

  5. Refactor docstring to accomodate for new changes

    @NISH1001
    NISH1001 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    a2c994a View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2023

  1. Move InstanceCountMixin to evalem._base.abc.py

    @NISH1001
    NISH1001 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    7038fa2 View commit details
    Browse the repository at this point in the history

  2. Compute average rouge score as an average of rouge1, rouge2, rougeL and 

    rougeLsum

Also, refactor test_metric_score
    @NISH1001
    NISH1001 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    d925fe5 View commit details
    Browse the repository at this point in the history

  3. Add MetricResult to evalem.__init__

    @NISH1001
    NISH1001 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    bed5db6 View commit details
    Browse the repository at this point in the history

  4. ignore pylint errors for evalem.__init__

    @NISH1001
    NISH1001 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    5839e04 View commit details
    Browse the repository at this point in the history

  5. Add pytest-cov to requirements.txt

    @NISH1001
    NISH1001 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    f5dcedd View commit details
    Browse the repository at this point in the history

  6. Add pytest-cov to pyproject.toml

    @NISH1001
    NISH1001 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    66944e1 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Add information about average total items evaluated while generating 

    comparison table
    @NISH1001
    NISH1001 committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    c14d35f View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2023

  1. Remove commented lines

    @NISH1001
    NISH1001 committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    a149a5f View commit details
    Browse the repository at this point in the history