🐈 Task: Integrate Ruff as Linter, Formatter, and Pre-commit Hook

[Abellegese]

Overview

Maintaining code quality and consistency is crucial in collaborative projects like Ersilia. I am inspired by the recent ersilia codebase formatting using black by @DhanshreeA and this issue aims to make it more robust, also contributor focused and easy. It will also isort(supported built in by ruff not need to install) alongside Ruff ensures our Python imports are logically grouped and consistently ordered, improving readability and reducing merge conflicts. So in the end we hope for this:

• Enforce coding standards with minimal manual intervention.
• Automate linting, formatting, and import sorting in a unified workflow.
• Ensure that contributors stick to the project's conventions, reducing review cycles.

Why Ruff?

Ruff is a fast(x21 faster than the best linter autoflake), all-in-one tool for Python linting and formatting. It consolidates multiple tools like Pyflakes, Pycodestyle, black and isort into a single solution.

Integration Plan

1. High-level ruff.toml configs:
   Two files, ruff.toml and .pre-commit-config.yaml will be created at project root. In the ruff.toml file includes key configurations:

• General Settings

  • Line length: Set to 88 characters (default for PEP 8).
  • Indent width: Use 4 spaces for Pythonic consistency.
  • Target Python version: Configured for Python 3.10.

• Linting Rules

  • Focus on common error and style codes like E (errors), F (Pyflakes), and W (warnings).
  • Enable additional checks for unused variables, annotations, and import sorting.
  • Configure per-file ignores for test files and specific rules that may not apply globally.

• Formatting Preferences

  • Use double quotes for strings.
  • Enable trailing commas for better diff readability.
  • Configure imports with logical grouping (FUTURE, STDLIB, THIRDPARTY, etc.). Relative import sorting in Ersilia case.

line-length = 88
indent-width = 4
target-version = "py310"

[int]
select = [
    "E4", "E7", "E9", "F", "E", "W", "C90", "I", "N", "D", "UP", "YTT", "ANN", "B", "A", "COM",
    "C4", "DTZ", "ISC", "ICN", "INP", "PIE", "PT", "RET", "SIM", "TID", "ARG", "PLC", "PLE", "PLR",
    "PLW", "RUF",
]
ignore = [
    "D105", "D107", "D203", "D212", "UP006", "UP007", "ANN101", "ANN102", "D400", "D406", "D407",
    "PLC1901", "UP035",
]
unfixable = ["F401", "F841"]
fixable = ["ALL"]

[lint.per-file-ignores]
"test_*.py" = ["ANN001", "ANN002", "ANN003", "D101", "D103"]

[lint.pyupgrade]
keep-runtime-typing = true

[format]
quote-style = "double"
indent-style = "space"
skip-magic-trailing-comma = false
line-ending = "auto"
docstring-code-format = false
docstring-code-line-length = "dynamic"
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

[isort]
sections = ["FUTURE", "STDLIB", "THIRDPARTY", "FIRSTPARTY", "LOCALFOLDER"]
line_length = 88
multi_line_output = 3
include_trailing_comma = true

2. Set Up Pre-commit Hooks:
   In the second file .pre-commit-config.yaml its going to be configured which ensures a easy and comprehensive workflow for linting and fixing issues with Ruff. It includes safe fixes for automated enforcement and an option for more advanced, potentially unsafe fixes during manual runs which in our case both are included.

repos:
  - repo: https://github.com/charliermarsh/ruff-pre-commit
    rev: v0.8.2 # latest
    hooks:
      - id: ruff # default is ruff
        args: [--fix]
        stages: [pre-commit]

3. Workflow Integration

To integrate Ruff linting into the CI/CD pipeline, the following two options are proposed:

Option 1: Create a New Workflow File
A new file named .github/workflows/lint.yml can be created to handle Ruff linting independently. This workflow will be triggered on push and pull_request events, ensuring code quality checks are run separately from existing workflows.

workflow samples:

name: Linting Workflow

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: 3.10

      - name: Install Ruff
        run: |
          python -m pip install --upgrade pip
          pip install ruff

      - name: Run Ruff Linting
        run: ruff check .

Option 2: Integrate Ruff Linting into pr_check.yml
Ruff can be added to the existing pr_check.yml workflow, executed before the current steps. This approach consolidates all validation steps, ensuring code is linted and validated in the same workflow.

Modification to pr_check.yml:

  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # [email protected]

      - name: Add conda to system path
        run: echo $CONDA/bin >> $GITHUB_PATH

      - name: Set Python to 3.10.10
        run:
         conda install -y python=3.10.10 
      
      - name: Source conda
        run: source $CONDA/etc/profile.d/conda.sh

      - name: Install dependencies
        run: |
          source activate
          conda init
          conda install git-lfs -c conda-forge
          git-lfs install
          conda install gh -c conda-forge

      - name: Install Ruff for linting
        run: |
          source activate
          pip install ruff

      - name: Run Ruff Linting
        run: |
          source activate
          ruff check .

4. Usage Documentation:
   Provide clear instructions for contributors to install and use these tools.

---

High-level Contributor Guide

1. Install Dependencies:
   Install the required tools using pip:

   pip install ruff pre-commit

2. Activate Pre-commit Hooks:
   Initialize the pre-commit hooks after cloning the repository:

   pre-commit install

3. During Development:

   • On commit, Ruff will automatically:
     • Lint code for errors and style issues.
     • Format code and sort imports using isort.
   • Fix any issues reported before committing.

4. Manual Usage:
   To manually lint, format, or sort imports, run:

   ruff check . --fix

   or unsafe mode

   ruff check . --fix --unsafe-fixes

5. Commit and Push Changes:
   Resolve all issues, then commit and push your code.

---

References

1. Ruff Documentation: https://beta.ruff.rs/docs/
2. isort Documentation: https://pycqa.github.io/isort/
3. Pre-commit Documentation: https://pre-commit.com/

Objective(s)

• Consistent and high-quality codebase.
• Automated linting and formatting to streamline development workflows.
• A smoother code review process with fewer style-related comments.

[DhanshreeA]

@Abellegese what is the update on this?

[Abellegese]

@DhanshreeA I finished it, I will create a PR.