Compare and visualise two reports in MD format

[ChristopherPHolder]

Motivation

Every change in the code base could potentially impact the application's performance.
We need a way to inform developers if the changes were performance slowdowns or improvements.

To accomplish this, we need a way to measures from a current audit with measures from a previous audit.

Implementation

There are different building blocks necessary to implement this feature:

Building blocks

• Report comparison - a way to compare results with previous results
  • Selecting comparison - a way to specify that the comparison should be created
  • Specifying baseline - a way to specify a report to compare the new measures with
• Report comparison visualization - a human-readable version of the comparison eg. MD table
  • Persistance option - logic to name and save the md report

Suggested solution

Report Comparison

The report would expand the reduced report by adding baseline measures.
This reduced report comparison would then be used to generate the Md table.

{
  "name": "Sandbox Setup StaticDist",
  "steps": [
    {
      "name": "Navigation report",
      "gatherMode": "navigation",
      "results": {
        "performance": 0.95,
        "accessibility": 1,
        "best-practices": 0.92,
        "seo": 1,
        "pwa": 0.3
      },
      "baseline":  {
        "performance": 0.89,
        "accessibility": 1,
        "best-practices": 0.95,
        "seo": 0.9,
        "pwa": 0.2
      }
    },
  ]
}

Selecting Comparison

TODO
Related question:

• Should this be the default?
• Should this be a optional RC & CLI param?

Specifying Baseline

TODO
Related question:

• Should specifying the baseline be the selection method ?
• Should specifying the baseline mean that Md and Stdout are automatically comparisions?

Report Comparison Visualization

The report would add the comparison number next to the current measurement

Step Name	Gather Mode	Performance	Accessibility	Best Practices	Seo	Pwa
Navigation report	navigation	95 (+10)	100	82 (-11)	100	30 (-5)
Timespan report	timespan	10/11 (-2)	-	5/7 (-1)	-	-
Snapshot report	snapshot	Ø 3/4	10/10	4/5	9/9	-

The report would be styled to make it easier to spot which measures when up or down.
Some options for this styling are:

• emojis ( 🟢 | ⚪ | 🔴 ), (💚| 🔻), (🔺| 🔻), ( ⏫ | ⏬ )
• line breaks <br />
• parenthesis and plus-minus (+ #), (- #)

Step Name	Gather Mode	Performance	Accessibility	Best Practices
Option 1	navigation	95 (+10 💚)	100	82 (-11 🔻)
Option 2	navigation	95 (+10 🟢)	100 (⚪)	82 (-11 🔴)
Option 3	navigation	95 (+10 ⏫)	100 (⚪)	82 (-11 ⏬ )
Option 4	navigation	95 💚 +10	100	82 🔻 -11
Option 5	navigation	95 🟢 +10	100 ⚪	82 🔴 -11
Option 6	navigation	95 +10 ⏫	100	82 -11 ⏬

Persistance Option

TODO
Related question:

• Should this be simplfy an extention of MD and Stdout or a seperate persist format?
• Should the persisted report include a naming convention so specify its not the same as the MD and avoid name colitions?

References

• Run User Flows as a GitHub Action
• GH comments
• Comparison visualizations:
  • 
  • 
  •