You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In order to use benchmarks as performance regression tests, we need a way to assert on a performance regression. To do that, we need a way to summarize benchmark results, compare these benchmark summaries, and fail a benchmark when the comparison suggests a problem.
Implementation considerations
Summaries should be user-defined if at all possible (it really depends on the application which metrics matter and which don't). This ticket will likely require extending RobotFramework libraries in lambkin.shepherd.
The text was updated successfully, but these errors were encountered:
Feature description
In order to use benchmarks as performance regression tests, we need a way to assert on a performance regression. To do that, we need a way to summarize benchmark results, compare these benchmark summaries, and fail a benchmark when the comparison suggests a problem.
Implementation considerations
Summaries should be user-defined if at all possible (it really depends on the application which metrics matter and which don't). This ticket will likely require extending
RobotFramework
libraries inlambkin.shepherd
.The text was updated successfully, but these errors were encountered: