Mark timeout tests as timed out instead of failed

[dotMorten]

Description

If you add a [Timeout(xxx)] attribute to a test, and it times out, the test is shown as failed with a System.Timeout exception. There's no way to tell if the test actually timed out due to this attribute, or whether the test itself threw this exception unexpectantly deeper down below the test-runner.

The set of TestOutcome values declared in the TRX Schema actually lists a Timeout option, however it appears this is never used, and the Failed status is used instead.

Instead I believe the test runner should declare a private timeout exception type to throw, and when the test result is set, it's checked if this is the exception that was thrown, and instead set the status to timeout, rather than setting it to failed with a useless stacktrace.

Steps to reproduce

Create a unit test with a timeout attribute, make it run longer than the timeout setting.

Expected behavior

Test should be marked timed out in Test Explorer. TRX report should flag it as timed out instead of Failed.

Actual behavior

Test is flagged as failed.

AB#1277751

[Sanan07]

Currently when we have TimeOut[1000] attribute and test timeouts we get error message in console "Test 'TestMethod1' exceeded execution timeout period.", but in trx logger we see that test marked as ResultSummary outcome as "Failed", but in Counter we have failed="1", instead of timeout="1"
Actual :

<ResultSummary outcome="Failed">
   <Counters total="1" executed="1" passed="0" failed="1" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
</ResultSummary>

Expected:

<ResultSummary outcome="Failed">
   <Counters total="1" executed="1" passed="0" failed="0" error="0" timeout="1" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
</ResultSummary>

Changes : In Counters tag timeout set to "1" as test actually timeouted.

[nohwnd]

This is a new feature and won't be implemented, we are focusing on adding new features to Testing.Platform instead. https://aka.ms/testingplatform