Single-Use Simulators

[SlaunchaMan]

This PR modifies the runTests action to automatically create a new simulator for each device it’s going to run on. This will prevent tests from interfering with one another, as each test will run in its own environment.

[jshier]

There doesn't seem to be any cleanup in this PR. Also, it would great if the cleanup (along with other cleanup steps) were resilient to failed jobs, as errors in the fastlane run means we don't get the cleanup steps.

[jshier]

I'm also curious what benefits this would bring, as I don't recall any CI issues that would've been helped by having a new simulator for every test run. Is the simulator the only thing that prevents parallel testing runs for multiple CI jobs? Also, have you measured the performance cost of creating and destroying a simulator for every iOS job?

[SlaunchaMan]

The included plugin does in fact catch errors during a test run and clean them up, as well as deleting the simulator after each use. The main benefit of doing this is that you get a fresh simulator every time, and if two jobs run in parallel, they will use their own simulators.