Test plan

IN4302TU Building Serious Games

Test plan

Target audience

The people that will play our Light Rail Game will be the operators of HTM. They are a small group of people highly acquainted with all the in and outs of how trams work. They are certainly domain experts. Our target audience does already play games in the Railway domain, especially in the form of BAHN, a German railway game.

Requirements

The main goal is to create a game for tram traffic controllers, which stimulates creativity in dealing with disruptions, while being fun and engaging. To do so, some existing options and performance indicators need to be modelled in our game, for the game to be realistic, and for it to be used in trainings from Macomi to HTM.

User actions

When a disruption occurs, the operators currently lookup what the required action is for a specific situation. Those actions mainly consists of the following sub-actions:

• Rerouting lines from origin to destination
• Cancel line parts by letting vehicles turn and continue the line in the opposite direction, only servicing back and forth until the point of disruption.
• Cancel entire vehicle runs on lines
• Provide travellers information and advise them.

For the game to be realistic those actions should be available. Besides these actions other actions are also allowed if this stimulates the creativity or makes the game more fun.

Performance indicators

For the scoring in the game the following KPI's will be used:

• Recovery time until the normal situation
• Scale of the impact (The extent to which lines where effected)
• Size of the impact (Additional travel time for passengers)

Also, Macomi suggested not to model the individual passengers, so in our design we decided to measure the additional travel time for passengers as the delay of a vehicle arriving at a station.

Common disruptions

Again to be realistic the following disruptions should occur, possibly supplemented by disruptions that increase the fun-factor or make the game more engaging:

• Vehicle collisions with each other or other traffic (cars, bus, etc)
• Power down at parts of the track
• Defect traffic lights (Traffic needs to be manually guided)
• Defect vehicle (standing on the track or within depots)
• Defect switch (unable to make certain turns)
• Unavailability of driver due to sudden sickness

Testing the Requirements

There are several types of requirements. First of all there are the given list of existing user actions, performance indicators and common disruptions. Testing if these are implemented is fairly simple. We will check:

• Is the feature there
• Is it implemented in a clear way, such that the operator recognizes the disruption/action?
• Is it implemented in an accurate way, such that the operator finds is realistic and does not have many complaints due to his domain knowledge.

Next, there are the requirements that the game should be fun and engaging. This is difficult to measure, as fun is a subjective thing, adhering to a persons internal model. To establish how people experience the game we will use the following questionnaire, which will be answered on a Likert-scale:

Denote how much you agree to the following statements:

• The game is fun
• My fellow [operators/students/test-volunteers] think the game is fun
• The disruptions I encountered are realistic
• The disruptions I encountered are fun
• I could solve the disruptions as [in reality/I would have done in my job]
• I did use different actions than [in reality/I would have done in my job]
• I liked that there were special actions that I normally would not have
• I liked the way I could re-route trams
• I like the interface I was presented with
• The user interface, menu's, timers, etc, helped me/provided good overview
• The visual representation of the trams and environment was suitable for a game

If you have any further remarks on how you experienced the game or if you have any suggestions, please leave them below:

$ TextArea here $

Test planning

We will conduct 3 major tests:

• First Playable : we will test the First Playable with friends. As the First Playable will not be a finished product, with possibly some glitches or bugs, we can not test with Macomi or with HTM yet. In this fase the main goal is to find if the game is playable, and if it's fun. However we should keep in mind that our friends are not the target audience and that the operators might have more fun with a tram game, as it relates to them more (as they play and like the BAHN game as well).
• week before Beta : at the day of the mid-term presentation we will conduct a test with Macomi. One or multiple representaties of Macomi will try the game. We will ask them to fill in the questionnaire as if they are HTM operators, because they are a client of HTM and know the company. By combining the moments of presentation with the test we will use the time of Macomi efficiently.
• Final : our final product - being Macomi's first prototype - will hopefully be tested with HTM. This depends of the state of the game at that time, the availability of HTM operators and if Macomi is satisfied with the game enough for it to be presented to HTM. If we do the test we will let them play the game for 15 minutes and ask them the questionnaire. After the final test we will incorporate the test results in the recommendations for further development, and optionally fix some of the encountered problems, if there is time left.