[REPORT] Automated Tests progress

[fenix-hub]

I want to share some progress on the development of an automated CI/CD pipeline for this addon.

2023/12/18

A new repository was set up for the development of a Github Action.
The goal of this action is to launch a Godot Engine project that tests the features of this addon.
Tests should run automatically before merging any new Pull Request.
A report should be created and attached as a comment to the PR.

Concept

The new repository should contain a ready-to-run project for Godot Engine. The project will be launched thorugh Godot CLI so no user interface is required.
The goal is to define a series of configurable and extensible test cases, and make a report out of it.
The testing process should follow such flow

--:INPUTS (addon, testing project, godot binaries)--> [ TESTING BOX ] --:OUTPUTS (report variable, passed variable)-->

• report variable: should consist in a Markdown table with the list of executed test cases and their outcomes.
• passed variable: a boolean passed to the workflow to indicate if all test passed or not. useful to directly know if there was something wrong during test and enable/disable next workflow steps.

The report table could look like this

TEST CASE	EXECUTION TIME	PASSED
test_name	30ms	✅ / ⛔

This table will be passed as a variable to the workflow in order to let the Github Action directly comment in a PR.

[fenix-hub]

2023/12/20

Automated testing can be considered completed.
The testing repository now contains a very small unit testing framework, that allows to write Test Suites in the form of classes and write test methods.

The current test procedure looks like this

# main.gd

func _init()

  var test_runner: TestRunner = TestRunner.new()
  
  test_runner.run([
      ExampleTestSuite.new("example_test_suite")
  ])
  
  TestReport.new(test_runner.summary).generate()

  quit()

And an example test suite looks like this:

extends TestSuite
class_name ExampleTestSuite

func test_generate() -> void:
	var test_secret: String = Crypto.new().generate_random_bytes(5).get_string_from_utf8()
	var jwt_algorithm: JWTAlgorithm = JWTAlgorithm.HS256.new(test_secret)
	var jwt_builder: JWTBuilder = JWT.create() \
		.with_expires_at(Time.get_unix_time_from_system()) \
		.with_issuer("Godot") \
		.with_claim("id","someid")
	var jwt: String = jwt_builder.sign(jwt_algorithm)
	var claims: Dictionary = JWTDecoder.new(jwt).get_claims()
	assert_has(claims, "iss")
	assert_eq(claims.iss, "Godot")
	assert_has(claims, "id")
	assert_eq(claims.id, "someid")

func test_verify() -> void:
	var test_secret: String = Crypto.new().generate_random_bytes(5).get_string_from_utf8()
	var jwt_algorithm: JWTAlgorithm = JWTAlgorithm.HS256.new(test_secret)
	var jwt_verifier: JWTVerifier = JWT.require(jwt_algorithm) \
		.with_any_of_issuers(["Godot"]) \
		.with_claim("my-claim","my-value") \
		.build() # Reusable Verifier
	var jwt_builder: JWTBuilder = JWT.create() \
		.with_expires_at(Time.get_unix_time_from_system() + 20) \
		.with_issuer("Godot") \
		.with_claim("my-claim","my-value")
	var jwt: String = jwt_builder.sign(jwt_algorithm)
	var exception: JWTVerifier.JWTExceptions = jwt_verifier.verify(jwt)
	assert_eq(exception, JWTVerifier.JWTExceptions.OK)

Any method that begins with test_* will be executed in the test suite.

At the end of the test, a summary is generated through Github actions. Example

If the action is triggered by a Pull Request, the action will add a comment summary:

Only if all tests are successful, the action is considered passed and the PR can be accepted. Otherwise, change requests should be attended.