Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce incremental structural Tests #377

Open
pvlov opened this issue Nov 20, 2024 · 1 comment
Open

Introduce incremental structural Tests #377

pvlov opened this issue Nov 20, 2024 · 1 comment

Comments

@pvlov
Copy link

pvlov commented Nov 20, 2024

Is your feature request related to a problem? Please describe.

Whenever you have an exercise which tasks the student to create lots of classes and may even include multiple levels of inheritance etc, you would usually use sequential testing to make this easier. This works perfectly fine, however this is most often quite annoying for the student since they first have to create all the classes and method stubs and so on. Ideally you should be able to test the structure incrementally for each class e.g. first test structure for class A then run the behaviour tests for class A.

Describe the solution you'd like
I do not want to change the way sequential tests are run, they work exactly as they should. What I'd ideally want is to expand the existing {Method | Class | ... }TestProviders. The new method would then return a Map<String, DynamicTest>, that maps the standard test method names to the actual test, which then can be run whenever you need it. You could then run the structural test per class by just putting it in the respective @BeforeEach method.

Describe alternatives you've considered
You can achieve the same by using the Dynamic{Method | Class |....} and then set boolean flags to then fail behavioural tests, but that doesn't really scale well.

Ares-Version you are using
Most recent one.

I'm up for implementing this, but I would like your opinion and feedback if this is desired or if I'm maybe overlooking something.

@pvlov
Copy link
Author

pvlov commented Nov 20, 2024

This would probably look something like this:

public Map<String, DynamicTest> generateTests() {
        Map<String, DynamicTest> tests = new HashMap<>();

	if (structureOracleJSON == null) {
		throw failure("The MethodTest test can only run if the structural oracle (test.json) is present. If you do not provide it, delete MethodTest.java!"); //$NON-NLS-1$
	}

	for (var i = 0; i < structureOracleJSON.length(); i++) {
		var expectedClassJSON = structureOracleJSON.getJSONObject(i);
		// Only test the classes that have methods defined in the structure oracle.
		if (expectedClassJSON.has(JSON_PROPERTY_CLASS) && expectedClassJSON.has(JSON_PROPERTY_METHODS)) {
			var expectedClassPropertiesJSON = expectedClassJSON.getJSONObject(JSON_PROPERTY_CLASS);
			var expectedClassName = expectedClassPropertiesJSON.getString(JSON_PROPERTY_NAME);
			var expectedPackageName = expectedClassPropertiesJSON.getString(JSON_PROPERTY_PACKAGE);
			var expectedClassStructure = new ExpectedClassStructure(expectedClassName, expectedPackageName, expectedClassJSON);

			var displayName = "testMethods[" + expectedClassName + "]";
			tests.put(displayName, dynamicTest(displayName, () -> testMethods(expectedClassStructure)));
		}
	}

	if (tests.isEmpty()) {
		throw failure("No tests for methods available in the structural oracle (test.json). Either provide attributes information or delete MethodTest.java!"); //$NON-NLS-1$
	}

	return tests;
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant