Support classpath testing (not changing Gradle default setup for whitebox)

[xenoterracide]

So from what I read here. By default this overrides gradle's default test stuff. I don't actually want that, most of the time I want whitebox testing. The only reason I want blackbox testing is to ensure that the right packages are exported. It's not clear from the documentation whether I could add a jvm test suite with this that is just the addition of blackbox testing, and leaves the default alone.

[jjohannes]

I am not sure I fully understand what you ask for. To clarify:

If you want blackbox testing n the module path (test suites with module-info.java) and run other tests on the classpath (test suites without module-info.java) you do not need this plugin. That's what Gradle is doing out of the box.
There is one setup issue if your blackbox test contains resources (which this plugin addresses). But that's a minor thing. If that is the only thing you want to have changed, I recommend doing that directly (register the "testJar" task and adjust the test classpath) without this plugin. As it is done in this sample: https://docs.gradle.org/current/samples/sample_java_modules_multi_project_with_integration_tests.html

The main reason for this plugin is whitebox testing: no module-info, but running on the module path by patching the "main" module. And with that have a setup where all tests run on the Module Path.

[xenoterracide]

appears to be whitebox to me, this test wouldn't pass if test wasn't whitebox, I'm not exporting the module. All I want is to be able to write a test that ensure the appropriate packages are exported. Having separate gradle modules is "expensive" regarding how much code I have to additionally write and maintain.

https://github.com/xenoterracide/java-commons/blob/chore/publication/module/tools/src/test/java/com/xenoterracide/tools0/java/internal/DemoWhiteBoxTest.java

https://github.com/xenoterracide/java-commons/blob/chore/publication/module/tools/src/main/java/com/xenoterracide/tools0/java/internal/DemoWhiteBox.java

https://github.com/xenoterracide/java-commons/blob/chore/publication/module/tools/src/main/java/module-info.java

[xenoterracide]

yeah, it looks like this is a must feature for me because of #51 if I could just turn it off for test suites where I want to test package private things that'd be perfect, but it appears that it wants to take over all jvm test suites, so I can't even make another one with the module path not added in some way.

Guess my option is fork this plugin and dig around?

[jjohannes]

I think we might have a bit of misunderstanding about what "Whitebox" means here. Because the plugin does something to do "Whitebox" testing with modules (merging main and test Module). Which in the terminology used in this plugin is Whitebox Testing (on the Module Path).

If I understand it correctly, you are talking about running tests on the Classpath. Which is what Gradle is doing when you do not use this plugin and do not have a module-info.java for your tests.

I still think a really good setup would be to not test on the Classpath at all. But in practice, this is difficult to achieve with certain test frameworks and libraries that do e.g. mocking stuff and need access to more things than the Module System would allow.

Not sure if that is your use case, but in any case I think we can add an option to tell the plugin to not touch a "Test Suite" to do "classpath-based" testing. I think I would call it something like that:

javaModuleTesting.classspath(testing.suites["test"])

[xenoterracide]

I choose to slice my modules both vertically and horizontally, e.g feature.model and feature.controller to avoid layer specific code from leaking into another layer and features being preserved in isolation. Most of my code can be tested using blackbox tests with a module-info. Vampire (on slack gradle) says that even adding a module-info to gradle without using this will still leak (or allow hiding?) of resources. Something like 10-20% of my code needs to be whitebox-ed because I'm hiding something super specific in the module behind package/protected code. I use the Testing Trophy instead of the Testing Pyramid. 99% of the time mocking is a bad idea.

What I think I want to do is alway use test for blackbox testing, and then make a jvm test suite like testWhiteBox for whitebox testing. I just don't see the need to have non classpath code overhead if I always have a blackbox suite.

javaModuleTesting.classspath(testing.suites["test"])

that would probably be fine. Bonus points if I can use the resulting variable name when I create the jvm suite instead of the stringy name. note: this may already be your intent, and I'm being obtuse because I'm not certain what that groovy code does and don't use the test suites api I enough ;)

val testWhiteBox by getting(JvmTestSuite::class) { 
            useJUnitJupiter() 
        }

javaModuleTesting.classpath(testWhiteBox)

p.s. speaking of mocks, and and otherwise entirely unrelated to this issue. I'm trying to create a module-info PR for mockito. They have excluded their internal package from javadoc creation, but even if I open/export the internal package javadoc is telling me it doesn't exist. I'm speculating this requires me to change the module-path option somehow but I haven't the foggiest what it needs to be and google eludes me mockito/mockito#3381

[xenoterracide]

any chance this will be seen anytime soon? was just going to go ahead and apply in my buildSrc and then I remembered this problem. Another reason this would be good is incremental progression towards a modular build.

[jjohannes]

I agree that this feature should be in the next release. I had some busy weeks. I'll try to get to this soon.