Compatibility issues with fatjar task

[itsnotoger]

I had an issue with my fatjar task, causing my project to be unable to build, and technically not even get past configuration time of tasks.

My setup is a root gradle project, and some subprojects. One subproject was a lib, and one was an app, that normally pulls in the lib as a dependency. It was done like dependencies { implementation(project(":lib")) }.
The fatjar task would collect files from the runtimeClasspath. The task itself comes from my own plugin, and it should just be generally available as a task in any project.
The extra-java-module-info plugin is not being actively used in either subproject. lib is just a vanilla modularized project with module-info file. (The extra-java-module-info plugin is still applied to it, because I apply it to all subprojects.)

The error I get is this

Could not determine the dependencies of task ':app:fatJar'.
> Could not resolve all files for configuration ':app:runtimeClasspath'.
   > Failed to transform lib.jar (project :lib) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.dependency.bundling=external, org.gradle.jvm.version=21, org.gradle.libraryelements=jar, org.gradle.usage=java-runtime}.
      > Execution failed for ExtraJavaModuleInfoTransform: D:\git\misc\Misc\lib\build\libs\lib-0.0.1-SNAPSHOT.jar.
         > Transform output D:\git\misc\Misc\lib\build\libs\lib-0.0.1-SNAPSHOT.jar must exist.

As I understand it, the issue happens because at task configuration time of app. The fatjar task eagerly declares at its configuration that the app classpath is to be consumed. I believe even just the evaluation of this classpath requires the dependency on lib, where now the extra-java-module-info plugin, requires a jar to be already built (presumably so that extra-java-module-info can read its module-info). And if the lib happens to be built separately beforehand, this works, but otherwise, the lib will not even get a chance to be built, because the issue happens at task configuration time already (before we get to execution).

I was able to create a workaround, by creating a separate classpath just for my fatjar task. Then set the javaModule Attribute to false on it. (This then also caused issues with the javafx plugin, which then is confused about two variants of producers being there. I had to add the right set of Attributes and also turn off classpath.isCanBeConsumed so it doesn't intefere with regular builds.)

I'm not sure if there is much on the side of the extra-java-module-info plugin to be done here. When working with thus plugin, I generally ran into problems a lot where during task configuration times, dependencies are already required, and I was thinking it would likely be better to delay such checks for execution time. But I'm also not sure how the plugin functions and why it works the way it does.
Maybe you have an idea how to help with this issue, otherwise we can also close the issue. I just wanted to document this in case someone comes across it.

A minor workaround could be if the extra-java-module-info plugin was able to read the module-info files from the src of a dependency, rather than the jar. This would certainly help in multi-module projects with native modularization.
If the dependency checking at configuration time was a deliberate choice, and easily moved to execution, then I would suggest to just print warnings during configuration time, without causing errors.

[jjohannes]

Hi. If you get this error during configuration time, it is an indication that something is ill-configured. Unfortunately, Gradle is not very helpful with information about this.

If you do the dependency resolution during configuration time, the transform cannot run because local artifacts are not yet built. But instead of giving a good error, Gradle still runs the transform without the file being present. I yet have to investigate if we can give a better/useful error from the plugin (this is #106).

In your case, if adding the fatJar task is causing this, I assume that the setup of that task somehow triggers the dependency resolution early. This can easily happen and is very common due to the messy API on the Configuration interface, which implements Set<File>. And as soon as you access it as a Set<File>, dependency resolution gets triggered under the hood (without warning). E.g. things like runtimeClasspath.iterator() or runtimeClasspath.stream().map... are all things you should not do during configuration time. If you need to do anything on the classpath lazily, you can use runtimeClasspath.elements.map {} (which is Gradle's Provider.map).

If this does not help, please share your setup and I can have a quick look.

[itsnotoger]

Thank you for the help. I was not aware of the .elements API that allows you to work with Providers.

This just works, and is a lot more straightforward. (You do need the double .map though, one to map the provider, one to map the set.)

It would of course be nice, if there was an error message that would have informed me about general solutions to issues, using Provider API being one of them.
It would also be nice for Gradle to guide me towards migration to Provider APIs in general though, but I'm glad it works nicely now.