Dependencies shared with mvn2nix itself do not have URLs in output

[charles-dyfis-net]

Given the following POM:

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>mvnTest</groupId>
  <artifactId>mvnTest</artifactId>
  <packaging>jar</packaging>
  <version>0.0.1-SNAPSHOT</version>
  <name>mvnTest</name>
  <description>A demonstration</description>
  <build>
    <sourceDirectory>src</sourceDirectory>
    <directory>target</directory>
    <outputDirectory>target/classes</outputDirectory>
    <plugins/>
  </build>
  <dependencies>
    <dependency>
      <groupId>org.clojure</groupId>
      <artifactId>tools.reader</artifactId>
      <version>0.10.0</version>
    </dependency>
  </dependencies>
</project>

...the generated project-info.json references several artifacts which are not given URLs:

$ mvn org.nixos.mvn2nix:mvn2nix-maven-plugin:mvn2nix &>/dev/null && jq -r '.dependencies[] | select(.url? == null) | [.groupId, .artifactId, .version, .extension] | join(":")' <project-info.json
org.apache.maven:maven-parent:21:pom
org.apache:apache:10:pom
org.sonatype.oss:oss-parent:7:pom
org.sonatype.spice:spice-parent:17:pom
org.codehaus.plexus:plexus-interpolation:1.14:jar
org.codehaus.plexus:plexus-interpolation:1.14:pom
org.codehaus.plexus:plexus-component-annotations:1.5.5:jar
org.codehaus.plexus:plexus:3.3.1:pom
org.apache.maven:maven-parent:11:pom
org.apache:apache:5:pom
org.codehaus.plexus:plexus-components:1.1.18:pom
org.codehaus.plexus:plexus:2.0.7:pom
org.apache.maven.wagon:wagon:1.0-beta-6:pom
org.codehaus.plexus:plexus-component-annotations:1.5.5:pom
org.codehaus.plexus:plexus-containers:1.5.5:pom
org.apache.maven.wagon:wagon-provider-api:1.0-beta-6:jar
org.apache.maven.wagon:wagon-provider-api:1.0-beta-6:pom

This is a strict subset of the dependency tree of the mvn2nix plugin itself, as shown by the following code:

(cd ~/VC/mvn2nix-maven-plugin &&
  mvn org.nixos.mvn2nix:mvn2nix-maven-plugin:mvn2nix >&2 &&
  { jq -r '.dependencies[] | [.groupId, .artifactId, .version, .extension] | join(":")' | sort; } <project-info.json >all-deps.txt
)
(cd ~/VC/mvnTest &&
  mvn org.nixos.mvn2nix:mvn2nix-maven-plugin:mvn2nix >&2 &&
  { jq -r '.dependencies[] | select(.url? == null) | [.groupId, .artifactId, .version, .extension] | join(":")' | sort; } <project-info.json >no-url-deps.txt
)
comm -13 ~/VC/mvn2nix-maven-plugin/all-deps.txt ~/VC/mvnTest/no-url-deps.txt

...which returns 0 (as the number of lines in no-url-deps.txt but not also in all-deps.txt).

[charles-dyfis-net]

...and, just to be clear that (some of) these items being missing represents a problem:

[chaduffy@app-build-04:~/VC/mvnTest]$ nix repl
Welcome to Nix version 2.1.3. Type :? for help.

nix-repl> :l ../nixpkgs/default.nix
Added 9721 variables.

nix-repl> :b (pkgs.buildMaven ./project-info.json).build
builder for '/nix/store/xcf18ykbx05ml0xa7dk5s2clj1xqjn2d-mvnTest-0.0.1-SNAPSHOT.jar.drv' failed with exit code 1; last 10 log lines:
  [INFO] Total time: 0.216 s
  [INFO] Finished at: 2018-11-29T22:26:05Z
  [INFO] ------------------------------------------------------------------------
  [ERROR] Failed to execute goal on project mvnTest: Could not resolve dependencies for project mvnTest:mvnTest:jar:0.0.1-SNAPSHOT: Failed to collect dependencies at org.clojure:tools.reader:jar:0.10.0: Failed to read artifact descriptor for org.clojure:tools.reader:jar:0.10.0: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.sonatype.oss:oss-parent:pom:7 has not been downloaded from it before. -> [Help 1]
  [ERROR]
  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
  [ERROR]
  [ERROR] For more information about the errors and possible solutions, please read the following articles:
  [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/xcf18ykbx05ml0xa7dk5s2clj1xqjn2d-mvnTest-0.0.1-SNAPSHOT.jar.drv' failed

[charles-dyfis-net]

Tracing through the code, there are two places where url can get populated:

• The first is only relevant if art.isSnapshot() is true.
• The second is only relevant if res.getRepository() returns a RemoteRepository. This isn't happening for the items being excluded as discussed in this ticket: a LocalRepository is being returned instead for the items which aren't being printed.

For Maven to operate, it and its dependencies are necessarily in the local repository, hence those items not having URLs assigned.

[charles-dyfis-net]

The above PR doesn't fix every broken case -- if an artifact can only be resolved with a repository added by a POM that included it, that URL still gets missed -- but that particular case has an immediate workaround (propagating such repository URLs up into the POM being operated against), and is rather distinct from the scenario this ticket is focused on.

[cartazio]

I've hit this problem to!

How would I use your PR / patchset to use mvn2nix with this fix? (i'm not very familiar with java/maven stuff, just trying to get nix setup for some colleagues)

[charles-dyfis-net]

I've hit this problem to!

How would I use your PR / patchset to use mvn2nix with this fix? (i'm not very familiar with java/maven stuff, just trying to get nix setup for some colleagues)

@cartazio, apologies about the delayed response -- I wanted to wait until I actually needed to do this again so my instructions could be audited. :)

Check out this branch of mvn2nix locally, navigate into its source directory, and run mvn install. This should install a copy with a -SNAPSHOT suffix in your local store (your ~/.m2 directory).

Then go to the source directory of the project you're trying to build a project-info.json for, and run mvn org.nixos.mvn2nix:mvn2nix-maven-plugin:1.2.2-SNAPSHOT:mvn2nix.