How to run multiple dependent commands in a task?

[nilscrm]

The concrete setting: I have two modules, one compiler and one runtime of sorts. I want to test the runtime on some compiled programs. For this I want to have a Task that compiles the test programs from the resource directory before running the tests.

So in the runtime module I have the following:

def hvmTestFiles = Task.Source(millSourcePath / "resources" / "hvm")

def allHvmTestFiles = T {
  os.walk(hvmTestFiles().path).filter(_.ext == "hvm")
}

Now I want to have another task than calls the compiler on all the test programs. My first thought was the following:

def compileHvmFiles = T {
  val files = allHvmTestFiles()
  files.map { file => compiler.run(Task.Anon(Args(Seq("--hvm-input", file.toString, "--output-dir", Task.dest.toString))))() }
}

This gives an error though as the input argument for the command was generated within the Task itself.:

Target#apply() call cannot use `value file` defined within the Task{...} block
[build.mill-58] [error]       files.map { file => compiler.run(Task.Anon(Args(Seq("--hvm-input", file.toString, "--output-dir", Task.dest.toString))))() }

The only way I see to circumvent this is to not use the compiler's run command and call it myself by running the jar file. This seems unsatisfactory though. Is there a way to do this properly?

[lihaoyi]

Have you seen the options presented on this page?

https://mill-build.org/mill/extending/running-jvm-code.html#_running_a_scalamodule_in_a_subprocess

I'm not sure if I understand your use case correctly, but it seems to me that it should be satisfied by the linked examples

[nilscrm]

Thanks, with that I was able to make it work:

def compileHvmFiles = T {
  val files = allHvmTestFiles()
  files.foreach { file => compiler.runner().run(Map("--hvm-input" -> file, "--output-dir" -> Task.dest)) }
  T.dest
}

Though I still don't quite get the error I was getting before with compiler.run().

[lihaoyi]

Yeah it's pretty tricky; the basic issue is that passing a Task[Args] to def run means that the Task[Args] gets evaluated before compileHvmFiles gets evaluated, due to Mill's evaluation model (see docs). That means that it cannot depend on anything that's computed in the body of compileHvmFiles, such as the file value.

It's possible to make the stuff above work, but it's pretty gnarly. Generally the runner().run API is much easier to use (at the cost of a bit of flexibility you probably don't need), so better to just use that and avoid calling run(args: Task[Args]) programmatically. That's what we currently recommend in the documentation.

[nilscrm]

the Task[Args] gets evaluated before compileHvmFiles gets evaluated, due to Mill's evaluation model (see docs). That means that it cannot depend on anything that's computed in the body of compileHvmFiles, such as the file value.

That makes total sense, thanks!