Use of mixin with spread gives unexpected result

[dshatokhin]

Hi, everyone!

I'm trying to deeply merge 2 dynamic objects and found out about Mixin from discussion #505 .
It's working as expected while using suggested approach:

// Working example

hidden defaultParams {
  options {
    enabled = false
    someOpt = "someValue"
  }
}

hidden overridesLiteral: Mixin<Dynamic> = new {
  options {
    enabled = true
  }
}

merged = defaultParams |> overridesSpread

Gives expected output:

merged {
  options {
    enabled = true
    someOpt = "someValue"
  }
}

However, when using spread for Mixin it's producing different result:

// Spread example

hidden defaultParams {
  options {
    enabled = false
    someOpt = "someValue"
  }
}

hidden customParams {
  options {
    enabled = true
  }
}

hidden overridesSpread: Mixin<Dynamic> = new {
  ...customParams
}

merged = defaultParams |> overridesSpread

Gives different output:

merged {
  options {
    enabled = true
  }
}

What am I missing here and could it even be archived with use of a spread?

[HT154]

This is an extension of the case that came up in #505: the spread operator performs a property/member-wise assignment. In contrast, applying a mixin performs a property/element-wise amend.

Accordingly, in your example, the spread usage:

hidden overridesSpread: Mixin<Dynamic> = new {
  ...customParams
}

Is "de-sugared" to:

hidden overridesSpread: Mixin<Dynamic> = new {
  options = customParams.options
}

And here it's clearer to see that the resulting value for merged.options is exactly what was specified in customParams and not amended by it.

It's a little tricky to determine what you're going for overall here, but if I'm inferring correctly, it seems like customParams should itself be defined as a mixin from the start here to achieve the desired results.

[dshatokhin]

Thanks for the explanation, now I see what is going on.

While reducing my problem to provided examples I completely missed the important context - I'm reading customParams from a yaml file:

# customParams.yaml
options:
  enabled: true

And then the full example would be as such:

import "pkl:yaml"

hidden defaultParams {
  options {
    enabled = false
    someOpt = "someValue"
  }
}

hidden customParams = new yaml.Parser {}.parse(read("file:customParams.yaml"))

hidden overridesSpread: Mixin<Dynamic> = new {
  ...customParams
}

merged = defaultParams |> overridesSpread

merged {
  options {
    enabled = true
  }
}

I've tried

overrides: Mixin = new yaml.Parser {}.parse(read("file:customParams.yaml"))

but got an error:

Expected value of type 'Function1', but got type 'Dynamic'.

Is it possible to parse the file directly to Mixin typed variable?

[holzensp]

Considering you call it customParams, you may not be asking for a Mixin because you want to do it for a bunch of files, but just because you want the value of a property in the yaml file to "win" over some default. If that is the case you can do something like this;

import "pkl:yaml"

local yamlContents = new yaml.Parser {}.parse(read("file:customParams.yaml")) as Dynamic

foo: String = yamlContents.getPropertyOrNull("foo") ?? "foo value in case no foo in yaml"

That does require you to name all the properties you want to keep.

[dshatokhin]

Thanks for insights, it helps a lot to break a tunnel vision I acquired while working on the problem.
And now I know the proper name for the problem with no in yaml files.

I have a class defined as well as default values for it, so I am expecting the same structure in the customParams.yaml.
The deepToTyped package used a bit later in my code, after the combining default and custom parameters.

Right now, I'm trying to implement something like helm where I have default values for generating kubernetes manifests and also want to have an ability to pass overrides.

I'm doing it for the sake of learning and showcasing it on a meetup.

So, the idea is to do it as simple as possible and I started with overrides = defaultParams.toMap() + customParams.toMap() where my problem first occurred. My approach is to find the easiest and most straightforward solution before digging deeper.

Basically, I defined a default for generating kubernetes manifest, and seeking for a way to override only some of them in a nested object:

defaultValues = new Values {
  livenessProbe = new {
    httpGet {
      path = "/"
      port = "http"
    }
  }
}

And from customParams.yaml I want only the absolute minimum to update the defaults:

livenessProbe:
  httpGet:
    path: /live

I might be looking at the problem from the wrong angle and I think I could just rethink it or leave it as is and add a mention to my list of known limitations. Or maybe nested for could do the trick.

[holzensp]

Quick question; must the overrides be a YAML file, or could it be a Pkl file?

[holzensp]

I ask, because you're defining Pkl's amending quite precisely there. It sounds like you want a MyParams.pkl (upper case, because a template with defaults and type definitions, meant to be amended) with something like:

class HttpRequest {
  hidden type: *"GET"|"POST"|"PUT" //...
  path: String(startsWith("/")) = "/"
  port: *"http"|"https"|Int(isBetween(1,65535))
}

class LivenessProbe {
  httpGet: HttpRequest(type == "GET")
}

livenessProbe: LivenessProbe

output {
  renderer = new YamlRenderer {}
}

and in a subdirectory, somewhere, you might have myParams.pkl (lower case, because amending a template);

amends ".../MyParams.pkl"

path = "/live"

Now when you pkl eval myParams.pkl, you get

livenessProbe:
  httpGet:
    path: /live
    port: "http"

[dshatokhin]

Well, I would prefer yaml instead of pkl but the point of what I'm doing is to showcase pkl as a templating language which aware of kubernetes objects (and it's working really well).

It seems amends exactly what I need and it's really straightforward, so I'll stick to it 👍

Once again, thanks for answering my questions and helping with examples 🥇