Kafka listeners sslClientAuth setting ignored for SSL configurations

[rquintos]

Name and Version

bitnami/kafka 31.0.0

What architecture are you using?

amd64

What steps will reproduce the bug?

Description

Installing kafka with a mix of SSL protocol listener auth types for sslClientAuth never sets a listener override, such as listener.name.<sslListenerName>.ssl.client.auth. If the listener protocol set is SSL you will always get what is set at tls.sslClientAuth even if you set the optional sslClientAuth for the listener protocol.

Note

Found this in chart 26.5.0, but I'm reporting it against 31.0.0 because I still see the same code in 31.0.0.

1. Setup SSL protocols for some or all of your listeners. (See custom values.yaml entries)
2. Perform a standard helm install

   helm install kafka bitnami/kafka --version 31.0.0 --values=values.yaml

Are you using any custom parameters or values?

This is a snippet of the core sections related to this issue. This is a legacy setup so we have an existing zookeeper installed via a separate helm install.

kraft:
  enabled: false
zookeeper:
  enabled: false
tls:
  truststorePassword: mypassword
  keystorePassword: mypassword
  type: JKS
  endpointIdentificationAlgorithm: ""
  sslClientAuth: "requested"
  existingSecret: my-existingsecret
  jksKeystoreKey: my.keystore.jks
  jksTruststoreKey: my.truststore.jks
listeners:
  client:
    containerPort: 9092
    protocol: SSL
    name: CLIENT
  controller:
    name: CONTROLLER
    containerPort: 9093
    protocol: SSL
    sslClientAuth: "none"
  interbroker:
    containerPort: 9094
    protocol: SSL
    name: INTERNAL
    sslClientAuth: "required"
  external:
    containerPort: 9095
    protocol: SSL
    name: EXTERNAL
    sslClientAuth: "required"

What is the expected behavior?

Main expectation is that I'd get separate protocol listeners for each SSL sslClientAuth I have set.

ConfigMap
Expected in the generated ConfigMap kafka-broker-configuration and the CLIENT protocol defaults to tls.sslClientAuth since sslClientAuth is not set.

listener.name.internal.ssl.client.auth=required
listener.name.external.ssl.client.auth=required
listener.name.controller.ssl.client.auth=none

The server.properties File in the Broker
The server.properties file should then in turn get configured with the same configurations from the ConfigMap

listener.name.internal.ssl.client.auth=required
listener.name.external.ssl.client.auth=required
listener.name.controller.ssl.client.auth=none

What do you see instead?

None of the listener.name.<sslListenerName>.ssl.client.auth settings are written to the ConfigMap that is used to generate the server.properties file.

Additional information

Possible Introduction of This Change
Upon reviewing KIP-684 it appears that functionality was added to create the override listener auth types for SASL_SSL. However, in the scenario where I also want different auth types for multiple SSL listeners they get defaulted to tls.sslClientAuth. This is because the _helpers.tpl logic that creates listener specific auth settings is nested in the # Listeners SASL JAAS configuration section here.

Workaround
As a workaround I explicitly add the values below to the extraConfig section.

extraConfig: |-
    listener.name.internal.ssl.client.auth=required
    listener.name.external.ssl.client.auth=required
    listener.name.controller.ssl.client.auth=none

Hints as to why this functionality is unintended

If I read the line 193 of the values.yaml file I'm under the impression that the setting of sslClientAuth should override tls.sslClientAuth.

If SSL protocol is enabled, overrides tls.sslClientAuth for this listener.

If I also review the Configuration Options in KIP-684 I see that it notes...

SSL listeners will continue to use this configuration with the following order of precedence (no change):

1. listener.name.<sslListenerName>.ssl.client.auth
2. ssl.client.auth

However, the point 1 will never be satisfied since we never hit that logic in the _helpers.tpl file, as noted previously.

Minor possible typo*
For the additional protocols listed in the values file, the comments read as:

If SSL protocol is enabled, overrides tls.authType for this listener.

Is tls.authType supposed to read as tls.sslClientAuth as it is in line 193 ?

If this is a Bug

If upon review of this issue this is confirmed to be missing functionality, I can push a potential pull request after testing if that helps out.

[carrodher]

Hi, the issue may not be directly related to the Bitnami container image/Helm chart, but rather to how the application is being utilized, configured in your specific environment, or tied to a particular scenario that is not easy to reproduce on our side.

If you think that's not the case and want to contribute a solution, we'd like to invite you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Please feel free to contact us if you have any questions or need assistance.

Suppose you have any questions about the application, customizing its content, or technology and infrastructure usage. In that case, we highly recommend that you refer to the forums and user guides provided by the project responsible for the application or technology.

With that said, we'll keep this ticket open until the stale bot automatically closes it, in case someone from the community contributes valuable insights.

[rquintos]

@carrodher, Much appreciated on the feedback. I'll look at testing something out and submitting a PR.