feat: Allow suppress diff line output by regex #475

jkroepke · 2023-06-28T21:16:12Z

This PR allow to suppress the diff report by using regex.
This option is more designed for power users and give them full control about the output.

There is a new diff option --suppress-output-line-regex which can be applied multiple time.

If a line from the report matches the regex, the line gets removed from the report. If the diff entry has no deltas, the whole diffEntry (file) gets removed, except the headline default, nginx, Deployment (apps) has changed:

Since --suppress-output-line-regex applied only to the output of an diff, the behavior of --detailed-exit-code is not touch. If there are differences which are suppressed, the exit code remains still 2.

Feature in action:

helm diff upgrade prometheus-node-exporter prometheus-community/prometheus-node-exporter --version 4.18.0

default, prometheus-node-exporter, DaemonSet (apps) has changed:
  # Source: prometheus-node-exporter/templates/daemonset.yaml
  apiVersion: apps/v1
  kind: DaemonSet
  metadata:
    name: prometheus-node-exporter
    namespace: default
    labels:
-     helm.sh/chart: prometheus-node-exporter-4.17.0
+     helm.sh/chart: prometheus-node-exporter-4.18.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: metrics
      app.kubernetes.io/part-of: prometheus-node-exporter
      app.kubernetes.io/name: prometheus-node-exporter
      app.kubernetes.io/instance: prometheus-node-exporter
-     app.kubernetes.io/version: "1.5.0"
+     app.kubernetes.io/version: "1.6.0"
  spec:
    selector:
      matchLabels:
        app.kubernetes.io/name: prometheus-node-exporter
        app.kubernetes.io/instance: prometheus-node-exporter
    updateStrategy:
      rollingUpdate:
        maxUnavailable: 1
      type: RollingUpdate
    template:
      metadata:
        annotations:
          cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
        labels:
-         helm.sh/chart: prometheus-node-exporter-4.17.0
+         helm.sh/chart: prometheus-node-exporter-4.18.0
          app.kubernetes.io/managed-by: Helm
          app.kubernetes.io/component: metrics
          app.kubernetes.io/part-of: prometheus-node-exporter
          app.kubernetes.io/name: prometheus-node-exporter
          app.kubernetes.io/instance: prometheus-node-exporter
-         app.kubernetes.io/version: "1.5.0"
+         app.kubernetes.io/version: "1.6.0"
      spec:
        automountServiceAccountToken: false
        securityContext:
          fsGroup: 65534
          runAsGroup: 65534
          runAsNonRoot: true
          runAsUser: 65534
        serviceAccountName: prometheus-node-exporter
        containers:
          - name: node-exporter
-           image: quay.io/prometheus/node-exporter:v1.5.0
+           image: quay.io/prometheus/node-exporter:v1.6.0
            imagePullPolicy: IfNotPresent
            args:
              - --path.procfs=/host/proc
              - --path.sysfs=/host/sys
              - --path.rootfs=/host/root
              - --path.udev.data=/host/root/run/udev/data
              - --web.listen-address=[$(HOST_IP)]:9100
            securityContext:
              readOnlyRootFilesystem: true
            env:
              - name: HOST_IP
                value: 0.0.0.0
            ports:
              - name: metrics
                containerPort: 9100
                protocol: TCP
            livenessProbe:
              failureThreshold: 3
              httpGet:
                httpHeaders:
                path: /
                port: 9100
                scheme: HTTP
              initialDelaySeconds: 0
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            readinessProbe:
              failureThreshold: 3
              httpGet:
                httpHeaders:
                path: /
                port: 9100
                scheme: HTTP
              initialDelaySeconds: 0
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: proc
                mountPath: /host/proc
                readOnly:  true
              - name: sys
                mountPath: /host/sys
                readOnly: true
              - name: root
                mountPath: /host/root
                mountPropagation: HostToContainer
                readOnly: true
        hostNetwork: true
        hostPID: true
+       nodeSelector:
+         kubernetes.io/os: linux
        tolerations:
          - effect: NoSchedule
            operator: Exists
        volumes:
          - name: proc
            hostPath:
              path: /proc
          - name: sys
            hostPath:
              path: /sys
          - name: root
            hostPath:
              path: /
default, prometheus-node-exporter, Service (v1) has changed:
  # Source: prometheus-node-exporter/templates/service.yaml
  apiVersion: v1
  kind: Service
  metadata:
    name: prometheus-node-exporter
    namespace: default
    labels:
-     helm.sh/chart: prometheus-node-exporter-4.17.0
+     helm.sh/chart: prometheus-node-exporter-4.18.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: metrics
      app.kubernetes.io/part-of: prometheus-node-exporter
      app.kubernetes.io/name: prometheus-node-exporter
      app.kubernetes.io/instance: prometheus-node-exporter
-     app.kubernetes.io/version: "1.5.0"
+     app.kubernetes.io/version: "1.6.0"
    annotations:
+     prometheus.io/scrape: "true"
      prometheus.io/scrape: "true"
  spec:
    type: ClusterIP
    ports:
      - port: 9100
        targetPort: 9100
        protocol: TCP
        name: metrics
    selector:
      app.kubernetes.io/name: prometheus-node-exporter
      app.kubernetes.io/instance: prometheus-node-exporter
default, prometheus-node-exporter, ServiceAccount (v1) has changed:
  # Source: prometheus-node-exporter/templates/serviceaccount.yaml
  apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: prometheus-node-exporter
    namespace: default
    labels:
-     helm.sh/chart: prometheus-node-exporter-4.17.0
+     helm.sh/chart: prometheus-node-exporter-4.18.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: metrics
      app.kubernetes.io/part-of: prometheus-node-exporter
      app.kubernetes.io/name: prometheus-node-exporter
      app.kubernetes.io/instance: prometheus-node-exporter
-     app.kubernetes.io/version: "1.5.0"
+     app.kubernetes.io/version: "1.6.0"

helm diff upgrade prometheus-node-exporter prometheus-community/prometheus-node-exporter --version 4.18.0 --suppress-output-line-regex "helm.sh/chart" --suppress-output-line-regex "version"

default, prometheus-node-exporter, DaemonSet (apps) has changed:
  # Source: prometheus-node-exporter/templates/daemonset.yaml
  apiVersion: apps/v1
  kind: DaemonSet
  metadata:
    name: prometheus-node-exporter
    namespace: default
    labels:
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: metrics
      app.kubernetes.io/part-of: prometheus-node-exporter
      app.kubernetes.io/name: prometheus-node-exporter
      app.kubernetes.io/instance: prometheus-node-exporter
  spec:
    selector:
      matchLabels:
        app.kubernetes.io/name: prometheus-node-exporter
        app.kubernetes.io/instance: prometheus-node-exporter
    updateStrategy:
      rollingUpdate:
        maxUnavailable: 1
      type: RollingUpdate
    template:
      metadata:
        annotations:
          cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
        labels:
          app.kubernetes.io/managed-by: Helm
          app.kubernetes.io/component: metrics
          app.kubernetes.io/part-of: prometheus-node-exporter
          app.kubernetes.io/name: prometheus-node-exporter
          app.kubernetes.io/instance: prometheus-node-exporter
      spec:
        automountServiceAccountToken: false
        securityContext:
          fsGroup: 65534
          runAsGroup: 65534
          runAsNonRoot: true
          runAsUser: 65534
        serviceAccountName: prometheus-node-exporter
        containers:
          - name: node-exporter
-           image: quay.io/prometheus/node-exporter:v1.5.0
+           image: quay.io/prometheus/node-exporter:v1.6.0
            imagePullPolicy: IfNotPresent
            args:
              - --path.procfs=/host/proc
              - --path.sysfs=/host/sys
              - --path.rootfs=/host/root
              - --path.udev.data=/host/root/run/udev/data
              - --web.listen-address=[$(HOST_IP)]:9100
            securityContext:
              readOnlyRootFilesystem: true
            env:
              - name: HOST_IP
                value: 0.0.0.0
            ports:
              - name: metrics
                containerPort: 9100
                protocol: TCP
            livenessProbe:
              failureThreshold: 3
              httpGet:
                httpHeaders:
                path: /
                port: 9100
                scheme: HTTP
              initialDelaySeconds: 0
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            readinessProbe:
              failureThreshold: 3
              httpGet:
                httpHeaders:
                path: /
                port: 9100
                scheme: HTTP
              initialDelaySeconds: 0
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: proc
                mountPath: /host/proc
                readOnly:  true
              - name: sys
                mountPath: /host/sys
                readOnly: true
              - name: root
                mountPath: /host/root
                mountPropagation: HostToContainer
                readOnly: true
        hostNetwork: true
        hostPID: true
+       nodeSelector:
+         kubernetes.io/os: linux
        tolerations:
          - effect: NoSchedule
            operator: Exists
        volumes:
          - name: proc
            hostPath:
              path: /proc
          - name: sys
            hostPath:
              path: /sys
          - name: root
            hostPath:
              path: /
default, prometheus-node-exporter, Service (v1) has changed:
  # Source: prometheus-node-exporter/templates/service.yaml
  apiVersion: v1
  kind: Service
  metadata:
    name: prometheus-node-exporter
    namespace: default
    labels:
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: metrics
      app.kubernetes.io/part-of: prometheus-node-exporter
      app.kubernetes.io/name: prometheus-node-exporter
      app.kubernetes.io/instance: prometheus-node-exporter
    annotations:
+     prometheus.io/scrape: "true"
      prometheus.io/scrape: "true"
  spec:
    type: ClusterIP
    ports:
      - port: 9100
        targetPort: 9100
        protocol: TCP
        name: metrics
    selector:
      app.kubernetes.io/name: prometheus-node-exporter
      app.kubernetes.io/instance: prometheus-node-exporter
default, prometheus-node-exporter, ServiceAccount (v1) has changed:

Signed-off-by: Jan-Otto Kröpke <[email protected]>

jkroepke · 2023-07-05T14:14:54Z

@yxxhero @databus23 Do you have the time to look into it?

jkroepke · 2023-07-17T20:45:40Z

@mumoshu @yxxhero @databus23 Please let me know, if I can assists here.

jkroepke · 2023-07-24T21:51:23Z

@mumoshu @yxxhero @databus23 I would appreciate a review here.

diff/diff.go

Co-authored-by: Yusuke Kuoka <[email protected]>

Signed-off-by: Jan-Otto Kröpke <[email protected]>

jkroepke · 2023-08-04T08:22:44Z

@mumoshu @yxxhero @databus23 I would appreciate a additional review here.

jkroepke · 2023-08-20T19:15:50Z

Hi @mumoshu @yxxhero @databus23

I would appreciate a additional review here.

mumoshu

LGTM. Thanks a lot for your patience and contribution @jkroepke!!

jkroepke · 2023-08-21T09:46:05Z

Thanks a lot! @mumoshu do you plan a release wich include this change?

mumoshu · 2023-08-21T09:52:43Z

@jkroepke Indeed! For transparency, the last thing we need before cutting the next release is to modify another feature we merged recently #458 somehow to make the dry-run=server an optional feature. That's to address @dudicoco's great insight shared in #449 (comment)

jkroepke · 2023-09-27T13:29:51Z

@yxxhero @mumoshu When do you plan the next release?

timmilesdw · 2023-11-27T11:24:51Z

when this feature will be released approximately? desperately waiting for it

0xStarcat · 2023-11-30T03:49:54Z

I too will sing your praises when this is released

dudicoco · 2024-04-18T09:40:54Z

Hi @jkroepke.

Can you please document the feature within the readme?
It's unclear if it can suppress multiple lines, for example a regex to find from ports to selector would suppress the entire ports block? Or can it only be used for single lines?

jkroepke · 2024-04-18T10:04:30Z

Hi @dudicoco

in our setup, we are using this:

--suppress-output-line-regex="chart: [0-9]" \
--suppress-output-line-regex="app.kubernetes.io/version: [0-9]"

it omits single lines.

About multiple lines and regex, you have to be a regex pro, maybe this can work for you: https://regex101.com/r/OHEFVb/1

dudicoco · 2024-04-18T12:42:59Z

Thanks @jkroepke.

The example you have provided did not work, I have even tried a more simple example which just captures the first two lines of the ports block and it also didn't work: (?m)ports:\s*-\s.*.

So is the issue with the regex or just the fact that the new feature does not support multi lines regex?

jkroepke · 2024-04-18T15:55:21Z

You are right, multi line is not supported.

Reason: The underlaying library generated a line-by-line diff and the regex will be matched to each line. Technically, multi line regex is supported, but since its a line-by-line diff, it a regex across multiple lines will never work.

The chances to support multiple are supper low because helm-diff have to merge the single lines first which would result into a too complex situation.

dudicoco · 2024-04-18T17:29:17Z

thanks for info @jkroepke

rbq · 2024-05-29T09:20:08Z

Does this affect only visible output or status codes/api results as well? I'm asking because Helmfile uses helm-diff under the hood to determine if a release is outdated and some Charts will always be re-deployed due to suboptimal design choices (e.g. random db passwords with no way to configure them via values).

jkroepke · 2024-05-29T09:50:23Z

only visible output

and some Charts will always be re-deployed due to suboptimal design choices (e.g. random db passwords with no way to configure them via values).

but maybe HELM_DIFF_USE_INSECURE_SERVER_SIDE_DRY_RUN=true can help to avoid that

jkroepke force-pushed the ignore-diff branch 2 times, most recently from e4d73c1 to e1d52fc Compare June 28, 2023 21:27

Suppress diff output by regex

c6c1502

Signed-off-by: Jan-Otto Kröpke <[email protected]>

jkroepke force-pushed the ignore-diff branch from e1d52fc to c6c1502 Compare June 28, 2023 21:39

jkroepke added 2 commits June 28, 2023 23:45

Added unit tests

445d42c

Signed-off-by: Jan-Otto Kröpke <[email protected]>

Keep changed filename in output

9cabf06

Signed-off-by: Jan-Otto Kröpke <[email protected]>

jkroepke marked this pull request as ready for review June 28, 2023 21:57

jkroepke changed the title ~~feat: Allow suppressing diff output by regex~~ feat: Allow suppress diff line output by regex Jun 28, 2023

mumoshu reviewed Jul 25, 2023

View reviewed changes

diff/diff.go Outdated Show resolved Hide resolved

mumoshu reviewed Jul 25, 2023

View reviewed changes

diff/diff.go Outdated Show resolved Hide resolved

jkroepke and others added 4 commits July 25, 2023 20:00

Update diff/diff.go

cfbd360

Co-authored-by: Yusuke Kuoka <[email protected]>

extract method

aa009a7

Signed-off-by: Jan-Otto Kröpke <[email protected]>

hide filteredReport

6952cae

Signed-off-by: Jan-Otto Kröpke <[email protected]>

skip doSuppress, if report is empty

3a0aa1e

Signed-off-by: Jan-Otto Kröpke <[email protected]>

jkroepke requested a review from mumoshu July 25, 2023 18:32

Add unit tests for DoSuppress

d9c642a

mumoshu approved these changes Aug 21, 2023

View reviewed changes

mumoshu merged commit 94d90a5 into databus23:master Aug 21, 2023

jkroepke deleted the ignore-diff branch August 21, 2023 09:45

z0rc mentioned this pull request Feb 6, 2024

Chart/version label noise on each chart upgrade #392

Closed

This was referenced May 25, 2024

Secret: kubectl.kubernetes.io/last-applied-configuration not redacted #390

Closed

feat: Allow suppressing annotations and labels #468

Closed

Option to ignore specific keys #406

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Allow suppress diff line output by regex #475

feat: Allow suppress diff line output by regex #475

jkroepke commented Jun 28, 2023 •

edited

Loading

jkroepke commented Jul 5, 2023

jkroepke commented Jul 17, 2023 •

edited

Loading

jkroepke commented Jul 24, 2023

jkroepke commented Aug 4, 2023

jkroepke commented Aug 20, 2023

mumoshu left a comment

jkroepke commented Aug 21, 2023

mumoshu commented Aug 21, 2023

jkroepke commented Sep 27, 2023

timmilesdw commented Nov 27, 2023

0xStarcat commented Nov 30, 2023

dudicoco commented Apr 18, 2024

jkroepke commented Apr 18, 2024 •

edited

Loading

dudicoco commented Apr 18, 2024

jkroepke commented Apr 18, 2024

dudicoco commented Apr 18, 2024

rbq commented May 29, 2024

jkroepke commented May 29, 2024 •

edited

Loading

feat: Allow suppress diff line output by regex #475

feat: Allow suppress diff line output by regex #475

Conversation

jkroepke commented Jun 28, 2023 • edited Loading

jkroepke commented Jul 5, 2023

jkroepke commented Jul 17, 2023 • edited Loading

jkroepke commented Jul 24, 2023

jkroepke commented Aug 4, 2023

jkroepke commented Aug 20, 2023

mumoshu left a comment

Choose a reason for hiding this comment

jkroepke commented Aug 21, 2023

mumoshu commented Aug 21, 2023

jkroepke commented Sep 27, 2023

timmilesdw commented Nov 27, 2023

0xStarcat commented Nov 30, 2023

dudicoco commented Apr 18, 2024

jkroepke commented Apr 18, 2024 • edited Loading

dudicoco commented Apr 18, 2024

jkroepke commented Apr 18, 2024

dudicoco commented Apr 18, 2024

rbq commented May 29, 2024

jkroepke commented May 29, 2024 • edited Loading

jkroepke commented Jun 28, 2023 •

edited

Loading

jkroepke commented Jul 17, 2023 •

edited

Loading

jkroepke commented Apr 18, 2024 •

edited

Loading

jkroepke commented May 29, 2024 •

edited

Loading