Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix deprecated Grafana dashboards #2242

Merged

Conversation

anders-elastisys
Copy link
Contributor

@anders-elastisys anders-elastisys commented Aug 21, 2024

Warning

This is a public repository, ensure not to disclose:

  • personal data beyond what is necessary for interacting with this pull request, nor
  • business confidential information, such as customer names.

What kind of PR is this?

Required: Mark one of the following that is applicable:

  • kind/feature
  • kind/improvement
  • kind/deprecation
  • kind/documentation
  • kind/clean-up
  • kind/bug
  • kind/other

Optional: Mark one or more of the following that are applicable:

Important

Breaking changes should be marked kind/admin-change or kind/dev-change depending on type
Critical security fixes should be marked with kind/security

  • kind/admin-change
  • kind/dev-change
  • kind/security
  • kind/adr

What does this PR do / why do we need this PR?

This PR updates Grafana Dashboards containing deprecated panels.

node-mixin was uncommented from the grafana-dashboards makefile and adjusted and seems to be working fine now, but the latest upstream dashboards still included deprecated panels and were fixed manually. There exists an issue regarding this: prometheus/node_exporter#3046. Also, running the make script for node-mixin pulled down two new dashboards, one which was exclusive for darwin/macOS which I removed manually as I am not sure how to configure jsonnet to only include certain files. The other dashboard was Node Exporter / USE Method / Multi-cluster which I kept in.

alertmanager-mixin did not get upgraded with make script, and can't see issues fixing the panels in the repo https://github.com/grafana/prometheus-alertmanager.

prometheus-mixin did not get upgraded with make script. There exists an issue regarding fixing deprecated panels: prometheus/prometheus#14404

thanos-mixin did not get upgraded with make script. Issue regarding fixing deprecated panels: thanos-io/thanos#7413

Also noticed that the fluentd dashboard was missing metrics used in some panels, see this commit for newly added config to get this metric.

Information to reviewers

Not sure if the migration script will be necessary, planning to upgrade Grafana to v11 once this gets merged, which will probably make the script obsolete.

I manually checked individual panels when migrating them that they looked close to the original, some panels needed more adjustments than others. Noticed we have some panels with metrics (besides the fluentd one I added) that I did at least not have in my dev environment, might be worth looking into. Might have missed something as there were a lot of dashboards and panels to go through. 😅

Tested running detect-angular-dashboards after having migrated and it did not detect any deprecated panels.

Checklist

  • Proper commit message prefix on all commits
  • Change checks:
    • The change is transparent
    • The change is disruptive
    • The change requires no migration steps
    • The change requires migration steps
    • The change upgrades CRDs
    • The change updates the config and the schema
  • Metrics checks:
    • The metrics are still exposed and present in Grafana after the change
    • The metrics names didn't change (Grafana dashboards and Prometheus alerts are not affected)
    • The metrics names did change (Grafana dashboards and Prometheus alerts were fixed)
  • Logs checks:
    • The logs do not show any errors after the change
  • Pod Security Policy checks:
    • Any changed pod is covered by Pod Security Admission
    • Any changed pod is covered by Gatekeeper Pod Security Policies
    • The change does not cause any pods to be blocked by Pod Security Admission or Policies
  • Network Policy checks:
    • Any changed pod is covered by Network Policies
    • The change does not cause any dropped packets in the NetworkPolicy Dashboard
  • Audit checks:
    • The change does not cause any unnecessary Kubernetes audit events
    • The change requires changes to Kubernetes audit policy
  • Falco checks:
    • The change does not cause any alerts to be generated by Falco
  • Bug checks:
    • The bug fix is covered by regression tests

Copy link
Contributor

@robinelastisys robinelastisys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like its going in the right direction! Will be awesome to get this fixed!

Only thing im slightly worried about is this:

node-mixin was uncommented from the grafana-dashboards makefile and adjusted and seems to be working fine now, but the latest upstream dashboards still included deprecated panels and were fixed manually

Not sure if thats the way to go but i let others have opinions on that!

@anders-elastisys anders-elastisys force-pushed the anders-elastisys/fix-deprecated-grafana-dashboards branch from b25a5af to 1df2c92 Compare August 26, 2024 14:28
@anders-elastisys anders-elastisys force-pushed the anders-elastisys/fix-deprecated-grafana-dashboards branch from 1df2c92 to 4c6c369 Compare September 4, 2024 14:00
@anders-elastisys anders-elastisys merged commit 5e1ce52 into main Sep 4, 2024
12 checks passed
@anders-elastisys anders-elastisys deleted the anders-elastisys/fix-deprecated-grafana-dashboards branch September 4, 2024 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[1] Upgrade Grafana Dashboards with deprecated Angular plugins
5 participants