apps wc: Fixed network policy to always allow ingress probe

[Xartos]

Warning

This is a public repository, ensure not to disclose:

• personal data beyond what is necessary for interacting with this pull request, nor
• business confidential information, such as customer names.

What kind of PR is this?

Required: Mark one of the following that is applicable:

• kind/feature
• kind/improvement
• kind/deprecation
• kind/documentation
• kind/clean-up
• kind/bug
• kind/other

Optional: Mark one or more of the following that are applicable:

Important

Breaking changes should be marked kind/admin-change or kind/dev-change depending on type
Critical security fixes should be marked with kind/security

• kind/admin-change
• kind/dev-change
• kind/security
• kind/adr

What does this PR do / why do we need this PR?

...

• Fixes #

Information to reviewers

On clusters that didn't use host port for nginx, the ingress to nginx health didn't always work (If the request was bounced between nodes). This fixes that.

Checklist

• Proper commit message prefix on all commits
• Change checks:
  • The change is transparent
  • The change is disruptive
  • The change requires no migration steps
  • The change requires migration steps
  • The change upgrades CRDs
  • The change updates the config and the schema
• Documentation checks:
  • The public documentation required no updates
  • The public documentation required an update - [link to change](set-me)
• Metrics checks:
  • The metrics are still exposed and present in Grafana after the change
  • The metrics names didn't change (Grafana dashboards and Prometheus alerts are not affected)
  • The metrics names did change (Grafana dashboards and Prometheus alerts were fixed)
• Logs checks:
  • The logs do not show any errors after the change
• Pod Security Policy checks:
  • Any changed pod is covered by Pod Security Admission
  • Any changed pod is covered by Gatekeeper Pod Security Policies
  • The change does not cause any pods to be blocked by Pod Security Admission or Policies
• Network Policy checks:
  • Any changed pod is covered by Network Policies
  • The change does not cause any dropped packets in the NetworkPolicy Dashboard
• Audit checks:
  • The change does not cause any unnecessary Kubernetes audit events
  • The change requires changes to Kubernetes audit policy
• Falco checks:
  • The change does not cause any alerts to be generated by Falco
• Bug checks:
  • The bug fix is covered by regression tests

[simonklb]

Could you explain why this is only needed for the workload cluster? I would have assumed that the ingress healthcheck is done the same way for both clusters?

[Zash]

Is this related to the sc→wc probes?

[Xartos]

Could you explain why this is only needed for the workload cluster? I would have assumed that the ingress healthcheck is done the same way for both clusters?

As @Zash mentions, this is for the sc -> wc probes. So when we use the "new" ingress-nignx.<base-domain> the request will be reverse-proxied and sent to the ingress-nginx service and will be LBd between the endpoints. So from networkpolicys point of view it's sent from the controller pod to itself more or less

[simonklb]

Talked offline. I was worried that this would make us miss unnecessary extra hops inside the cluster for ingress traffic. This actually seem to already be the case because we do not make use externalTrafficPolicy: "Local" which we likely should be able to (since the external loadbalancers should only target nodes that runs the ingress-nginx controller).

I'm fine with merging this now but I'd like this to be removed if we are able to use externalTrafficPolicy: "Local" instead to prevent in-cluster routing for ingress traffic. Maybe add a TODO comment to re-evaluate this?

[Xartos]

@simonklb Is this fine 38ab1e1 ? Then we support both