Non-existing statefulsets are being deleted over and over

[wonko]

Report

I noticed that the percona mongodb operator fills my kube API logs with deletes of non-existant stateful sets every couple of seconds. This clutters the logs and spills resources on the API server.

In my case, where I run 2 mongos and an arbiter, I get a delete for sts's:

• clustername-cfg
• clustername-mongos
• clustername-rs0-nv

every 6 seconds. See the below screenshot.

More about the problem

Checking the codepaths, I see that it might be intended behaviour (the delete is done, and a k8serror.IsNotFound(err) is actually in place to ignore this error, see https://github.com/percona/percona-server-mongodb-operator/blob/main/pkg/controller/perconaservermongodb/psmdb_controller.go#L995 for one example).

This seems wrong, I'd assume that after a single attempt, the logic can be stopped to try and delete a non-existing object.

Steps to reproduce

1. deploy the example crd, just adjust to have 1 arbiter and 2 mongos.
2. make sure to disable the sharding
3. observe the kube API logs once everything is deployed

Versions

1. Kubernetes - GKE 1.31
2. Operator - latest
3. Database - latest

Anything else?

No response