Best practices for managing resources that are not managed by Kubernetes cluster

[j-zimnowoda]

The applications from Kubernetes cluster may require resources that are not managed by the cluster.
In most cases these are: object storage and KMS.

I am constantly asking myself the following questions:

1. what are the consequences of having applications in managed cluster of one cloud provider, depending on resources from the other cloud provider?
2. what is the proper way of managing credentials to that resources?
3. how to manage access control to that resources?
4. how to protect users from configuration hell?

[j-zimnowoda]

Ad1)
If resources are provided by different cloud provider then it is not possible to leverage Service Account annotations
Ad2)
The most clean solution is by leveraging Service Account annotations. Unfortunately that's not the case for AKS and OnPrem clusters.
Shall we then use it wherever it is possible and fallback to storing credentials in secrets otherwise?
Ad3)
It is possible to generate ServiceAccount with a token that is able to access all kinds of resources. But is it wise?
Would we like to harbor having credentials to manage vault object storage?
Ad4)
Technically it is possible to have one object storage in Google the other in AWS etc.
I am afraid that this freedom may turn out against the user, who will end up managing resource in three different cloud providers, while having one k8s cluster. Should I care about it?

[Morriz]

I wouldn't spend too much time on this. I personally don't have an issue with the way we use secrets now. After I set up my repos I never have to touch anything anymore and all works fine. I would just add one liner examples in the docs that show how to login with cloud clis and where to find the secrets files for the repos.

[j-zimnowoda]

I closing this conversation, because lack of interest. Maybe we can come back to this topic when customer triggers that issue.