KubernetesPodOperator is not supported currently -> needs kubeconfig

[kirajano]

As per current setup, the local credentials are not mounted and have to be somehow manually copied. The file ~/.kube/config is not mounted and hence inside the container you cannot interact with K8S clusters. Solution is needed to make K8s workloads testable in Airflow.

Opened PR #15 please feel free to check

[piotrwrotniak]

I think one can always use mounted GCP credentials to retrieve GKE cluster kubeconfig by invoking gcloud: https://cloud.google.com/sdk/gcloud/reference/container/clusters/get-credentials.

I'm not sure whether we want to expose any additional credentials stored locally to the running Composer Local Environment, let me get back to you on this.

[kirajano]

Thanks for you reply,
The suggestion makes sense if your run only GKE clusters but for others you have to then manually make additional setup. In addition, you already have your local config with aliases, preferences etc. already there. Fetching credentials from GCP would then be creating a new config from scratch - although this is really more about convenience.

Looking forward to your verdict.

[fpopic]

I think one can always use mounted GCP credentials to retrieve GKE cluster kubeconfig by invoking gcloud: https://cloud.google.com/sdk/gcloud/reference/container/clusters/get-credentials.

@piotrwrotniak it would be nice to either provide support to add custom docker volume mounts or support the airflow to K8s auth order as-is without the need for additional setup, the same way ADC works for gcloud auth without any intervention within the container.

[msgongora]

I think one can always use mounted GCP credentials to retrieve GKE cluster kubeconfig

@piotrwrotniak What if you use a local cluster through minikube, Docker Desktop, etc? Aren't we talking about local development? This is a legit issue.

[fpopic]

Hi @piotrwrotniak, any update or estimation when issues/PRs in this repository are getting some attention?

Otherwise forking will start very early, and it would be nicer if the OSS community is focused on one repository than in forks (esp. since the whole airflow/composer is relying on OSS)

[michalmodras]

The way forward we'll be supporting custom mounts - we'll be working on this (Cloud Composer).

[mbaltrusitis]

@michalmodras do you know when you can expect an update?

[ahidalgob]

See #15 (comment), it will be included in the next release