Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] native Kubernetes client #29

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

timothyb89
Copy link
Contributor

This adds a native Kubernetes client to replace the kubectl proxy subprocess. That is, a kubeconfig parser with some integration with reqwest to configure proxies and auth, or the rust equivalent of tiny-kubernetes. It only strictly needs serde_yaml, reqwest, and base64.

Ideally this should end up being much smaller than a 'proper' client and can easily support the few functions we actually care about (auth, proxies, streamed responses, websockets, exotic kubeconfig formats, ...).

This at least properly parses all the kubeconfigs I have handy, including minikube, microk8s, eks, kubeadm, and a few others.

Closes #28

This adds a WIP implementation of a lightweight Kubernetes client,
or in other words, a kubeconfig parser with some integration with
reqwest to configure auth. It only needs serde_yaml and reqwest.

Ideally this should end up being much smaller than a 'proper' client
and can easily support the few functions we actually care about
(auth, proxies, streamed responses, websockets, exotic kubeconfig
formats, ...).

This at least properly parses all the kubeconfigs I have handy,
including minikube, microk8s, eks, kubeadm, and a few others.
@timothyb89 timothyb89 added enhancement New feature or request wip work in progress labels Jul 19, 2019
 - bumps reqwest version for proxy support
 - mostly implements KubernetesClient::new()
 - adds notes about all the tls libs being broken :/

Signed-off-by: Tim Buckley <[email protected]>
@timothyb89
Copy link
Contributor Author

timothyb89 commented Jul 22, 2019

I can at least auth against DigitalOcean now (uses embedded client + CA certs). Minikube and some kubeadm clusters are broken since rustls doesn't support IP address hosts (rustls/hyper-rustls#56 (comment)). Also, native-tls (via reqwest, at least) doesn't support PEM certificates so we're stuck with rustls.

Should look into whether or not ClientBuilder::danger_accept_invalid_hostnames() is enough to at least make this functional, otherwise we'll need to keep kubectl proxy support around...

@timothyb89
Copy link
Contributor Author

Also, still need some in-cluster auth support...

@timothyb89
Copy link
Contributor Author

More TLS issues...

This is all to say, there is probably a path forward with native-tls, however it:

  • requires vendored openssl
    • this massively increases build times and binary size (adds ~1 MiB per cargo-bloat)
    • complicates Windows and macOS builds
  • still requires ugly hacks to even read certs

I think rustls + the kubectl proxy hack will continue to be the best solution, and hopefully we can drop the hack if/when rustls gets IP address support.

 - separate enum structs
 - fully implement exec auth
 - honor insecure-skip-tls-verify
 - implement reauthentication and expiry checks for exec auth
   - split `KubernetesClient::new()` and `KubernetesClient::from_context()`
@timothyb89
Copy link
Contributor Author

Additional ssl-related complications if we want websocket support:

  • neither rust-websocket nor ws-rs support rustls, so native-tls is required
  • use of client/server certs is undocumented in either library; may just mean configuring native-tls's TlsConnectorBuilder

So the only way forward if we want websockets looks like:

  • use native-tls and accept the increased static linux binary size
  • figure out some hack to convert pem client certs into der for native-tls / reqwest, preferably without vendoring openssl
  • implement separate auth codepaths for reqwest and websocket/ws

Frankly, I'm tempted to sit on this a while and see if the library situation improves.

@timothyb89 timothyb89 mentioned this pull request Aug 19, 2019
8 tasks
@timothyb89
Copy link
Contributor Author

Not a ton of movement over the last 8 months, unfortunately:

  • there's been a hint of movement on webpki so ip hostnames may be supported in the near future.
  • native-tls still doesn't support .pem certificates
  • no websocket libraries support rustls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request wip work in progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proper Kubernetes reader
1 participant