[Helm] Support deploying a Fleet Server Elastic Agent

[eedugon]

We should support the installation of a Fleet Server on Kubernetes with the helm chart.
Deploying a Fleet Server agent is slightly different than deploying a normal Fleet Managed Elastic Agent.

It requires:

• FLEET_SERVER_ENABLE set to true.
• A policy ID of the policy prepared in Kibana including Fleet Server integration, for FLEET_SERVER_POLICY_ID env var.
• A Service Token (and not an enrollment token), for FLEET_SERVER_SERVICE_TOKEN
• Elasticsearch URL, for FLEET_SERVER_ELASTICSEARCH_HOST
• The CA certificate of Elasticsearch, as a file pointed by ELASTICSEARCH_CA environment variable, or the SHA fingerprint of the CA through FLEET_SERVER_ELASTICSEARCH_CA_TRUSTED_FINGERPRINT env var (both options are in theory possible).
• A certificate and key for the HTTP endpoint (if not provided the Fleet Server generates a self-signed cert, which is called quick start mode in the documentation, but we should clearly support providing the cert and key as it's the requirement for production).

^^ From the previous items we don't expect the helm chart to take care of everything, probably the Policy ID and Service Token should be prepared in advance by the user and provided through values.yaml. Same for the Elasticsearch URL and the CA cert of Elasticsearch.

This type of Agent would NOT require any interaction with Kubernetes API, so I would suggest to NOT add any RBAC resources (probably neither a serviceAccount) --> automountServiceAccountToken: false should be the way to go here.

We have recently added a doc about installing Fleet Server on Kubernetes (without the chart of course :) ) that could be used as a reference of the needed resources (a Deployment + 2 secrets would be enough).

About the certificate + key we need to decide how to implement it, as there are multiple options:

• Request the user to provide the cert + key for the fleet server in a separate secret (this is probably the easiest).
• Offer some kind of automation or integration with something like cert manager.

Note that the previous item about cert + key will also appear as soon as we implement the support of mutual TLS authentication for the normal Elastic Agents.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[swiatekm]

Does this need to be part of the agent Helm Chart? I'm concerned about increasing its scope to the point where it becomes unmaintainable. Intuitively, fleet-server should have its own K8s deployment mechanism.

[nimarezainia]

I had similar concerns about using Helm here. At best when a customer is on-prem, they may deploy a single container configured as fleet server. The value of Helm is for the larger scale, repetitive provisioning.

[eedugon]

Does this need to be part of the agent Helm Chart?

That's a good question, @swiatekm , I don't know if the helm chart needs to support Fleet Server, but what is a fact is that we have users and customers who try to deploy the fleet server on Kubernetes without any success, because we don't have that documented at all.

@nimarezainia , yes, it's as simple as deploying a single container configured as fleet server, but it's not that straight forward as it looks, because you need a service token, to have the policy prepared, to know the CA certificate of the Elasticsearch server, and some other things. Not complex or difficult but not that easy either. Users, customers and Elastic Support engineers struggle with this.

Please take a look at this PR where we are creating a new document to solve this use case without helm, with a static manifest: elastic/ingest-docs#1518

That PR has this doc preview if you want to take a look: https://ingest-docs_bk_1518.docs-preview.app.elstc.co/guide/en/fleet/master/add-fleet-server-kubernetes.html

As you can see in that doc, the deployment itself is not a big deal, and that's the reason I consider a good case of not big effort and high benefits for the helm chart. But it's ok if you believe it's not worthy to add it at this stage (or ever).

Sorry for asking an off-topic question here, but... do you also believe that document I've been working on (this one) shouldn't be published either?

I'd like to check if we are aligned at least on the need of supporting a way to run Fleet Server on Kubernetes.

[swiatekm]

@eedugon to me, that sounds like we need a separate Helm Chart for fleet-server itself. The reason I don't want the agent Chart to do this, is that it's a separate concern. You can easily have multiple agent installations report to the same fleet-server installation, for example.

As an end goal, I would envision installing fleet-server in K8s the following way:

1. Get credentials from Kibana
2. Install fleet-server using its Helm Chart, putting credentials in the values file
3. Install agent using the agent Helm Chart - we provide the configuration for agent o talk to the fleet-server installed in 2

Does that make sense?

[eedugon]

@swiatekm , thanks for your explanations. That would make all sense.

We can clearly close this enhancement and just create a new helm chart in the future if consider it needed.

In the meantime it would be great to get your feedback on the PR and doc we are currently creating to support the fleet server installation without helm. I'll ping you in private, as I'm pretty sure we could make the installation smoother by using the kibana setup instead of the manual approach that I've documented (creating the policy and obtaining a service token manually prior to the Fleet Server deployment).

Feel free to close this and thanks for the feedback!

[cmacknz]

We can clearly close this enhancement and just create a new helm chart in the future if consider it needed.

Let's create an implementation issue for a fleet-server helm chart so the need is documented somewhere, probably this should live in https://github.com/elastic/fleet-server

[ycombinator]

Let's create an implementation issue for a fleet-server helm chart so the need is documented somewhere, probably this should live in https://github.com/elastic/fleet-server

I've moved this issue here to https://github.com/elastic/fleet-server since it has the requirements and also the context on why we want a separate Fleet Server Helm Chart from the Agent one.