We believe that the best Enterprise Instant Messaging Solution is the one built specifically for your client. With MongooseIM, we offer the flexibility of a bespoke in-house built chat system with the ease of use, reliability and scalability of a battle-tested enterprise solution.
MongooseIM is a mobile messaging platform with a focus on extensibility, performance and scalability.
helm repo add mongoose https://esl.github.io/MongooseHelm/
helm install my-mongooseim mongoose/mongooseim --set replicaCount=1This chart will install a MongooseIM cluster composed of a single node (set replicaCount=n for more nodes), sitting behind a regular k8s load-balancer, with a default sane configuration.
To uninstall, simply run helm uninstall my-mongooseim, where my-mongooseim is the release name you've given to your installation.
Configuration can be done by providing parameters with --set Key=Value to the helm install command.
| Parameter | Description | Default |
|---|---|---|
nodeName |
Name of the nodes as containers | mongooseim |
nodeCookie |
Cookie to be used by the BEAM | mongooseim |
nodeSelector |
Node selector for the statefulset | not set |
replicaCount |
Default number of replicas to be clustered | 1 |
resources |
MongooseIM container cpu and memory resource requests and limits. If resources is defined, then requests must be present, while limits is optional. |
not set (unconstrained) |
loadBalancerIP |
Exposed external IP address for the Load Balancer, it exposes the XMPP TCP interface | not set (will be automatically assigned) |
loadBalancerAnnotations |
Additional annotations for the load balancer, e.g. to bind it to an AWS Elastic IP | not set |
nodeport.enabled |
Whether the k8s nodeport service is desired | false |
rolloutId |
Random value used to force rolling update for each execution of helm upgrade. If you want a rolling update to happen only if the manifests change, set it to a constant value. |
random string |
tlsCertSecret |
Kubernetes secret with the certificates referenced in mongooseim.toml. All files will be mounted in priv/ssl, replacing the default fake certificates, which should not be used in a production setup. |
not set |
certs.* |
File names for the certificate and private key used by the server. Use tlsSecretCert to provide the actual files. |
Fake certificates, see values.yaml |
volatileDatabase |
Database used to share in-memory data between cluster nodes. Set it to cets to use CETS instead of Mnesia. When doing so, you need to set persistentDatabase to rdbms for node discovery. |
mnesia |
persistentDatabase |
Database used for storing persistent data. Set it to rdbms to use an external relational database instead of Mnesia. |
mnesia |
rdbms.* |
Database options | See values.yaml |
mimConfig |
User-given mongooseim.toml configuration file |
Generated from template |
vmConfig |
User-given vm.args file (used for tweaking the Erlang VM itself) |
Generated from template |
appConfig |
User-given app.config configuration file |
not set |
We recommend the following way of configuring your setup:
- Configure the Kubernetes resources by with parameters like
nodeSelector,loadBalancerIPandloadBalancerAnnotations, and make sure MongooseIM is starting with the default configuration. - Configure MongooseIM, especially DB (
volatileDatabase,transientDatabase,rdbms.*) and TLS (tlsCertSecret,certs.*). These parameters affect themongooseim.tomlconfiguration file generated from a template. - If you still need more customization, use the generated
mongooseim.tomlas a starting point and provide your own version of it with themimConfigparameter. You can also tweak the Erlang VM itself withvmConfigor customize Erlang application config withappConfig. These files should be given using helm's--set-filedirective, as below:
helm install mim mongoose/mongooseim --set-file mimConfig=<path-to-mim-toml-config-file.toml>An alternative way of customization is to clone the chart repo, edit the configuration templates or values.yaml, and install the chart with helm install mim <REPO_DIR>/MongooseIM.
By default, MongooseIM is configured to use the Mnesia internal database for storing both in-memory and persistent data. This setup is prone to various Mnesia-related consistency and replication issues. Moreover, Mnesia requires persistent volumes, which consume your Kubernetes resources and need to be cleaned up manually after pod deletion.
To completely avoid Mnesia with its issues, use different databases instead with the following parameters:
- Set
persistentDatabase=rdbmsto use a relational database (e.g. PostgreSQL or MySQL) for persistent data. Connection details and credentials are provided with therdbms.*parameters. DB setup is described in the documentation. - Set
volatileDatabase=cetsto use CETS for in-memory data. CETS needs a relational database for node discovery (unless you manually change the chart to use the file backend), so using it withpersistentDatabase=rdbmsis both required and recommended.
Sometimes you want to use a non-default Docker image. It is possible by specifying image.tag value:
helm install "my-mongooseim" mongoose/mongooseim --set image.tag=PR-3796Be aware, that the APP VERSION in the output of the helm list would not be correct in this case.
When working with a Kubernetes cluster it's convenient to see the results of the actions taken. In order to do that, let's start a terminal window and run:
watch kubectl get node,pod,pv,pvc,sts,svcThis is going to be our monitoring window. We'll input commands in another terminal.
MongooseIM pods will automatically initiate communication between one another and form a cluster, but they require a DNS service for that. We'll use a Kubernetes headless service and StatefulSet stable network IDs to provide DNS for our cluster members, described in mongoose-svc.yaml
Assuming we've installed MongooseIM with a command as follows:
helm install mim mongoose/mongooseim --set replicaCount=3A mongooseim StatefulSet should appear in the monitoring window, followed by its pods. It will take a few minutes for all the pods to start and become ready to serve traffic:
NAME READY STATUS RESTARTS AGE
pod/mongooseim-0 1/1 Running 0 4m
pod/mongooseim-1 1/1 Running 0 2m
pod/mongooseim-2 1/1 Running 0 1m
NAME DESIRED CURRENT AGE
statefulset.apps/mongooseim 3 3 4mOnce all the pods are up we can check the cluster status:
$ kubectl exec mongooseim-0 -- /usr/lib/mongooseim/bin/mongooseimctl mnesia running_db_nodes
['[email protected]',
'[email protected]',
'[email protected]']The above command should return the same list of nodes (possibly in different order) no matter whether we exec it on mongooseim-0, mongooseim-1, or any other member of the cluster.
If we want to add or remove nodes, we can simply run the following command, where N is the new desired count:
helm upgrade mim mongoose/mongooseim --set replicaCount=NThere are two basic ways to expose the XMPP service to public internet. The first and easiest is using a public IP provided by your k8s cluster. The second, somewhat more involved and less convenient, but at the same time cheaper, is to use the public IPs of the Kubernetes worker nodes.
The most straightforward way to expose the service is via a load balancer of the cluster provider (this is the chosen default for this helm package). This Load Balancer is defined in mongoose-svc-lb.yaml.
It will take up to a few minutes to get an IP address from the provider's pool of public addresses, but once that's done, we should see it in the monitoring window:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.43.240.1 <none> 443/TCP 2h
service/mongooseim ClusterIP None <none> 4369/TCP,5222/TCP,5269/TCP,5280/TCP,9100/TCP 15m
service/mongooseim-lb LoadBalancer 10.43.247.53 35.204.210.47 5222:31810/TCP 1mThe service should now be reachable from any host with access to the internet (including our local machines). A simple test would be running telnet and pressing Ctrl-D to send the end of file symbol:
$ telnet 35.204.210.47 5222
Trying 35.204.210.47...
Connected to 47.210.204.35.bc.googleusercontent.com.
Escape character is '^]'.
<?xml version='1.0'?><stream:stream xmlns='jabber:client'...Success! Since we're connecting with telnet, not an XMPP client, the connection dies instantly, but we got the XMPP stream initiation element from the server - the service is up, running, and reachable from everywhere.
In case we don't want to use this access method to the service we can tear the load balancer down with:
kubectl delete svc mongooseim-lbThe LoadBalancer is the more convenient, but also the more expensive way of publishing a service. We can also access our MongooseIM instances by using Kubernetes worker nodes' external IP addresses.
In order to do that, we first have to create a NodePort service:
helm upgrade mim mongoose/mongooseim --set nodeport.enabled=trueAfter a while it should appear in our monitoring window:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.43.240.1 <none> 443/TCP 3h
service/mongooseim ClusterIP None <none> 4369/TCP,5222/TCP,5269/TCP,5280/TCP,9100/TCP 1h
service/mongooseim-nodeport NodePort 10.43.253.16 <none> 5222:32709/TCP 41sThe last line tells us that the service will be available on the port 32709 of Kubernetes workers' external interfaces. We can find their external addresses (for example) with:
$ kubectl describe nodes | grep ExternalIP
ExternalIP: 35.204.145.243
ExternalIP: 35.204.114.176
ExternalIP: 35.204.144.193However, unlike with LoadBalancer, using NodePort we have to manually open the relevant port (32709 in this case) in the firewall, For example for gcloud (see Creating a Service of type NodePort for the source of this command):
gcloud compute firewall-rules create mongooseim-c2s-nodeport --allow tcp:32709We can now check that the service is available on the public internet (e.g. our machine) in a similar fashion to testing the LoadBalancer setup (run telnet and immediately type Ctrl-D):
$ telnet EXTERNALIP 32709
Trying EXTERNALIP...
Connected to ...
Escape character is '^]'.
<?xml version='1.0'?><stream:stream xmlns='jabber:client'...Again, the opening <stream> element means we've successfully connected to our MongooseIM cluster.