docs: infrastructure-as-code using CDKTF (#92)

Signed-off-by: Jared Weinfurtner <[email protected]> Signed-off-by: Sven Trieflinger <[email protected]> Co-authored-by: Sven Trieflinger <[email protected]>
carbynestack · Sep 1, 2023 · 0768b51 · 0768b51
1 parent f1f943d
commit 0768b51
Show file tree

Hide file tree

Showing 12 changed files with 264 additions and 87 deletions.
diff --git a/NOTICE.md b/NOTICE.md
@@ -13,3 +13,4 @@ file in the Carbyne Stack
 
 - Trieflinger Sven <[email protected]>
 - Veselin Vlasakiev <[email protected]>
+- Weinfurtner Jared <[email protected]>
diff --git a/config/nav.yml b/config/nav.yml
@@ -10,8 +10,16 @@ nav:
       - getting-started/index.md
       - Overview: getting-started/overview.md
       - Prerequisites: getting-started/prerequisites.md
-      - Platform Setup: getting-started/platform-setup.md
-      - Stack Deployment: getting-started/deployment.md
+      - Deployment:
+          - getting-started/deployment/index.md
+          - Manual:
+              - getting-started/deployment/manual/index.md
+              - Platform Setup: getting-started/deployment/manual/platform-setup.md
+              - Stack: getting-started/deployment/manual/stack.md
+          - Infrastructure as Code:
+              - getting-started/deployment/infrastructure-as-code/index.md
+              - Local: getting-started/deployment/infrastructure-as-code/local.md
+      - Configuring the CLI: getting-started/cli.md
       - Millionaires Problem: getting-started/millionaires.md
   - Blog: https://blog.carbynestack.io
   - Community:

diff --git a/docs/getting-started/cli.md b/docs/getting-started/cli.md
@@ -0,0 +1,80 @@
+# Configuring the CLI
+
+In this guide, you will learn how to download and configure the Carbyne Stack
+CLI that can be used to interact with a virtual cloud from the command line.
+
+1. Install the CLI using:
+
+    ```shell
+    export CLI_VERSION=0.4.1
+    curl -o cs.jar -L https://github.com/carbynestack/cli/releases/download/cli-v$CLI_VERSION/cli-$CLI_VERSION.jar
+    ```
+
+1. Export the IP addresses of the Istio *Ingress Gateways* from the
+   [deployment tutorial](../deployment):
+
+    ```shell
+    export APOLLO_FQDN="172.18.1.128.sslip.io"
+    export STARBUCK_FQDN="172.18.2.128.sslip.io"
+    ```
+
+1. Next, configure the CLI to talk to the virtual cloud you just deployed by
+   creating a matching CLI configuration file in `~/.cs` using:
+
+    ```shell
+    mkdir -p ~/.cs
+    cat <<EOF | envsubst > ~/.cs/config
+    {
+      "prime" : 198766463529478683931867765928436695041,
+      "r" : 141515903391459779531506841503331516415,
+      "noSslValidation" : true,
+      "trustedCertificates" : [ ],
+      "providers" : [ {
+        "amphoraServiceUrl" : "http://$APOLLO_FQDN/amphora",
+        "castorServiceUrl" : "http://$APOLLO_FQDN/castor",
+        "ephemeralServiceUrl" : "http://$APOLLO_FQDN/",
+        "id" : 1,
+        "baseUrl" : "http://$APOLLO_FQDN/"
+      }, {
+        "amphoraServiceUrl" : "http://$STARBUCK_FQDN/amphora",
+        "castorServiceUrl" : "http://$STARBUCK_FQDN/castor",
+        "ephemeralServiceUrl" : "http://$STARBUCK_FQDN/",
+        "id" : 2,
+        "baseUrl" : "http://$STARBUCK_FQDN/"
+      } ],
+      "rinv" : 133854242216446749056083838363708373830
+    }
+    EOF
+    ```
+
+    Alternatively, you can use the CLI tool itself to do the configuration by
+    providing the respective values (as seen above in the HEREDOC) when prompted
+    using:
+
+    ```shell
+    java -jar cs.jar configure
+    ```
+
+    You can verify that the configuration works by fetching telemetry data from
+    castor using:
+
+    !!! attention
+        Replace `<#>` with either `1` for the `apollo` cluster or `2` for the
+        `starbuck` cluster.
+
+    ```shell
+    java -jar cs.jar castor get-telemetry <#>
+    ```
+
+    !!! warning
+        Before actually using the Carbyne Stack Virtual Cloud please make sure
+        that enough correlated randomness has been generated by Klyshko in the
+        background. You can verify that this is the case by fetching telemetry
+        data via
+
+        ```shell
+        java -jar cs.jar castor get-telemetry 1
+        ```
+
+    For the next part of the tutorial, it is recommended to wait until
+    Klyshko has generated at least one batch of tuples for each tuple type.
diff --git a/docs/getting-started/deployment/index.md b/docs/getting-started/deployment/index.md
@@ -0,0 +1,13 @@
+# Deployment
+
+This guide describes how to deploy a two-party Carbyne Stack Virtual Cloud on
+your local machine using [kind](https://kind.sigs.k8s.io/) clusters.
+
+We support two distinct deployment methods:
+
+1. [Manual deployment](manual) using mainly `kubectl` and the shell is presently
+   the most instructive and thoroughly tested method for deploying Carbyne
+   Stack.
+2. [Automated deployment](infrastructure-as-code) using Terraform CDKTF is a
+   very convenient way to deploy Carbyne Stack, but the implementation
+   is still early stage and should be considered experimental.
diff --git a/docs/getting-started/deployment/infrastructure-as-code/index.md b/docs/getting-started/deployment/infrastructure-as-code/index.md
@@ -0,0 +1,69 @@
+# Infrastructure as Code
+
+!!! warning
+    Carbyne Stack Infrastructure as Code (IaC) is still in *proof-of-concept*
+    stage. Reach out in case you encounter problems.
+
+Carbyne Stack has adopted Infrastructure as Code (IaC) as a core principle.
+IaC is the process of managing and provisioning infrastructure through code
+instead of manually deploying resources via kubectl, helm, etc. This allows
+for the infrastructure to be versioned, tested, and deployed in a repeatable
+manner.
+
+## Terraform
+
+Carbyne Stack uses [Terraform by Hashicorp](https://www.terraform.io/) to
+provision infrastructure and deploy Carbyne Stack on top of it. Terraform
+has a large community and is widely used in the industry, with support for
+most major cloud providers through a large set
+of [providers](https://registry.terraform.io/browse/providers?product_intent=terraform).
+
+## Cloud Development Kit for Terraform (CDKTF)
+
+Terraform uses a custom syntax for their configuration language called
+*HashiCorp Configuration Language* (HCL). While this is great for simple use
+cases, it can become difficult to manage as the complexity of the
+infrastructure grows. To address this, Carbyne Stack uses
+[CDKTF](https://learn.hashicorp.com/tutorials/terraform/cdktf), which
+allows you to use a common programming language, in Carbyne Stack's
+case, Typescript, to define the infrastructure. This allows you to
+leverage the full power of a programming language to define your
+infrastructure, including the ability to use loops, conditionals,
+functions, and abstractions.  
+
+### Stacks
+
+In CDKTF,
+[stacks](https://developer.hashicorp.com/terraform/cdktf/concepts/stacks)
+represent a collection of resources that are deployed together. Carbyne Stack
+uses a separate stack for each deployment target with each stack being
+responsible for provisioning the infrastructure and deploying Carbyne Stack on
+top of it.
+
+As of today, the following deployment targets are supported:
+
+- [Local deployment](./local) to [kind](https://kind.sigs.k8s.io/) clusters
+
+### Constructs
+
+In CDKTF,
+[constructs](https://developer.hashicorp.com/terraform/cdktf/concepts/constructs)
+are the building blocks of the Carbyne Stack infrastructure. Constructs are used
+to define resources, modules, and data sources. Carbyne Stack uses constructs
+to define the resources that make up the infrastructure, including the
+Kubernetes cluster, networking, and storage. Constructs are also used to
+define the Carbyne Stack Helm chart, which is used to deploy Carbyne Stack on
+top of the infrastructure.
+
+## Prerequisites
+
+Before you proceed, make sure you meet the following requirements:
+
+- [Node.js](https://nodejs.org/en/download) v18.17.1
+- [Terraform CLI](https://developer.hashicorp.com/terraform/downloads) v1.5.5
+- [CDKTF CLI](https://developer.hashicorp.com/terraform/tutorials/cdktf/cdktf-install)
+  v0.18.0
+- [Docker](https://docs.docker.com/engine/install/ubuntu/) v23.0.1
+- [Kind](https://kind.sigs.k8s.io/) v0.17.0
+- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/) v1.26.1
+- [Helm](https://helm.sh/docs/intro/install/) v3.11.1
diff --git a/docs/getting-started/deployment/infrastructure-as-code/local.md b/docs/getting-started/deployment/infrastructure-as-code/local.md
@@ -0,0 +1,69 @@
+# Local Deployment using Infrastructure as Code
+
+This guide describes how to deploy a two-party Carbyne Stack Virtual Cloud
+onto local [kind](https://kind.sigs.k8s.io/) cluster using CDKTF.
+
+## Setup
+
+1. Clone the carbynestack/carbynestack repository
+
+    ```shell
+    git clone [email protected]:carbynestack/carbynestack.git
+    ```
+
+1. Change into the `carbynestack` directory
+
+    ```shell
+    cd carbynestack
+    ```
+
+1. Checkout the tag `sdk-v0.4.0`
+
+    ```shell
+    git checkout tags/sdk-v0.4.0
+    ```
+
+1. Change directory to the `deployments` folder
+
+    ```shell
+    cd deployments
+    ```
+
+1. Install npm dependencies:
+
+    ```shell
+    npm install
+    ```
+
+1. Generate CDKTF provider bindings and import modules (located in the `.gen` folder):
+
+    ```shell
+    cdktf get
+    ```
+
+## Deploy
+
+In the `./deployments` folder:
+
+1. Deploy the stack using
+
+    ```shell
+    cdktf deploy local-kind
+    ```
+
+## Destroy and Clean Up
+
+If you no longer need the stack or want to tear it down to apply changes to the
+infrastructure as code, run the following command:
+
+```bash
+cdktf destroy
+```
+
+Alternatively, you can use:
+
+```bash
+kind delete clusters cs-1 cs-2
+```
+
+And delete the CDKTF state files (like `terraform.local-kind.tfstate`) manually.
diff --git a/docs/getting-started/deployment/manual/index.md b/docs/getting-started/deployment/manual/index.md
@@ -0,0 +1,9 @@
+# Manual Deployment
+
+By following this guide, you will learn how to:
+
+1. [Set up Kubernetes clusters](platform-setup.md) with the prerequisites
+   for deploying a Carbyne Stack Virtual Cloud Provider including Istio
+   and Knative.
+2. [Deploy a fully functional two-party Carbyne Stack Virtual Cloud](stack.md)
+   to these clusters.
diff --git a/docs/getting-started/platform-setup.md → ...arted/deployment/manual/platform-setup.md b/docs/getting-started/platform-setup.md → ...arted/deployment/manual/platform-setup.md
@@ -42,7 +42,7 @@ metallb-system       metallb-speaker-vmqj4                            1/1     Ru
     specified below. Deviating from this _battle tested_ configuration may
     create all kinds of issues.
 
-    Do not forget to perform the [post installation steps](https://docs.docker.com/engine/install/linux-postinstall/ )
+    Do not forget to perform the [post installation steps](https://docs.docker.com/engine/install/linux-postinstall/)
     for Docker.
 
 !!! info
@@ -73,13 +73,13 @@ a kind cluster with K8s context name `kind-<name>`, as follows:
 === "Apollo"
 
     ```shell
-    kind create cluster --name apollo --image kindest/node:v1.26.0
+    kind create cluster --name apollo --image kindest/node:v1.26.6
     ```
 
 === "Starbuck"
 
     ```shell
-    kind create cluster --name starbuck --image kindest/node:v1.26.0
+    kind create cluster --name starbuck --image kindest/node:v1.26.6
     ```
 
 You can switch between the clusters easily using:

diff --git a/docs/getting-started/deployment.md → ...etting-started/deployment/manual/stack.md b/docs/getting-started/deployment.md → ...etting-started/deployment/manual/stack.md
@@ -53,14 +53,14 @@ clusters using the kind tool as described in the
 
         ```shell
         git clone https://github.com/carbynestack/carbynestack.git
-        cd carbynestack
+        cd carbynestack/deployments
         ```
 
     === "SSH"
 
         ```shell
         git clone [email protected]:carbynestack/carbynestack.git
-        cd carbynestack
+        cd carbynestack/deployments
         ```
 
 1. Checkout Carbyne Stack sdk version 0.2.0 using:
@@ -110,77 +110,6 @@ clusters using the kind tool as described in the
 
 You now have a fully functional Carbyne Stack Virtual Cloud at your hands.
 
-## Configuring the CLI
-
-1. Carbyne Stack comes with a CLI that can be used to interact with a virtual
-   cloud from the command line. Install the CLI using:
-
-    ```shell
-    export CLI_VERSION=0.4.1
-    curl -o cs.jar -L https://github.com/carbynestack/cli/releases/download/cli-v$CLI_VERSION/cli-$CLI_VERSION.jar
-    ```
-
-2. Next configure the CLI to talk to the just deployed virtual cloud by creating
-   a matching CLI configuration file in `~/.cs` using:
-
-    ```shell
-    mkdir -p ~/.cs
-    cat <<EOF | envsubst > ~/.cs/config
-    {
-      "prime" : 198766463529478683931867765928436695041,
-      "r" : 141515903391459779531506841503331516415,
-      "noSslValidation" : true,
-      "trustedCertificates" : [ ],
-      "providers" : [ {
-        "amphoraServiceUrl" : "http://$APOLLO_FQDN/amphora",
-        "castorServiceUrl" : "http://$APOLLO_FQDN/castor",
-        "ephemeralServiceUrl" : "http://$APOLLO_FQDN/",
-        "id" : 1,
-        "baseUrl" : "http://$APOLLO_FQDN/"
-      }, {
-        "amphoraServiceUrl" : "http://$STARBUCK_FQDN/amphora",
-        "castorServiceUrl" : "http://$STARBUCK_FQDN/castor",
-        "ephemeralServiceUrl" : "http://$STARBUCK_FQDN/",
-        "id" : 2,
-        "baseUrl" : "http://$STARBUCK_FQDN/"
-      } ],
-      "rinv" : 133854242216446749056083838363708373830
-    }
-    EOF
-    ```
-
-    Alternatively, you can use the CLI tool itself to do the configuration by
-    providing the respective values (as seen above in the HEREDOC) when prompted
-    using:
-
-    ```shell
-    java -jar cs.jar configure
-    ```
-
-    You can verify that the configuration works by fetching telemetry data from
-    castor using:
-
-    !!! attention
-        Replace `<#>` with either `1` for the `apollo` cluster or `2` for the
-        `starbuck` cluster.
-
-    ```shell
-    java -jar cs.jar castor get-telemetry <#>
-    ```
-
-    !!! warning
-        Before actually using the Carbyne Stack Virtual Cloud please make sure
-        that enough correlated randomness has been generated by Klyshko in the
-        background. You can verify that this is the case by fetching telemetry
-        data via
-
-        ```shell
-        java -jar cs.jar castor get-telemetry 1
-        ```
-
-        For the next part of the tutorial, it is sufficient to wait until
-        Klyshko has generated a single batch of tuples for each tuple type.
-
 ## Teardown the Virtual Cloud
 
 You can tear down the Virtual Cloud by tearing down the Virtual Cloud Providers