This documentation tries to make the Weave GitOps Enterprise architecture visible in a simple way.
You would benefit of this documentation if you are in any of the following journeys.
- I want to understand how Weave GitOps Enterprise looks as a system in a simple way.
- I want to understand the high level building blocks or component within Weave GitOps Enterprise in a simple way.
- I want to understand the different business domains served by Weave GitOps Enterprise in a simple way.
- For any of the domains, I want to go deeper in terms of behaviour, api or code in.
Diagrams aim to be self-explanatory however:
- They are based on C4 Model. If you have problems understanding them please take some time to get familiar via skimming abstractions and notation or watch this.
- They are using concepts from Domain Driven Design. If it gets difficult to read, please have a look to the following article.
- We are using mermaid for diagramming. It currently has support for C4 in early stage with limitations on editing experience or features.
It depends on your expected outcome. This section assumes that you are an engineering within Weave GitOps and you are onboarding to the product within a particular business domain. We suggest you use this documentation as part of the onboarding to get started:
- Head to Weave GitOps System as it provides you a high-level understanding on who are the users and Weave GitOps Dependencies.
- Review Weave GitOps Tiers as it provides with the layering of the application and access to the codebase by tier.
- View the Weave GitOps Domain to provide you a high level overview of the problem spaces that Weave GitOps faces.
- Select the domain that you would be contributing to. Understand the solution space via
- its diagram and user flows,
- follow the tiers approach UI -> API -> Backend,
- access the codebase.
This outside-in approach should provide you some structure when you face your first ticket.
Here there are few definitions that might help you to get along the documentation:
- Domain: A domain or business domain defines the problem space that you are interested to address for your users. For example, an infrastructure business domain addresses the problem of providing the infrastructure layer that developers would need for their applications.
- Sub-Domain: a problem space within a domain. For example, cluster provisioning is a subdomain within the infrastructure domain, so it addresses the problem of providing the infrastructure layer for running containerised apps.
Weave GitOps Enterprise is an enterprise application made of a
- management web application, deployed in a management Kubernetes cluster and
- a set of Kubernetes controllers and operators for executing different workflows.
It is a tiered web application with the following main characteristics:
Frontend: Weave GitOps Enterprise UI is the web interface that provides the experience layer to the users. It is built in react and uses yarn as the package manager.
More info about this tier here.
API: provides the integration layer for creating applications based in Weave GitOps Enterprise domain capabilities. It is defined mainly as protobuf and automatically converted to openapi. It is served as http/json and converted to grpc via grpc-gateway.
More info about this tier here.
Application or Backend: this layer provides the server and business logic for Weave GitOps Enterprise. It is built in Go using its latest version 1.20.
More info about this tier here.
Data and Storage: Weave GitOps Enterprise uses Git as its primary storage. It uses the Kubernetes API for other resources like configmaps for Weave GitOps Enterprise application configuration. It also uses caching based on SQLite for improving application UX and responsiveness.
More info about this tier here.
Git Providers: Weave GitOps Enterprise supports Gitops workflows in the major Git providers: GitHub, GitLab, Bitbucket or Azure Devops.
Authentication: any user or machine request flowing in Weave GitOps Enterprise is authenticated. User authentication is done via OIDC. JWT tokens are then used for auth then used for user authentication in the Weave GitOps Enterprise UI and in Kuberentes.
Authorization: any user or machine request flowing in Weave GitOps Enterprise is authorised. Weave GitOps Enterprise leverages authorization to Kubernetes RBAC.
More info about authentication and authorization in IAM
Deployment: Weave GitOps Enterprise is packaged and delivered via helm charts.
To extend Weave GitOps Enterprise the following services, controllers and operators are integrated:
From Weaveworks, Weave GitOps Enterprise integrates:
- Flux is the core component for delivery gitops applications.
- Flagger is the component within the Flux Ecosystem for providing advance deployment strategies or progressive delivery.
- Terraform Controller is the controller that provides capabilities to manage terraform infrastructure within WGE.
- Cluster Controller: provides the capability to integrate Kubernetes clusters for Weave GitOps Enterprise.
- Cluster Bootstrap Controller: responsible for bootstrapping Kubernetes clusters joining Weave GitOps Enterprise with baseline capabilities like flux.
- GitopsSet Controller: provides the capability to have a single definition for an application for multiple environments and a fleet of clusters.
- Pipeline Controller: provides continuous delivery pipelines capabilities for Weave GitOps Enterprise.
- Policy Agent: controller to enforces rego policies by controlling admission of violating resources.
These controllers are built in Go following Kubernetes industry-standard approach based on kubebuilder and controller runtime
From wider cloud-native ecosystem, Weave GitOps Enterprise integrates:
- External Secrets is the system that provides the capability to integrate external secret stores within WGE.
More info about each of them could be found within their domain. See domains view.
In terms of scalability Weave GitOps Enterprise Management Console presents the following features:
- It uses Golang's HTTP infrastructure which handles concurrent connections efficiently, so API server instances scale well vertically
- In terms of scalability models, Weave GitOps Enterprise management app could scale vertically to simplify operations, or
it could also do it horizontally as:
- User sessions are self-contained in cookies that comes in each user request.
- We don't hold application state, but we serve it from Kubernetes.
- Our data layer is cache layer with read-only capabilities that lives alongside each of the server instances.
- High Availability can be achieved by having multiple API server instanced behind a load balancer with sticky sessions enabled.
- Disaster Recovery for the app, as the state is hold in Kubernetes, follows the same approach as any other stateless application and to be determined by the DR strategy of the team running the platform running Weave GitOps Enterprise.
About the Kubernetes controllers the following statements are true:
- They use controller-runtime, which makes it convenient to scale by increasing the number of concurrent workers
- Given the reconcile resources that usually require write operation or mutating state, they are generally vertically scalable except Flux that supports horizontal scaling
- These controllers are built in Go following Kubernetes industry-standard approach based on kubebuilder and controller runtime, so they could achieve HA via multiple instances with enabled leader-election.
- Disaster Recovery for controllers, as the state is hold in kubernetes, follows the same approach as any other stateless application and to be determined by the DR strategy of the team running the platform running Weave GitOps Enterprise.
This outer layer provides a look to Weave GitOps Enterprise (wge) in three views:
- As a System including its wider context and external dependencies.
- As Tiers that provides a high level overview of the application tiers.
- As Domains to provide an overview of the problem spaces that WGE addresses.
Weave GitOps Enterprise (WGE) as a system, provides management capabilities, for Platform Engineers and Applications Developers, in the context of delivering and operating gitops applications based on Flux across Kubernetes clusters:
As mentioned, there are two main personas to serve:
- Platform Engineers who are in charge of admin and operate the platform.
- Application Developers that deliver business application within the platform.
WGE sits on top or integrate a set of external systems for gitops delivery and runtime:
- Kubernetes clusters are the core component for running applications.
- Flux is the core component for delivery gitops applications.
- Flagger is the component within the Flux Ecosystem for providing advance deployment strategies or progressive delivery.
- GitProviders hosts the git repositories that hosts any gitops configuration manifest.Serves as sources of truth. Examples are GitHub or GitLab.
- Identity Provider: provides identity services for Weave GitOps Enterprise to integrate in the context of user authentication and authorization. Examples of identity providers could be Google or Okta.
- External Secrets is the system that provides the capability to integrate external secret stores within WGE.
- Terraform Controller is the system that provides capabilities to manage terraform infrastructure within WGE.
The following diagram represents the system as a whole. The following considerations might be useful to be fully understand the diagram:
-
Weave GitOps Cluster is represented once for simplicity but in reality is presented as:
- Weave GitOps Management Cluster: single instance that holds the management console and control plane controllers.
- Weave GitOps Leaf Cluster: multiple instances that run business applications and managed by Weave GitOps Management Cluster.
-
In terms of data-flows, the line starts by the source entity starting the connection and commonly starting a request. For example, Weave GitOps Enterprise opens connections to a Git Provider api for example to request creating a Pull Request.
-
In terms of networks, we could say that:
- Weave GitOps Cluster:
- All components from a cluster lives in the same kubernetes cluster so under the same kubernetes control plane that would be commonly deployed in the same network (for example in a VPC).
- Communication between Weave GitOps Management and Weave GitOps Leaf clusters happens via Kubernetes API so needs to be accessible.
- Platform Engineers or Developers consumes the app via a web browser that would communicate with the API server via either public or private networks depending on the customer deployment.
- External Systems outside Management or Leaf could communicate via public or private network depending on the customer needs and requirements. For example, it could communicate with GitHub via internet if using the hosted version, or via internal network if using GitHub Enterprise self-hosted.
- Weave GitOps Cluster:
C4Context
title Weave GitOps Enterprise Context
Person(PlatformEngineer, "Platform Engineer", "operates WGE platform for applications")
Person(ApplicationDeveloper, "Application Developer","writes and operates Line-of-Business applications")
System(WeaveGitopsEnterprise, "Weave GitOps Enterprise")
Rel(WeaveGitopsEnterprise, externalPromotion, "promotes applications via pipelines")
UpdateRelStyle(WeaveGitopsEnterprise, externalPromotion, $offsetY="40")
System_Ext(Idp, "Identity Provider", "provides authn services for customer users")
System_Ext(externalPromotion, "External Promotions", "handles application promotions: ex Jenkins")
System_Ext(GitProvider, "GitProvider", "source storage in Git. Ex. GitHub")
Rel(PlatformEngineer, WeaveGitopsEnterprise, "Manages Platform")
UpdateRelStyle(PlatformEngineer, WeaveGitopsEnterprise, $offsetY="-10")
Rel(ApplicationDeveloper, WeaveGitopsEnterprise, "Delivers Application")
UpdateRelStyle(ApplicationDeveloper, WeaveGitopsEnterprise, $offsetX="50", $offsetY="-10")
Rel(WeaveGitopsEnterprise, GitProvider, "Gitops flow for changes")
UpdateRelStyle(WeaveGitopsEnterprise, GitProvider, $offsetX="-60", $offsetY="-20")
Rel(WeaveGitopsEnterprise, KubernetesCluster, "read resources via api")
UpdateRelStyle(WeaveGitopsEnterprise, KubernetesCluster, $offsetX="10", $offsetY="40")
Rel(WeaveGitopsEnterprise, Idp, "authenticate users")
UpdateRelStyle(WeaveGitopsEnterprise, Idp, $offsetX="-50", $offsetY="-20")
Boundary(Runtime, "Weave GitOps Cluster") {
Boundary(Kubernetes, "Kubernetes") {
System_Ext(KubernetesCluster, "Kubernetes Cluster", "run customer applications")
}
Boundary(FluxRuntime, "Flux Runtime") {
System_Ext(Flux, "Flux", "deploy customer applications")
System_Ext(Flagger, "Flagger", "provides progressive delivery capabilities")
Rel(Flux, KubernetesCluster, "deploy apps to")
UpdateRelStyle(Flux, KubernetesCluster, $offsetX="-40", $offsetY="-20")
Rel(Flux, GitProvider, "syncs app manifests")
UpdateRelStyle(Flux, GitProvider, $offsetX="0", $offsetY="-70")
Rel(Flux, artifactsRepo, "syncs app artifacts")
Rel(Flagger, KubernetesCluster, "manage canary resources from")
UpdateRelStyle(Flagger, KubernetesCluster, $offsetX="50", $offsetY="0")
}
Boundary(Terraform, "Terraform") {
System_Ext(TerraformController, "Terraform Controller", "provides infrastructure provisioning via terraform")
Rel(TerraformController, KubernetesCluster, "manage terraform resources from")
UpdateRelStyle(TerraformController, KubernetesCluster, $offsetX="-180", $offsetY="0")
Rel(TerraformController, CloudProvider, "provisions apps infra")
UpdateRelStyle(TerraformController, CloudProvider, $offsetX="-110", $offsetY="0")
}
Boundary(Secrets, "Secrets") {
System_Ext(ExternalSecrets, "External Secrets", "provides capability to integrate external secret stores")
Rel(ExternalSecrets, KubernetesCluster, "manage external secrets resources from")
UpdateRelStyle(ExternalSecrets, KubernetesCluster, $offsetX="120", $offsetY="40")
Rel(ExternalSecrets, secretsStore, "syncs secrets to Kubernetes")
}
}
Container_Boundary(CustomerEnterpriseSystem, "Other Customer Enterprise Systems") {
System_Ext(artifactsRepo, "Artifacts Repository", "hosts packages, images, charts, oci")
System_Ext(secretsStore, "Secrets Store", "customer secrets system of record")
}
Container_Boundary(Cloud, "Cloud") {
System_Ext(CloudProvider, "Cloud Provider", "p")
}
UpdateLayoutConfig($c4ShapeInRow="2", $c4BoundaryInRow="2")
Weave GitOps Enterprise could be also viewed as a three-tier architecture where:
Weave GitOps Enterprise UI is the web interface that provides the experience layer to the users. It is built in React and uses yarn as the package manager.
As experience layer is oriented to end user in order to enable Platform Engineers or Developers to achieve their workflows. These workflows are delivered through components that mostly falls into the following categories:
- To view and filter a collection of resources that spans multiple Kubernetes clusters. it also aims to have summary single pane of glass by resources. Example could be Cluster List
- Once determined the resource context, to drill down in the resources via its detail view. This details view contains not only the desired state but also its reconciled or runtime state, in addition to general and specific management operations. An example could be ClusterDetails.tsx
- Add operations for any resource, via gitops flow. For example Add Application
- Other management operations specific for the resource. For example for pipelines, exercise a promotion PromotePipeline.tsx
The pattern to execute those workflows are the same:
- There is a React component modeling the interface to the user for showing data and gathering input.
- The data is gathered from the backend via api clients automatically generated from their protobuf definitions
As mentioned in the previous section, the API layer provides the integration layer for creating experience applications on top of Weave GitOps Enterprise domains, as well, as to enable other type of integrations, for example integration with other systems.
The primary API definitions are protobuf and defined here. The API it is
- defined as grpc
- consumed as rest/json from the UI
- converted http/json to grpc via grpc-gateway.
Weave GitOps Enterprise application layer or backend, it is an api server developed in Go built around its domain entities and API definition.
Given so, a good way to understand the chain of responsibility is through the flow of an API request:
- the experience layer sends a request to the backend endpoint for example "GET /v1/pipelines" to retrieve the pipeline collection
- the request is being received by the backend server that goes through a set of filter or middleware.
- One of those filters is authentication of the request that could be seen in authentication
- The request is being then handled by the api endpoint handler, in our example list.go
that usually:
- Requests the resource to its cluster via Kubernetes API, in this case, a Pipeline resource to the Management cluster,
- Aggregates and process the data
- Returns the result set to the api client
The backend application is a modular application based on aggregation of domain-specific servers:
- Clusters service server.go provides that aggregation layer for the domain-specific servers (as well as service its specific domain).
- Domain-specific server are self-contained within their package for example pipelines contains the pipelines server.
- The server integration is controlled via feature flagging like:
if featureflags.Get("WEAVE_GITOPS_FEATURE_PIPELINES") != "" { if err := pipelines.Hydrate(ctx, grpcMux, pipelines.ServerOpts{}); err != nil { return fmt.Errorf("hydrating pipelines server: %w", err) } }
In terms of data and storage:
- Weave GitOps Enterprise uses Git as its primary storage.
- It also uses the Kubernetes API for other resources like configmaps for Weave GitOps Enterprise application configuration.
- It also uses caching based on SQLite for improving application UX and responsiveness:
- Search engine store.go
- Helm chart indexer.go
The previous layers are seen in the following diagram
C4Container
title Weave GitOps Enterprise
Person(PlatformEngineer, "Platform Engineer")
Person(developer, "Application Developer")
Rel(PlatformEngineer, WeaveGitopsEnterpriseUi, "Manages Platform")
UpdateRelStyle(PlatformEngineer, WeaveGitopsEnterpriseUi, "", "", "-115", "-40")
Rel(developer, WeaveGitopsEnterpriseUi, "Delivers Application")
UpdateRelStyle(developer, WeaveGitopsEnterpriseUi, "", "", "50", "-40")
Container_Boundary(Frontend, "Frontend") {
Container(WeaveGitopsEnterpriseUi, "Weave GitOps Enterprise UI","javascript and reactJs","Weave GitOps Enterprise experience via web browser")
Rel(WeaveGitopsEnterpriseUi, WeaveGitopsEnterpriseBackend, "consume api")
UpdateRelStyle(WeaveGitopsEnterpriseUi, WeaveGitopsEnterpriseBackend, "", "", "-35", "-15")
}
Container_Boundary(Application, "Application") {
Container(WeaveGitopsEnterpriseBackend, "Weave GitOps Enterprise Backend","go","backend application with api")
Rel(WeaveGitopsEnterpriseBackend, Kubernetes, "reads configuration and resources")
Rel(WeaveGitopsEnterpriseBackend, Cache, "read cached resources")
System_Ext(Flux, "Flux", "stores gitops deployments configuration")
Rel(Flux, GitProvider, "read gitops deployments")
Rel(Flux, Kubernetes, "deploy gitops deployments")
}
Container_Boundary(Data, "Data & Storage") {
Container(Cache, "Cache","sqlite","caches resources for UX")
System_Ext(Kubernetes, "Kubernetes", "stores configuration and resources manifests")
System_Ext(GitProvider, "GitProvider", "stores gitops deployments configuration")
}
UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="2")
There is another high-level architectural view to understand Weave GitOps Enterprise: the problem spaces it faces or business domains and its intersection with GitOps. The following diagram presents them altogether:
C4Container
title Weave GitOps Enterprise - Domains
Container_Boundary(WeaveGitopsEnterprise, "Weave GitOps Enterprise") {
Container_Boundary(Infrastructure, "Infrastructure") {
Component(ClusterManagement, "Cluster Management",, "management of Kubernetes clusters")
Component(Terraform, "Terraform",, "management of Terraform infrastructure")
}
Container_Boundary(SelfService, "Self Service") {
Component(Templates, "GitopsTemplates",, "component reusability for security,best practices and reduced time to value")
Component(GitopSets, "GitopsSets",, "creation of archetypal components to simplify applications maintainability")
Component(Profiles, "Profiles",, "provision clusters with common baseline components in a repeatable way")
}
Container_Boundary(IAM, "Identity and Access Management") {
Component(Idp, "Identity Provider",, "integration of existing customer user identities")
Component(Authentication, "Authentication",, "validate users identity while accessing resources")
Component(Authorization, "Authorization",, "authorise users intent while doing actions over resources")
}
Container_Boundary(ApplicationDelivery, "Application Delivery") {
Component(Flux, "Flux",, "deployment of applications")
Component(Pipelines, "Pipelines",, "delivery of applications across environments in its way to production")
Component(ProgressiveDelivery, "Progressive Delivery",, "releasing applications while protecting its availability")
}
Container_Boundary(TrustedDelivery, "Trusted Delivery") {
Component(Policy, "Policy",, "policy definition and enforcement for applications")
Component(Tenancy, "Tenancy",, "allow multiple tenant to consume platform services while keeping isolation")
}
Container_Boundary(DevEx, "Development Experience") {
Component(GitopsRun, "GitopsRun",, "simplifies development in gitops")
Component(Notifications, "Notifications",, "higher-level workflows based on events")
}
Container_Boundary(Explore, "Explore") {
Component(Explorer, "Explorer",, "addresses the problem of discovering data across multiple dimensions from a single place")
}
}
UpdateLayoutConfig($c4ShapeInRow="1", $c4BoundaryInRow="3")
To these problems, WGE provides solutions for. The next section goes into each of the domains.
Our users require both infrastructure to run applications and the applications itself requires infrastructure services like data stores for example. Infrastructure domain addresses both of the problems:
- Provides a general purpose approach to provision and manage Kuberentes clusters.
- Provides a general infrastructure management solution based on Terraform to allow application temas to self-service their infrastructure needs.
Subdomains:
- Cluster Management: to manage Kubernetes clusters within WGE.
- Terraform: to manage infrastructure via Terraform within WGE.
For a deeper look into the domain Infrastructure
Our users want to streamline the process of getting started while reusing organisational knowledge and ensuring alignment with organisational standards. Self-service domains focuses in this problem space by the following subdomains:
- GitopsTemplates: how to create reusable components in order to enable best practices or reduce time to value for platform or application teams.
- GitopsSets: how to create archetypal components that simplifies maintainability of applications in a complex a dynamic platform.
- Profiles: how to bootstrap clusters with baseline components in a repeatable way.
For a deeper look into the domain Self Service
While we want to empower developers and DevOps teams to build better software faster through we also want them to be able to do it confidently. Moving fast often means risking quality and security, and counting on humans to spot problems is often error-prone and is not scalable. Trusted Delivery covers this part of the value we're trying to deliver to our customers.
Subdomains:
- Policy: policy definition and enforcement for applications.
- Tenancy: allow multiple tenant to consume platform services while keeping isolation.
For a deeper look into the domain Trusted Delivery
Organisations provide value to customer through business capabilities, those business capabilities are created in the form of applications that needs to be managed and delivered to customers.
Subdomains:
- Progressive Delivery: how to release applications to environments protecting application availability.
- Pipelines: deploying software across a set of stages (a pipeline), each becoming more production-like. This is usually done in an automated fashion, but not exclusively.
- Flux: how to deploy gitops applications to Kubernetes clusters.
For a deeper look into the domain Application Delivery
Development within gitops represents different challenges to traditional software development. To reduce the adoption barriers is the problem to solve within this domain.
Subdomains:
- GitopsRun: to remove the complexity for developers so that Platform Engineers can create developer environments easily, and Developers can benefit from GitOps and focus on writing code.
- Notifications: to broadcast and ingest events that enables higher-level workflows.
For a deeper look into the domain Development Experience
Users want to use their existing (or new) identities to consume services and to establish access control to them. The following subdomains are part of it:
- Authentication: identifies an anonymous user within the platform.
- Authorization: validate users identity while accessing resources.
- Identity Providers Integration: address the problem of integrating existing identity providers within Weave GitOps Enterprise keeping their identities.
For a deeper look into the domain IAM
Modern applications and infrastructure presents challenges to discover and aggregate data coming from different sources. Explore domain addresses this problem space to provide users a single and simple way.
- Explorer: is the capability that allows searching across multiple infrastructure domains in a single place.
For a deeper look into the domain Explore