Skip to content
@olyn-chef

Olyn DevOps Framework for Chef

Olyn - An Automated DevOps Framework for Chef

Olyn is a bundle of custom Chef cookbooks to build and deploy functional clustered Debian web servers with globally replicated multi-write MySQL databases. All of this is done using infrastructure-as-code and configuration files.

Features

  • Configures a Percona cluster to securely replicate databases asynchronously to other nodes using their public IPs
  • Builds a virtual web root from multiple Git repos
  • Installs and configures openlitespeed web server with HTTPS
  • Uses HAProxy to enable a reverse proxy that handles internal load balancing and SSL offloading
  • Enables dynamic web page caching using Varnish and a custom warmer routine based on sitemap URLs
  • Handles secrets and certificates securely using a data bag
  • Sets up users, SSH keys, sudo, disables root login, and enforces security best practices
  • Configures UFW to allow installed services and communications from other nodes in the cluster and blocks all other traffic
  • Installs fail2ban to lock out intrusion attempts
  • Sets up logwatch to email important alerts and summaries of log events
  • Does your homework and the dishes

Tech

Olyn uses a number of open source projects to work properly. You can mix and match packages as needed using Berkshelf to build your ideal IaC setup.

  • Debian Linux - Reliable Linux distribution
  • Percona - Deploys replicated MySQL database reliably on any IaaS provider
  • openlitespeed - Open-source web server that boosts performance and supports Apache rewrites
  • HAProxy - Load balancing, reverse proxy, and SSL offloading services
  • Fail2Ban - Scans log files for malicious activity
  • Logwatch - Log monitoring and alerts
  • openSSH - Secure SSH tunneling
  • UFW - Software firewall
  • Varnish - Highly configurable HTTP full page cache

And of course Olyn itself runs on Chef.

Cookbooks

Each cookbook has its own repo and individual instructions if needed.

Cookbook Description
olyn_init Initializes and runs all installed Olyn cookbooks.
olyn_apt Configures APT. Sets up custom repos.
olyn_database Initializes databases. Creates database users. Configures user permissions. Imports SQL files.
olyn_fail2ban Installs Fail2Ban. Configures jails and ban durations.
olyn_git Installs Git. Sets up Git users. Maps CI/CD repos to file directory locations. Builds virtual WWW roots. Configures folder and file permissions. Syncs and deploys new commits.
olyn_haproxy Installs HAProxy. Configures front-end and back-end listeners. Sets up SSL offloading.
olyn_litespeed Configures Openlitespeed apt repos. Installs Openlitespeed package. Configures VHOSTS, TLS certificates, PHP, admin portal, and security.
olyn_logwatch Installs LogWatch. Sets up log monitoring and email alerts.
olyn_openssh Installs openSSH. Configures SSH ports and security.
olyn_percona Configures Percona apt repos. Installs Percona. Configures replication settings, TLS encryption, and node member list. Bootstraps and/or joins the replicated MySQL database cluster.
olyn_postfix Installs Postfix. Configures relay host.
olyn_sendmail Uninstalls sendmail.
olyn_sudo Installs sudo. Configures sudo group membership for users.
olyn_system Configures Debian OS. Adds cluster nodes to hosts file. Installs base apt packages. Securely installs public and private keys from TLS certificates in data bag. Sets timezone. Creates users and configures permissions.
olyn_ufw Installs UFW. Configures rules for ports, hosts, and protocols. Adds default deny rule. Reloads configuration.
olyn_varnish Installs Varnish. Creates front-end and back-end listeners. Compiles VCL rules for content expiration.
olyn_warmer Installs Nokogiri Ruby gem. Imports sitemaps URLs. Rewarms URLs from sitemaps in Varnish HTTP cache.

Local Dev Setup

Chef Secret Key File

Before deploying to a new environment a secret key file must be generated and saved at [CHEF_ROOT]/provision/chef_configs/encrypted_data_bag_secret.

To generate a new secret key file run the following in a Linux server:

openssl rand -base64 4096 | tr -d '\r\n' > encrypted_data_bag_secret

Berks Install Script

This script calls Berks to install all cookbooks and their dependencies into [CHEF_ROOT]/cookbooks from specified sources. Call this script from [CHEF_ROOT] during development. A Berksfile needs to be present in [CHEF_ROOT] with all of the required cookbooks listed. If a Berksfile.lock file already exists and the dependency versions are still valid, the existing cookbook version will be used.

To execute this script run the following in a Windows Terminal at [CHEF_ROOT]:

.\cookbooks\olyn_init\scripts\dev\berks\install.bat

Berks Update Cookbooks Script

This script calls Berks to update all cookbooks and their dependencies into [CHEF_ROOT]/cookbooks from specified sources. Call this script from [CHEF_ROOT] during development. A Berksfile needs to be present in [CHEF_ROOT] with all of the required cookbooks listed. Unlike the install.bat script, this will attempt to download the latest acceptable versions of all cookbooks and their dependencies.

To execute this script run the following in a Windows Terminal at [CHEF_ROOT]:

.\cookbooks\olyn_init\scripts\dev\berks\update.bat

Data Bags Encryption Script

This script encrypts any raw data bags stored under [CHEF_ROOT]/.unencrypted. Call it during development only. Unencrypted databag contents should never hit a live server or a final git repo. Encrypted databags are saved to [CHEF_ROOT]/data_bags using the secret key installed to Chef.

To encrypt all data bags run the following in a Windows Terminal at [CHEF_ROOT]:

.\cookbooks\olyn_init\scripts\dev\encrypt\data_bag.bat

To encrypt only specific data bags run the following in a Windows Terminal at [CHEF_ROOT]:

.\cookbooks\olyn_init\scripts\dev\encrypt\data_bag.bat [folder_1] [folder_2]

Initial Server Setup

Create The Chef Root Folder

As root, create the folder where the bootstrap Chef will reside:

mkdir ~/chef

Upload Chef Files

Connect via SFTP and upload the root of Chef into ~/chef.

Run the Bootstrap script

From the server execute the bootstrap script. It will update and install all required system packages, install Chef itself, and copy the secret key file into place:

sudo chmod +x ~/chef/cookbooks/olyn_init/scripts/bootstrap/boot.sh && sudo bash ~/chef/cookbooks/olyn_init/scripts/bootstrap/boot.sh && sudo chef-solo -c ~/chef/solo.rb -o "olyn_init"

After the Bootstrap script finishes, the first Chef run can be started:

sudo chef-solo -c ~/chef/solo.rb

You can now run any of the standard Chef commands below. After the first run is complete, remove the uploaded Chef files:

sudo rm ~/chef -R

Chef Commands

Standard Chef Run

Runs the default runlist of Chef recipes as specified in the [CHEF_ROOT]/node.json file.

sudo chef-solo -c ./solo.rb

Run A Single Chef Recipe

Overrides the default run list to run a single specified recipe.

sudo chef-solo -c ./solo.rb -o "[RECIPE_NAME]"

Written by

Scott Richardson

License

MIT

Popular repositories Loading

  1. olyn_postfix olyn_postfix Public

    Installs Postfix. Configures relay host. Part of the Olyn framework for Chef.

    Ruby

  2. olyn_sendmail olyn_sendmail Public

    Uninstalls sendmail. Part of the Olyn framework for Chef.

    Ruby

  3. olyn_sudo olyn_sudo Public

    Installs sudo. Configures sudo group membership for users. Part of the Olyn framework for Chef.

    Ruby

  4. olyn_system olyn_system Public

    Configures Debian OS. Adds cluster nodes to hosts file. Installs base apt packages. Securely installs public and private keys from TLS certificates in data bag. Sets timezone. Creates users and con…

    Ruby

  5. olyn_ufw olyn_ufw Public

    Installs UFW. Configures rules for ports, hosts, and protocols. Adds default deny rule. Reloads configuration. Part of the Olyn framework for Chef.

    Ruby

  6. olyn_varnish olyn_varnish Public

    Installs Varnish. Creates front-end and back-end listeners. Compiles VCL rules for content expiration. Part of the Olyn framework for Chef.

    HTML

Repositories

Showing 10 of 18 repositories
  • .github Public

    Main Readme file for the Olyn organization repo.

    olyn-chef/.github’s past year of commit activity
    0 MIT 0 0 0 Updated Apr 14, 2023
  • olyn_init Public

    Initializes and runs all installed Olyn cookbooks. Part of the Olyn framework for Chef.

    olyn-chef/olyn_init’s past year of commit activity
    Shell 0 0 0 0 Updated Mar 28, 2023
  • olyn_percona Public

    Configures Percona apt repos. Installs Percona. Configures replication settings, TLS encryption, and node member list. Bootstraps and/or joins the replicated MySQL database cluster. Part of the Olyn framework for Chef.

    olyn-chef/olyn_percona’s past year of commit activity
    Ruby 0 0 0 0 Updated Nov 7, 2022
  • olyn_openssh Public

    Installs openSSH. Configures SSH ports and security. Part of the Olyn framework for Chef.

    olyn-chef/olyn_openssh’s past year of commit activity
    Ruby 0 0 0 0 Updated Nov 7, 2022
  • olyn_logwatch Public

    Installs LogWatch. Sets up log monitoring and email alerts. Part of the Olyn framework for Chef.

    olyn-chef/olyn_logwatch’s past year of commit activity
    Ruby 0 0 0 0 Updated Nov 7, 2022
  • olyn_litespeed Public

    Configures Openlitespeed apt repos. Installs Openlitespeed package. Configures VHOSTS, TLS certificates, PHP, admin portal, and security. Part of the Olyn framework for Chef.

    olyn-chef/olyn_litespeed’s past year of commit activity
    HTML 0 0 0 0 Updated Nov 7, 2022
  • olyn_haproxy Public

    Installs HAProxy. Configures front-end and back-end listeners. Sets up SSL offloading. Part of the Olyn framework for Chef.

    olyn-chef/olyn_haproxy’s past year of commit activity
    Ruby 0 0 0 0 Updated Nov 7, 2022
  • olyn_git Public

    Installs Git. Sets up Git users. Maps CI/CD repos to file directory locations. Builds virtual WWW roots. Configures folder and file permissions. Syncs and deploys new commits. Part of the Olyn framework for Chef.

    olyn-chef/olyn_git’s past year of commit activity
    Ruby 0 0 0 0 Updated Nov 7, 2022
  • olyn_fail2ban Public

    Installs Fail2Ban. Configures jails and ban durations. Part of the Olyn framework for Chef.

    olyn-chef/olyn_fail2ban’s past year of commit activity
    Ruby 0 0 0 0 Updated Nov 7, 2022
  • olyn_database Public

    Initializes databases. Creates database users. Configures user permissions. Imports SQL files. Part of the Olyn framework for Chef.

    olyn-chef/olyn_database’s past year of commit activity
    Ruby 0 0 0 0 Updated Nov 7, 2022

Top languages

Loading…

Most used topics

Loading…