diff --git a/inventory/README.md b/inventory/README.md index 7d131893..d4b730a8 100644 --- a/inventory/README.md +++ b/inventory/README.md @@ -1,17 +1,40 @@ -### .. magic inside -this inventory consists of three inventories. - - -# base_inventory -the first one prints some ansible-compatible hostlist based on all the folders inside host_vars. - - -# keyed_groups_stage_1.config -this is a configuration for the "constructed" inventory plugin. it dynamically constructs group memberships based on the host_vars. as the first inventory doesnt output any host & group vars, the constructed inventory has to fetchit on its own. This pretty new feature is available since ansible 2.11 . Its controlled by the "use_vars_plugins" key. Unfortunately its searching for the host&group vars in the folder of the first inventory plugin. As its located here, we need to symlink 'em here in order to allow it accomplishing its job. - -.. pretty hacky, but less hacky than before :) ..lets hope ansible will continue improving - - -# keyed_groups_stage_2.config -this is yet another configuration for the "constructed" inventory plugin. in contrast to the first stage its purpose is to dynamically construct even more group memberships based on inherited vars from the previous stage. +# Inventory construction + +Inventory data is used to generate host-specific OpenWrt config files, +which are then combined into an image file that can be flashed to a device. + +Data model concepts are most easily explain by example, +so please also check out the location files in `locations/` directory. +We have one file per network location, and one or more hosts per location. +Ansible had groups and hosts, one file per host, which wasn't practical for us. + +There are five (now four) stages to construct our inventory data: + +1. The base inventory script + - Translates our location-centered data model to Ansible's data model. + - It collects the hostvars for all hosts, which consist of the host object + in a location file, merged with the surrounding location object. + - See documentation in `base_inventory` for more details. +2. `host_vars/` + - In earlier times we used individual files in `host_vars/`. +3. Keyed groups, pt. 1 + - These are a mechanism from the "constructed" Ansible plugin, which we use + to load additional group variables based on certain property values. + - There are two parts so that new data from part 1 can set properties + that result in more new data in part 2 (e.g. `model` and OpenWrt version). + - This stage handles `location` (old), `target`, `model`, and `role`. +4. Keyed groups, pt. 2 + - Same as before, but for `target` and `openwrt_version`. +5. Merge vars + - All hostvars construction so far was only able to overwrite properties, + but in some cases we need to merge with the existing property. + - For specific properties, a "merge var" can be set: + `packages: ["some-pkg"]` and `xxx__packages__to_merge: ["another-pkg"]` + where `xxx` is an arbitrary name to allow for multiple merge vars. + For this arbitrary name we usually pick something that describes the + scope we're currently in, e.g. `location__packages__to_merge`. + - These merge vars are merged together into one + before any templates or tasks make use of hostvars. + - Handles `ssh_keys`, `packages`, `sysctl`, `rclocal`, + `disabled_services`, `wireless_profiles`, `channel_assignments_*`. diff --git a/inventory/base_inventory b/inventory/base_inventory index b713ea1b..ad8e72d4 100755 --- a/inventory/base_inventory +++ b/inventory/base_inventory @@ -1,18 +1,68 @@ #!/usr/bin/env bash +# +# This script's output is a JSON object containing an array of all host names, +# and an array with the initial hostvars for each host. +# More information on inventory construction can be found at +# https://docs.ansible.com/ansible/latest/dev_guide/developing_inventory.html +# +# The queries and conversions are done with the help of the `jq` and `yq` tools. +# +# We first grab the JSON representation of every location YAML definition, +# then read `.hosts[].hostname` for each location to create the list of all hosts. +# +# Usually Ansible then calls this script with `--host ` for every host. +# That does get very slow with hundreds of hosts, +# which is why Ansible allows for constructing all hostvars objects in advance. +# +# To construct the hostvars object for a host, +# we take as a base the full location object (without `.hosts`), +# then merge the object from `.hosts[]` with a matching `hostname` value. +# This way, host values overwrite location values. +# (We actually merge host<-location<-host to preserve JSON ordering.) +# +# Example location file: +# +# --- +# location: pktpls +# hosts: +# - hostname: pktpls-core +# role: corerouter +# string: host-var-has-precedence +# object: { two: 456 } +# array: [ bar ] +# string: will-be-overridden +# object: { one: 123 } +# array: [ foo ] +# +# Resulting hostvars object, before keyed groups being applied: +# +# { +# "location": "pktpls", +# "hostname": "pktpls-core", +# "role": "corerouter", +# "string": "host-var-has-precedence", +# "object": { +# "two": 456 +# }, +# "array": [ +# "bar" +# ] +# } +# + set -e # set -x case "$1" in --host) - # No op - won't be called by Ansible anymore because --list contains all data. - # See https://docs.ansible.com/ansible/latest/dev_guide/developing_inventory.html#tuning-the-external-inventory-script + # No op, only ever called with --list echo "{}" exit 0 ;; --list) # Print all location files as consecutive JSON objects. - # Later jq -s/--slurp will read these as one top-level array of objects. + # Further down, jq -s/--slurp reads these as one top-level array of objects. locjson="$(yq '.' locations/*.yml)" cat <