Skip to content
This repository has been archived by the owner on Apr 21, 2023. It is now read-only.

Commit

Permalink
Added support for Windows
Browse files Browse the repository at this point in the history
  • Loading branch information
jorritfolmer committed Jun 25, 2017
1 parent 841f9a0 commit f5f2ff2
Show file tree
Hide file tree
Showing 29 changed files with 635 additions and 278 deletions.
8 changes: 4 additions & 4 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ env:
- PUPPET_GEM_VERSION=2.7.14
- PUPPET_GEM_VERSION=3.8.5
- PUPPET_GEM_VERSION=4.6.2
- PUPPET_GEM_VERSION=4.9.2
- PUPPET_GEM_VERSION=4.10.4
matrix:
fast_finish: true
notifications:
Expand All @@ -19,12 +19,12 @@ matrix:
- rvm: 1.8.7
env: PUPPET_GEM_VERSION=4.6.2
- rvm: 1.8.7
env: PUPPET_GEM_VERSION=4.9.2
env: PUPPET_GEM_VERSION=4.10.4
- rvm: 1.9.3
env: PUPPET_GEM_VERSION=4.9.2
env: PUPPET_GEM_VERSION=4.10.4
- rvm: 2.0.0
env: PUPPET_GEM_VERSION=2.7.14
- rvm: 2.0.0
env: PUPPET_GEM_VERSION=4.9.2
env: PUPPET_GEM_VERSION=4.10.4
- rvm: 2.1
env: PUPPET_GEM_VERSION=2.7.14
18 changes: 15 additions & 3 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ end

# rspec must be v2 for ruby 1.8.7
if RUBY_VERSION >= '1.8.7' and RUBY_VERSION < '1.9'
gem 'metadata-json-lint', '0.0.11'
gem 'fast_gettext', '~> 1.0.0'
gem 'gettext-setup', '<= 0.13'
gem 'metadata-json-lint', '<= 0.0.11'
gem 'rspec', '~> 2.0'
gem 'rake', '~> 10.4.2'
gem 'puppet-lint', '~> 1.1.0'
Expand All @@ -22,7 +24,8 @@ end

# json > v2.0 requires ruby>2.0
if RUBY_VERSION >= '1.9' and RUBY_VERSION < '2.0'
gem 'metadata-json-lint'
gem 'fast_gettext', '~> 1.1.0'
gem 'metadata-json-lint', '~> 1.1.0'
gem 'rspec', '~> 2.0'
gem 'rake', '~> 10.4.2'
gem 'puppet-lint', '~> 1.1.0'
Expand All @@ -33,7 +36,16 @@ if RUBY_VERSION >= '1.9' and RUBY_VERSION < '2.0'
gem 'json_pure', '~> 1.8.3'
end

if RUBY_VERSION >= '2.0'
if RUBY_VERSION >= '2.0' and RUBY_VERSION < '2.1'
gem 'fast_gettext', '~> 1.1.0'
gem 'metadata-json-lint'
gem 'puppet-syntax'
gem 'puppetlabs_spec_helper'
gem 'puppet-lint'
gem 'facter'
end

if RUBY_VERSION > '2.1'
gem 'metadata-json-lint'
gem 'puppet-syntax'
gem 'puppetlabs_spec_helper'
Expand Down
105 changes: 90 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Puppet module to deploy Splunk into any imaginable topology.
# Puppet module to deploy Splunk into any imaginable topology on Windows and Linux.

[![Travis CI build status](https://travis-ci.org/jorritfolmer/puppet-splunk.svg?branch=master)](https://travis-ci.org/jorritfolmer/puppet-splunk)

This Puppet module can be used to create and arrange the following Splunk instances into simple, distributed or (multisite) clustered topologies:
This Puppet module can be used on Windows and Linux to create and arrange the following Splunk instances into simple, distributed or (multisite) clustered topologies:

- Splunk indexers
- Splunk search heads
Expand All @@ -27,9 +27,14 @@ It does so with the following principles in mind:
- Admin password can be set using its SHA512 hash in the Puppet manifests instead of plain-text.
4. **Supports any topology.** Single server? Redundant multi-site clustering? Heavy forwarder in a DMZ?

## Prerequisites

1. A running Puppet master
2. A running repository server with splunk and splunkforwarder packages. See below if you need help setting it up.

## Quick-start

Define a single standalone Splunk instance that you can use to index and search, for example with the trial license:
Define a single standalone Splunk instance on Linux that you can use to index and search, for example with the trial license:

![Example 1a](example1.png)

Expand All @@ -43,12 +48,21 @@ node 'splunk-server.internal.corp.tld' {
}
```

See the other examples below for more elaborate topologies.
Or define a single standalone Splunk instance on Windows with:

## Prerequisites
```puppet
node 'splunk-server.internal.corp.tld' {
class { 'splunk':
package_source => '//dc01/Company/splunk-6.6.1-aeae3fe0c5af-x64-release.msi',
httpport => 8000,
kvstoreport => 8191,
inputport => 9997,
}
}
```

See the other examples below for more elaborate topologies.

1. A running Puppet master
2. A running repository server with splunk and splunkforwarder packages. See below if you need help setting it up.

### Splunk YUM repository (Red Hat based)

Expand Down Expand Up @@ -102,21 +116,32 @@ file { "/etc/apt/sources.list.d/splunk.list":
}
```

## Installation
### CIFS share with .msi files (Windows based)

For Windows installations just put the .msi Splunk installation files for
Windows on a share that is accessible from all your Windows servers.

1. create a share that can be accessed by all your Windows servers
2. download the relevant Splunk .msi files from the Splunk website into this share
3. specify `package_source` and point to one of these .msi files


## Puppet-Splunk installation

1. SSH to your Puppet master
2. `cd /etc/puppet/modules`
2. `cd /etc/puppet/modules` or `cd /etc/puppetlabs/code/environments/production/modules`, depending on your Puppet version
3. `puppet module install jorritfolmer-splunk` or `git clone https://github.com/jorritfolmer/puppet-splunk.git; mv puppet-splunk splunk`
4. Create your Splunk topology, see below for examples.

## Usage

To give this module a try, you don't necessarily have to setup a Certiticate Authority for the various SSL certificates that Splunk uses.

1. By default Splunk already uses its own CA (1024 bits) that is used to create and sign the certificate for the 8089/tcp management port and 8000/tcp web interface: /opt/splunk/etc/auth/ca.pem. However, since everyone can grab the key from a Splunk trial download, it's an unlikely candidate for real production use.
2. Because there is already a Puppet CA in place, this module reuses the client key (4096 bits) and client certificate signed by the Puppet CA. For quick testing in heterogeneous non-production environments you can revert to using the Splunk provides certs and CA with `reuse_puppet_certs => false`.
By default, this module reuses the Puppet client SSL key (4096 bits) and client certificate, so we can save us the trouble of setting up and maintaining our own certificate authority.

For quick testing in heterogeneous non-production environments you can revert to using the Splunk provides certs and CA with `reuse_puppet_certs => false`. Or you can point to your own certificates with `sslcertpath` and `sslrootcapath`.

By default, the Splunk module doesn't manage the state of the splunk service, except configure to start Splunk or Splunkforwarder at boot time. However, if you do want Puppet to interfere while performing a cluster rolling restart or an indexer restart, have a look at the `service` parameter.
The Splunk module doesn't manage the state of the splunk service, except configure to start Splunk or Splunkforwarder at boot time. However, if you do want Puppet to interfere while performing a cluster rolling restart or an indexer restart, have a look at the `service` parameter.

### Example 1:

Expand All @@ -138,6 +163,22 @@ node 'splunk-server.internal.corp.tld' {
}
```

To define a standalone Splunk instance running on Windows:

```puppet
node 'splunk-server.internal.corp.tld' {
class { 'splunk':
package_source => '//dc01/Company/splunk-6.6.1-aeae3fe0c5af-x64-release.msi',
httpport => 8000,
kvstoreport => 8191,
inputport => 9997,
reuse_puppet_certs => false,
sslcertpath => 'server.pem',
sslrootcapath => 'cacert.pem',
}
}
```

### Example 2a:

Extends the example above with a node that will run the Splunk universal forwarder. It uses the first server as Deployment Server (`ds =>`) where apps, inputs and outputs can be managed and deployed through Forwarder Management.
Expand All @@ -161,6 +202,28 @@ node 'some-server.internal.corp.tld' {
}
```

The equivalent for Windows environments:

```puppet
node 'splunk-server.internal.corp.tld' {
class { 'splunk':
package_source => '//dc01/Company/splunk-6.6.1-aeae3fe0c5af-x64-release.msi',
httpport => 8000,
kvstoreport => 8191,
inputport => 9997,
}
}
node 'some-server.internal.corp.tld' {
class { 'splunk':
package_source => '//dc01/Company/splunkforwarder-6.6.1-aeae3fe0c5af-x64-release.msi',
type => 'uf',
ds => 'splunk-server.internal.corp.tld:8089',
}
}
```


### Example 2b:

Almost identical to example 2a, except with some SSL downgrading, not suitable for production.
Expand Down Expand Up @@ -576,6 +639,11 @@ node 'splunk-idx2.internal.corp.tld',
Set `tcpout => 'splunk-idx1.internal.corp.tld:9997'` if you do want to
forward events to a Splunk indexer.

#### `package_source`

Optional and for Windows only. Use this to point to the .msi installation file.
This can be a UNC path like \\DC01\Company\splunkforwarder-6.6.1-aeae3fe0c5af-x64-release.msi

#### `splunk_os_user`

Optional. Run the Splunk instance as this user. Defaults to `splunk`
Expand Down Expand Up @@ -658,6 +726,10 @@ node 'splunk-idx2.internal.corp.tld',
- `fn` (Full name)
- `email` (Email address)

#### `minfreespace`

Optional. Used to specify the minimum amount of freespace in kb before Splunk stops indexing data.

#### `service`

Optional. Used to manage the running and startup state of the
Expand Down Expand Up @@ -703,7 +775,6 @@ node 'splunk-idx2.internal.corp.tld',
Optional. Used to request indexer acknowlegement when sending data.
Defaults to false.


#### `version`

Optional. Specify the Splunk version to use.
Expand Down Expand Up @@ -753,10 +824,14 @@ If you have version >= 6.2.0 servers but with stock settings from a previous Spl

Moved to CHANGELOG.md

## Test coverage

Moved to TEST_COVERAGE.md

## Roadmap

- Data Collection Node
- Add defined type so multiple splunk instances can be deployed on a single system
- Managed service account for Windows installations
- Convert examples to patterns or building blocks

## Out-of-scope

Expand Down
18 changes: 14 additions & 4 deletions TEST_COVERAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,14 @@
| version | tested | total |
|---------|--------|-------|
| v3.1.3 | 22 | 40 |
| v3.2.0 | 23 | 42 |

## By operating system:

| os | tested | total |
|---------|--------|-------|
| linux | 23 | 42 |
| windows | 0 | 42 |

## By parameter:

Expand All @@ -29,6 +37,7 @@
| `kvstoreport`| Y |
| `lm`| Y |
| `minfreespace` | no |
| `package_source` | no |
| `pass4symmkey` | no |
| `phonehomeintervalinsec` | no |
| `replication_port`| Y |
Expand All @@ -41,12 +50,13 @@
| `shclustering => { mode => 'searchhead'`| Y |
| `splunk_bindip` | no |
| `splunk_os_user` | no |
| `splunk_os_group` | no |
| `sslcertpath`| Y |
| `sslcompatibility` | no |
| `sslrootcapath`| Y |
| `sslrootcapath` | Y |
| `sslversions_intermediate` | no |
| `sslversions_modern` | no |
| `tcpout`| Y |
| `type => 'uf'`| Y |
| `use_ack` | no |
| `tcpout` | Y |
| `type => 'uf'` | Y |
| `use_ack` | yes |
| `version` | no |
21 changes: 12 additions & 9 deletions manifests/authentication.pp
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
class splunk::authentication
(
$splunk_home = $splunk::splunk_home,
$splunk_os_user = $splunk::splunk_os_user,
$splunk_os_user = $splunk::real_splunk_os_user,
$splunk_os_group = $splunk::real_splunk_os_group,
$splunk_dir_mode = $splunk::real_splunk_dir_mode,
$splunk_file_mode = $splunk::real_splunk_file_mode,
$auth = $splunk::auth,
$splunk_app_precedence_dir = $splunk::splunk_app_precedence_dir,
$splunk_app_replace = $splunk::splunk_app_replace,
Expand Down Expand Up @@ -59,14 +62,14 @@
"${splunk_home}/etc/apps/${splunk_app_name}_saml_base/metadata",]:
ensure => directory,
owner => $splunk_os_user,
group => $splunk_os_user,
mode => '0700',
group => $splunk_os_group,
mode => $splunk_dir_mode,
}
-> file { "${splunk_home}/etc/apps/${splunk_app_name}_saml_base/${splunk_app_precedence_dir}/authentication.conf":
ensure => present,
owner => $splunk_os_user,
group => $splunk_os_user,
mode => '0600',
group => $splunk_os_group,
mode => $splunk_file_mode,
replace => $splunk_app_replace,
content => template("splunk/${splunk_app_name}_saml_base/local/authentication.conf"),
}
Expand All @@ -85,14 +88,14 @@
"${splunk_home}/etc/apps/${splunk_app_name}_ldap_base/metadata",]:
ensure => directory,
owner => $splunk_os_user,
group => $splunk_os_user,
mode => '0700',
group => $splunk_os_group,
mode => $splunk_dir_mode,
}
-> file { "${splunk_home}/etc/apps/${splunk_app_name}_ldap_base/${splunk_app_precedence_dir}/authentication.conf":
ensure => present,
owner => $splunk_os_user,
group => $splunk_os_user,
mode => '0600',
group => $splunk_os_group,
mode => $splunk_file_mode,
replace => $splunk_app_replace,
content => template("splunk/${splunk_app_name}_ldap_base/local/authentication.conf"),
}
Expand Down
Loading

0 comments on commit f5f2ff2

Please sign in to comment.