2014/12/2
(GH-40) Display expected and actual module versions
When displaying the state of a deployment, modules would report a version but
would not indicate if that was the expected version or desired version. This has
been changed so that both the expected and actual module version information is
displayed. Because determining the actual version for modules can be slow the
--detail flag must be passed to display this information.
(GH-43) Using a relative moduledir in the Puppetfile is relative to the Puppetfile
The moduledir setting in the Puppetfile could be used to set a custom
directory for Puppetfile modules, but if a relative path was used it was
relative to the current working directory. As of 1.4.0 a relative moduledir
setting will be expanded to the Puppetfile, which should make it much easier to
use a directory other than 'modules' for the Puppetfile installed modules.
(GH-68) Add alias for r10k deploy display
The r10k deploy display subcommand had unfriendly syntax, and now has an
alias of r10k deploy list.
(GH-92) Support mod 'owner/module' for Git and SVN modules
The original implementation of Git and SVN based modules assumed that the module
name was only the name component, and did not include the owner component of the
module. Because of this the full module name was added to the path, which meant
that a Git module or SVN module called foo/bar would be created as
$moduledir/foo/bar, after which r10k would check for stale modules, see a
module called foo, and delete it.
This has now been corrected so that all modules may have an owner and name, and only the name will be used when constructing the module path.
Issues also closed as part of GH-92:
- GH-78
(GH-96) Provide output from Git command failures
When r10k encounters an error while running a command, it will always log the failed command and the output of the command. This should make it dramatically easier to diagnose issues with the underlying commands used by r10k without having to re-run the failing r10k command with a myriad of command line flags.
Issues also closed as part of GH-96:
- GH-46
- GH-94
(GH-121) Support for calling an arbitrary script after deployment
Users can now specify a postrun setting in r10k.yaml to run an arbitrary
command after an environment deployment finishes. The current implementation is
fairly simple and isn't passed additional information about the deployment and
is mainly meant to notify other services that the deployment has completed.
Future versions of r10k will provide more information to this command so that
more complex actions can be taken on success and failure.
(GH-155) Allow SVN credentials to be added for modules/environments, disallow interactive SVN
When interacting with SVN modules, r10k could run SVN commands that could try to prompt the user for input but were not provided access to stdin. R10k is meant to be non-interactive so credentials need to be provided in some manner, but cannot be provided directly by the user.
As of 1.4.0 SVN environments and modules can supply a username and password to
be used when interacting with the SVN remote, and SVN commands are run with
--non-interactive to prevent commands from trying to grab stdin.
Note: SVN credentials are passed as command line options, so the SVN credentials may be visible in the process table when r10k is running. If you choose to supply SVN credentials make sure that the system running r10k is appropriately secured.
(GH-169) Perform non-blocking reads on stderr in Subprocess
When using SSH with a Control Master, the first time an SSH connection is launched it will persist and will hang onto stderr from the parent process. R10k was using blocking IO to read from child processes and assumed that the file descriptors would be closed when the launched process exited, and the persistent SSH process would cause r10k to hang.
The subprocess code has been updated to perform nonblocking reads of stderr; when the child process exits it assumes that even if stderr is held open nothing else should be written to it, drains the buffered pipe content and returns with that.
This is working around buggy behavior in SSH, but this problem doesn't look like it'll go away so the best course of action is to incorporate this fix into downstream.
(GH-180) Don't leak FDs when executing subcommands
R10k was not explicitly closing all file descriptors, and on Ruby 1.8.7 these would not be closed by the garbage collector, causing file descriptors to leak. This has been fixed and all file descriptors should be closed after each subprocess is invoked.
Major thanks to Jeremy Asher for discovering and fixing this.
(GH-193) Don't purge environments if environment sources can't be fetched
The original behavior for deploying environments with r10k was to fetch sources, deploy environments from those sources, and then clean up any orphaned environments. If a source had been fetched before but could not be reached then r10k would use the previously fetched branch information to update environments. In normal cases this would provide a reasonably robust failure mode.
However, this meant that if no sources had been be fetched or the source remote information was typoed, r10k would have no source information, could enumerate no environments, and then enter HULK SMASH mode where it would delete all unmanaged environments - AKA everything. This is an uncommon failure mode but could bite environments that were setting up r10k for the first time.
To resolve this issue, r10k will try to fetch all sources and if any source fails to be fetched then r10k will abort the entire deployment. This means that r10k will fail very early before any changes have been made which is a safe time to fail. If the errors were transitory then r10k can be run again, and if the failures are permanent then it's hard to safely update anything and extremely dangerous to try to delete any environments.
(GH-202) Different environments cannot manage the same path
R10k allowed for multiple sources to create environments but did not have semantics for environments from different sources managing the same path. If this happened, the resulting behavior would be undefined and could do any number of strange things. The approach to this was by convention and prefixing was recommended to avoid this, but it's still not a great approach.
This was resolved by looking for environment collisions before deploying; if collisions exist then r10k will abort the deployment.
Note: The commit that fixed this referenced GH-123 instead of GH-202.
(GH-214) Rescue SyntaxError and LoadError when evaluating Puppetfiles
Since Puppetfiles are just a Ruby DSL it's possible to have multiple Puppetfiles
that are aggregated by using require or load. However these methods raise
exceptions that don't descend from StandardError, so the normal error handling
would not catch them, so a malformed Puppetfile could catastrophically crash
r10k.
Because the Puppetfile is not an actual source file in the conventional sense we can recover from these errors and continue, so r10k will now recover from these errors. If a SyntaxError or LoadError is raised while evaluating a Puppetfile, r10k will rescue them and wrap them in an R10K::Error and then raise that, which can be caught and handled as a normal, non-critical exception.
(GH-221) Only deploy modules once for each environment deploy
If an environment was deployed for the first time and --puppetfile was
specified, the initial creation would create the environment and then deploy
modules, and then the modules would be deployed again because --puppetfile was
separate code. The deploy logic has been refactored to consider --puppetfile
being passed or the environment being created to be equivalent and will only
deploy modules once.
R10K::Environment::Base#sync is no longer recursive; callers are expected to
handle recursive updates if needed.
The R10K::Action namespace has been added to provide an API to r10k
functionality. These should be used by any code that wants to interact with r10k
and should provide an alternative to shelling out to r10k or using R10K::CLI
directly. The individual action objects should be called through
R10K::Action::Runner so that application setup and teardown is handled.
Thanks to the following contributors for their work on this release:
- Robert Nelson for his work on documenting
workflows and best practices with r10k and adding convenience aliases for
r10k deploy display - Wolf Noble for homogenizing markdown file extensions
- Thomas Bartelmess for updating the required version of cri
- Theo Chatzimichos for correcting the license format to conform to SPDX
- Richard Raseley for writing a quickstart guide for r10k
- Matthew Haughton for fixing documentation typos
- Markus Frosch for fixing a regression in how duplicate environments are checked for, before the regression was ever released. Good catch!
- Jeremy Asher for fixing file descriptor leaks and hanging on open file descriptors in the Subprocess module. Great sleuthing on this!
- Guzman Braso for adding Debian support and fixing some bugs in the quickstart documentation.
- David Schmitt for fixing markdown syntax errors so that link URLs would render properly in GitHub
- Christophe Bliard for fixing the Puppetfile moduledir setting to treat relative paths as relative to the Puppetfile
- Christoph Föhrdes For fixing a copy/paste error in the Puppetfile documentation
In addition to those who contributed to the code base, thanks to all those that reported and commented on issues; user input makes it much easier to make r10k a better project!
2014/11/13
(GH-212) Force use of json_pure on Ruby 1.8.7
Ruby 1.8.7 does not ship with a JSON library, so r10k has depended on json_pure to ensure that there's always a JSON library available. However there is a quirk in multi_json in how it probes for JSON implementations and may load the wrong gem, which percolates up and breaks the JSON parsing code used when querying for forge module versions. To resolve this, json_pure is always used on ruby 1.8.7.
2014/09/14
This bugfix release incorporates all fixes added in 1.2.4.
2014/09/11
(GH-178) Failing to fetch a source git repo can wipe out environments
When updating Git sources at the beginning of a deployment, if the fetch was interrupted r10k could cache an empty list of environments. This could cause r10k to remove all environments for that source. This was due to a method directly using a value that was supposed to be lazily evaluated and memoized. It has been fixed so that even if r10k cannot fetch a source it will still be able to deploy modules for environments, clean up removed environments, and correctly remove unmanaged environments.
(GH-186) Rescue SyntaxError when checking Puppetfile syntax
When a Puppetfile with invalid syntax is parsed it raises a SyntaxError,
and the r10k puppetfile check code was not specifically handling that.
Thus when checking an invalid file r10k was actually crashing and not
gracefully handling the error. This was fixed to cleanly rescue the SyntaxError,
optionally print stacktraces, and print an all ok message on success.
2014/07/27
This bugfix release incorporates all fixes added in 1.2.3.
2014/07/16
(GH-161) Deployments fail where a branch has \W in the git branch name
In 1.3.0 environment naming was partially reworked to allow better handling of per-environment deployment, but unfortunately this caused a regression where environments would be differently named in 1.3.0. This fix changes the environment deployment on a per-name basis to use the normalized name instead of the raw Git branch name.
This bugfix release also incorporates all fixes added in 1.2.2.
Thanks to Chris Spence for his work on this release.
2014/06/07
R10k can now dynamically generate enviroments based on SVN repositories. SVN repositories must SVN repositories must conform to the conventional SVN repository structure with the directories trunk/, branches/, and optionally tags/ in the root of the repository. The trunk/ directory is specifically mapped to the production environment, branches are created as environments with the name of the given branch.
Please note that since SVN support for environments should be considered preliminary and may still have some kinks to be worked out, so use it with caution in production.
When deploying modules with r10k deploy module <modules>, users can specify
the -e <environment> flag to update modules in a single environment.
Git sources can now tune how r10k behaves when it encounters a git branch with a non-word character. Valid values are 'correct_and_warn' which emits a warning and sanitizes the environment name, 'correct' which silently corrects the environment, and 'error' which emits an error and ignores the environment.
There are a lot of internal APIs in r10k that need to be improved or overhauled, and making changes in a backwards compatible manner has been impeding development on a number of important features. There's no indication that there are any consumers of the r10k internal APIs, and if that's the case then r10k doesn't help anyone by maintaining SemVer for its internal APIs.
As of 1.3.0, r10k is dropping guarantees about API compatibility for most of
the core functionality. The R10K::Task* classes were designed to be the
primary interface for external use and those will remain backwards compatible
for 1.x. However any code around configuration parsing, deployments, sources,
environments, and modules may have API changes in minor versions.
That being said, if you are using any of these APIs and you experience breakage, you're not out of luck. If an API change actually does affect you please report it as a bug and those specific APIs can probably be fixed up. Hopefully this will ease development of r10k while not making the lives of external developers too painful.
This allows the groundwork for allowing users to implement plugins for sources and environments. A real API specification for sources and environments has been started, and sources and environments can be defined at runtime. In the long run r10k will add a plugin system for loading additional code from Rubygems and other sources, so r10k will be extensible without requiring modifications to the source.
2014/09/14
(GH-188) Call puppet module tool install with --force for downgrades
2014/07/27
(GH-173) Fixed a bug with Ruby 1.8.7 with Pathname objects.
2014/07/16
(GH-165) r10k puppetfile only consumes handled command line options.
Previously, passing -v or other commands when running r10k puppetfile *
could result in this error:
r10k puppetfile install --help --trace
Error while running: #<RuntimeError: Unrecognized options: help>
This was due to overly greedy code passing in all options from the command line to the TaskRunner. This has been fixed so only known options are passed along, and options that aren't relevant (such as :verbose) will be ignored.
(GH-158) Log levels are now documented in the command line --help pages.
(GH-137) Git remotes are now correctly updated.
A regression in the Git remote handling meant that git remotes would never be properly updated when switching Git environments and modules from one remote to another, and the git alternates file was never updated properly. This has been fixed so that when the Git remote is updated, all references to the remotes and alternates will be updated.
(GH-163) All Git tags are deleted when switching Git remotes
Git tags cannot necessarily be transferred from one Git repository to another, so when a Git repo has its remotes changed all tags are deleted to prevent stale tags from overwriting tags from the new repo.
2014/04/21
(GH-93) r10k deploy subcommands now respect the --help flag.
(GH-100) The addition of a faster command execution library was POSIX centric, but there were a number of users that were running r10k on Windows, which brike their environments. Support for Windows has been re-added and Windows is now a supported platform for using the r10k puppetfile commands. Please note that r10k now requires Ruby 1.9.3 on Windows to function.
Thanks to Sam Kottler and Daniel Dreier for their their work on this release.
2014/03/07
Preliminary support for Puppetfile modules from SVN sources. SVN repositories can track the latest available revision or may be pinned to a specific revision.
Forge modules can now track the latest available version. This can be enabled by
setting the module version to :latest.
Git based Puppetfile modules can now be specified as branches, tags, and commits. When tags and commits are specified r10k can perform optimizations when updating the given repositories to reduce network accesses.
Command execution has been greatly improved. The old library for executing commands (systemu) had very high overhead and was 50 - 100 times slower than %x[] or fork/exec. It's been replaced with a custom process execution implementation.
Modules can swap out sources. When an existing module is changed from Forge to Git, for instance, the existing module will be removed before the new module is installed. (GH-30)
2014/02/27
Git repositories were not tracking their upstream remotes, repos should now properly update upstream changes.
Git reference clones now fetch their cache remotes immediately after the initial clone.
2014/02/08
Release Candidate 1 for 1.2.0
2014-02-24
This is a backwards compatible bugfix release.
- (GH-90) Multiple environments with the same name but with different sources were previously colliding and some environments were being ignored. This has been fixed and all environments should be deployed when updates are run.
Thanks to the following contributors for their their extraordinary patience and help in for chasing down GH-90:
- Andreas Ntaflos (antaflos)
- Igor Galić (igalic)
2014-01-26
This is a backwards compatible maintenance release.
- (GH-82) Added all git managed files, including README.markdown, CHANGELOG, and LICENSE to the gemspec for better compatibility with non-gem packages.
2014-01-06
This is a backwards compatible maintenance release.
- If Puppet and r10k are required in the same namespace, it's possible for the vendored copy of SemVer to conflict with the Puppet version. This was fixed by renaming the copy vendored in r10k and putting it under a namespace.
2013-12-11
This is a backwards compatible bugfix release.
- (GH-48) Environment prefixing always defaults to off. Users were already using r10k with multiple sources but in different directories, and prefixing breaks this behavior. Since this was a backwards incompatible change this has to be rolled back.
- (GH-64) Multiple sources in a single directory no longer purge each other.
Thanks to the following contributors for their help in 1.1.1:
- Stig Sandbeck Mathisen
- Gabriel M Schuyler
2013-09-30
This is a backwards compatible bugfix and feature release.
(GH-35) Puppetfiles can now specify a path to the moduledir, instead of assuming
'/modules'. It can be set with the Puppetfile moduledir directive. Note that
this is not compatible with librarian-puppet.
(GH-53) Multiple environment sources can now be specified in a single
directory. When multiple sources are specified, each environment will be
prefixed with the source name. This can be enabled and disabled with the
source prefix option.
(GH-45) Documentation has been greatly expanded.
(GH-56) New subcommand: r10k puppetfile check allows you to validate the
syntax of a Puppetfile.
(GH-66) Initial clones use git checkout when switching to a new branch
instead of just git reset; without this change it would look like the wrong
branch was checked out.
(GH-59) r10k can now pull from Pulp repositories for Forge based modules.
(GH-70) Handle unset HOME - in case that HOME is unset, assume that the current
user is root. This mainly occurs when r10k is being run as the prerun command
under Puppet.
The method mocking framework has been switched from mocha to rspec-mocks. Mocha is notoriously bad about breaking changes between versions and rspec-mocks is very robust, so Mocha has been ripped out and dropped as a dependency.
Rspec expectations now use the expect(thing).to syntax instead thing.should
A quasi settings framework has been extracted to make application settings less bad. In the long term a general application framework will be extracted from r10k to handle generic 'application' problems like this, but for now the settings framework is the way to handle singleton data.
R10K:Git::Cache object memoization has been extracted into a standalone class instead of being grafted onto the class. All hail the single responsibility principle!
R10K::Module code has been refactored. There's now a real base class instead of a hacky mixin with some metadata magic on top.
Thanks to the following contributors for their help in 1.1.0:
- Alex Linden Levy
- Abhay Chrungoo
- Adam Vessey
- Chuck Schweizer
- Elias Probst
- Greg Baker
- Jochen Schalanda
- Theo Chatzimichos
2013-05-30
This is a backwards incompatible bugfix and feature release.
The configuration file format of 0.0.9 should be compatible with 1.0.0, and any issues with that should be considered a bug.
A longstanding issue was confusion between symbols and strings in r10k.yaml (GH-18). To resolve this, symbols and strings will be treated equally and should produce the same behavior. In the long run, symbols will probably be deprecated for the sake of conformity.
A number of commands have been renamed. They still but will emit a deprecation
warning and will redirect to the new command implementation. The only
exceptions is the are the r10k environment cache and r10k environment stale
commands, but they were pretty much useless anyways.
Log level verbosity can now be specified by level name instead of level number.
If --verbose is passed without a level, it will set the log level to info.
r10k can be used to deploy modules from a standalone Puppetfile. See
r10k puppetfile for more information.
Modules without a version in the format of 'foo/bar' will be assumed. (GH-21)
r10k handles versioning according to SemVer; since this is a major release this is a backwards incompatible API change. It's unlikely that this has had any extensions written on top of it, but if you have, then heads up. However, all versions of 1.x should be backwards compatible.
A number of bugs were due to underlying architecture flaws. Part of 1.0.0 has been a significant architectural overhaul, so on top of all of the above changes there should be a lot of other bugs that have been fixed.