README.md

Nitra

Nitra is a multi-process, optionally multi-server rspec and cucumber runner that uses forking to reduce memory usage and IPC to distribute builds amongst available CPUs efficiently.

Philosophy

• Nitra attempts to do the simplest thing possible
• Nitra (ab)uses unix primitives where possible
• Nitra doesn't do things that unix already does better (eg. rsync)
• IPC is accomplished via pipes and select
• Forking is used heavily for several reasons
• Running nitra locally should be easy
• Running nitra on a cluster should be easy too (though verbose)
• Config files are a nuisance and should be stuffed into rake files (deals with the verbosity)

Usage

  nitra [options] [spec_filename [...]]
      -c, --cpus NUMBER                  Specify the number of CPUs to use on the host, or if specified after a --slave, on the slave
          --cucumber [PATTERN1,PATTERN2] Full cucumber run, causes any files you list manually to be ignored.
                                         Default pattern is "features/**/*.feature".
          --cucumber-format FORMATTER    Additional cucumber format to use. Nitra will always use the "pretty" format in addition to the format defined here.
                                         Refer to the cucumber documentation for the '--format' flag for more information.
          --cucumber-out FILE            Use with --cucumber-format FORMATTER to output that formatter to a file.
          --debug                        Print debug output
      -p, --print-failures               Print failures immediately when they occur
      -q, --quiet                        Quiet; don"t display progress bar
          --rake-after-runner task:1,task:2,task:3
                                         The list of rake tasks to run, once per runner, in the runner's environment, just before the runner exits
          --rake-before-runner task:1,task:2,task:3
                                         The list of rake tasks to run, once per runner, in the runner's environment, after the runner starts
          --rake-before-worker task:1,task:2,task:3
                                         The list of rake tasks to run, once per worker, in the worker's environment, before the worker starts
          --reset                        r
                                         Reset database, equivalent to --rake-before-worker db:reset
          --slave-mode                   Run in slave mode; ignores all other command-line options
          --slave CONNECTION_COMMAND     Provide a command that executes "nitra --slave-mode" on another host
          --rspec [PATTERN1,PATTERN2]    Full rspec run, causes any files you list manually to be ignored.
                                         Default pattern is "spec/**/*_spec.rb".
          --rspec-format FORMATTER       Additional rspec formatter to use. Nitra will always use the "pretty" formatter in addition to the formatter defined here.
                                         Refer to the rspec documentation for the '--format' flag for more information.
          --rspec-out FILE               Use with --rspec-format FORMATTER to output that formatter to a file.
          --slave-mode                   Run in slave mode; ignores all other command-line options
      -e, --environment ENV              Set the RAILS_ENV to load
      -h, --help                         Show this message

Getting started

First things first add nitra to your Gemfile:

gem 'nitra'

Then run your specs locally across your cpu's cores:

bundle exec nitra --rspec

This will just run all your specs using the default number of cpu's reported by your system. Hyperthreaded intels will report a high number which might not be a good fit, you can tune this with the -c option.

Clustered commands run slightly differently. Effectively nitra will fork and exec a command that it expects will be a nitra slave, this means we can use ssh as our tunnel of choice. It looks something like this:

bundle exec nitra --rspec --slave "ssh your.server.name 'cd your/project && bundle exec nitra --slave-mode'"

When nitra --slave command it forks, execs it, and assumes it's another process that's running "nitra --slave-mode".

Running a build cluster

Nitra doesn't prescribe how you get your code onto the other machines in your cluster. For example you can run a git checkout on the boxes you want to build on if it's the fastest way for you. For our part - we've had the most success with rsync.

Our build is run via rake tasks so we end up with a bunch of generated code to rsync files back and forth - here's a basic version that might help get you up and running:

namespace :nitra do
  task :config do
      @servers = [
        {:name => "server1", :port => "66666", :c => "4"},
        {:name => "server2", :port => "99999", :c => "2"},
        {:name => "server3", :port => "77777", :c => "8"},
        {:name => "server4", :port => "88888", :c => "4"}
      ]
  end

  desc "Sync the local directory and install the gems onto the remote servers"
  task :prep => :config do
    @servers.each do |server|
      fork do
        system "ssh -p #{server[:port]} #{server[:name]} 'mkdir -p nitra/projects/your_project'"
        exec %(rsync -aze 'ssh -q -p #{server[:port]}' --exclude .git --delete ./ #{server[:name]}:nitra/projects/your_project/ && ssh -q -t -p #{server[:port]} #{server[:name]} 'cd nitra/projects/your_project && bundle install --quiet --path ~/gems')
      end
    end
    Process.waitall
  end

  task :all => :config do
    cmd = %(bundle exec nitra -p -q --rspec --cucumber --rake-before-worker db:reset)
    @servers.each do |server|
      cmd << %( --slave "ssh -p #{server[:port]} #{server[:name]} 'cd nitra/projects/your_project && bundle exec nitra --slave-mode'" -c#{server[:c]})
    end
    system cmd
  end
end

Copyright

Copyright 2012-2013 Roger Nesbitt, Powershop Limited, YouDo Limited. MIT licence.