This is a library that allows for fault injection testing by intercepting
system calls.

It will be configured by setting environment variables.

INSTALL:
To compile run:
autoreconf --install
./configure
make

# Ensure backtrace symbols are available in WiredTiger. I built with:
../configure CFLAGS="-ggdb -O0" LDFLAGS="-rdynamic" --enable-strict --enable-python --with-builtins=snappy,zlib

CHANGING THE CODE:

Most of the c-code is autogenerated from templates, since each intercepted
function shares a lot of code. The correct method for updating the source 
code is to edit src/faultinject.c.in or dist/gen_source.py, and run
``(cd dist && python gen_source.py)`` to auto-generate src/faultinject.c

TODO:
* Add python generator script to makefile
* Implement generic environment variable handling

* It is not obvious how to make this work without LD_PRELOAD. LD_PRELOAD
  causes difficulties if you only want to intercept (or alter) calls that come
  via one of several libraries. For example if using WiredTiger via the Python
  API, it isn't desirable to alter the behavior of the Python functions.
  It is possible to build WiredTiger against faultinject and have faultinject
  load when WiredTiger is dynamically loaded into Python, but by then the
  symbols aren't overridden. It may be possible to forcibly overide the
  symbols in the faultinject static initializer - but I suspect that would also
  override them for the Python executable as well.


To run a WiredTiger python test case with the faultinject library, the following
is a good starting template:

rm -rf ./res && mkdir -p ./res
for i in $(seq 1 100);
do
	env FAULTINJECT_FAIL_COUNT=$i FAULTINJECT_LIBRARY_NAME=__wt LD_LIBRARY_PATH=.libs LD_PRELOAD=~/work/faultinject/faultinject/.libs/libfaultinject.so PYTHONPATH=../lang/python:lang/python:../test/suite timeout python ../test/suite/run.py -v 2 test_cursor01 > res/t$i.out 2>&1
done

A description of the components of the above command:

FAULTINJECT_FAIL_COUNT -
   The number of operations to pass to the default implementation prior to failing
   subsequent operations.
FAULTINJECT_LIBRARY_NAME -
    A string that is used to filter out callers we are interested in. Must be
    present in the call stack of callers we want, and not in those we don't.
LD_PRELOAD -
    The faultinject library to be used during the execution
timeout -
    Some WiredTiger operations will go into infinite loops if operations
    continually fail. Use this to send kill signals to those processes.