Rebase TokuDB Patches

Rebase TokuDB Patches

This wiki describes one method to add TokuDB patches to the latest MySQL or MariaDB.

Step 1: Import the new MySQL version into our repository. For example, we import MySQL 5.5.37 into the mysql-5.5 repository.

git clone [email protected]:Tokutek/mysql-5.5
cd mysql-5.5
git checkout --orphan mysql-5.5.39
git rm -rf .
tar xzf ~/mysql-5.5.39.tar.gz
git add --all --force
git commit

Step 2: Get the reverse commit id list from the previous version. We will apply the commit id's from this list later.

git log --reverse >../commits.log
cat ../commits.log | egrep ^commit | awk '{print $2}' >../commits

Step 3: Edit out the commit id's that already occurred.

Step 4: Build a debug environment.

make.mysql.debug.env.bash --mysql=mysql-5.5.39

The build probably will fail because the configuration options in hatoku_defines.h depend on patches that do not exist yet.

Step 5: Hack on hatoku_defines.h until TokuDB compiles without any patches. We may need to add #define TOKU_USE_DB_TYPE_UNKNOWN 1 since handler.h may not know about TokuDB yet. At various points of the rebase, we can turn compile time flags on.

Step 6: Apply commits from the previous MySQL version onto the new MySQL version. The follow script cherry picks commit id's from the commit list onto the new MySQL version and runs builds the code.

$ cat apply.bash
#!/usr/bin/env bash

if [ $# != 1 ] ; then exit 1; fi
mysql=$1

cp commits new.commits
for c in $(cat commits | egrep -v ^#); do
    echo $c
    if [ $c = stop ] ; then exit 0; fi
    sed -e"1,\$s/\($c\)/#\1/" -i new.commits

    pushd $mysql
    if [ $? != 0 ] ; then exit 1; fi
    echo git cherry-pick $c
    git cherry-pick $c >../cherry.out 2>&1
    if [ $? != 0 ] ; then exit 1; fi
    popd

    pushd build
    echo make -j4
    make -j4 >../build.out 2>&1
    if [ $? != 0 ] ; then exit 1; fi    
    popd
done

apply.bash mysql-5.5.37
echo $?

Sometimes, there are commits that fix bugs in some previous commit. These related commits should be squashed together.

Step 7: Fixup any merge conflicts or build errors. mv new.commits commits and repeat step 6 until all of the commits are applied.

Step 8: Push commits to github.