Project Description:
This is a test framework for the bias bounties project.
Getting Started as a Bounty Hunter:
If you are interacting with this codebase as a "bounty hunter", you'll need to have a way to run Jupyter notebooks. The easiest way to do this is to download Anaconda, which will also manage all of your python packages for you. See here for installation instructions: https://docs.anaconda.com/anaconda/install/index.html.
There are three Jupiter notebooks contained in the folder, which were used to generate the plots in the paper.
-
Algorithm-Visualizations.ipynb generates a series of bounty hunters who submit models trained on various demographic features of the ACS folktables dataset, and then feeds these into the updater algorithms in a random order, and visualizes performance across the different groups.
-
CSC-experiments.ipynb runs an implementation of the CSC algorithm on the ACS folktables dataset.
-
CSC-batch.ipynb is a wrapper for running many CSC updaters over many different datasets and visualizes their performance.
Getting Started as a Code Developer:
If you are running this code because you want to work with it directly, here is a breakdown of the different files.
-
updater.py: This is where the code lives that integrates submitted models into the existing model. It also contains helper functions that measure group errors. The updater here assumes that the groups are known in advance in order to optimize for the plots generated in the paper.
-
cscUpdater.py: This is where the CSC model is generated and its updater (which does not assume that the groups are known in advance)
-
verifier.py: This contains code for verifying submissions by a bounty hunter against a holdout dataset.
-
model.py: Our prediction model that incorporates the bounty hunters' inputs is a modified decision list, which is stored as an object. This file defines all of the attributes and methods of that object.
-
acsData.py: Wrapper code that cleans and loads in the ACS folktables data.
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.