README_WHEEL.md

Python Wheel

Build

The wheel adds dependencies including optional dependencies, except flash-attention, wiki-processing, metric, and training. To build do:

python setup.py sdist bdist_wheel

To install the default dependencies do:

pip install dist/h2ogpt-0.1.0-py3-none-any.whl

replace 0.1.0 with actual version built if more than one. To install additional dependencies, for instance for faiss on GPU, do:

pip install dist/h2ogpt-0.1.0-py3-none-any.whl
pip install dist/h2ogpt-0.1.0-py3-none-any.whl[FAISS]

once whl file is installed, two new scripts will be added to the current environment: h2ogpt_finetune, and h2ogpt_generate.

The wheel is not required to use h2oGPT locally from repo, but makes it portable with all required dependencies.

See setup.py for controlling other options via extras_require.

Run

from h2ogpt.generate import main
main()

See src/gen.py for all documented options one can pass to main(). E.g. to start LLaMa7B:

from h2ogpt.generate import main
main(base_model='meta-llama/Llama-2-7b-chat-hf',
          prompt_type='llama2',
          save_dir='save_gpt7',
          score_model=None,
          max_max_new_tokens=2048,
          max_new_tokens=1024,
          num_async=10,
          top_k_docs=-1)

Checks

Once the wheel is built, if you do:

python -m pip check

you may see:

h2ogpt 0.1.0 has requirement numpy==1.24.3, but you have numpy 1.23.5.
h2ogpt 0.1.0 has requirement pandas==2.0.2, but you have pandas 1.5.3.

but that is expected.