The End-to-End LLM (Large Language Model) Bootcamp is designed from a real-world perspective that follows the data processing, development, and deployment pipeline paradigm. Attendees walk through the workflow of preprocessing the SQuAD (Stanford Question Answering Dataset) dataset for Question Answering task, training the dataset using BERT (Bidirectional Encoder Representations from Transformers), and executing prompt learning strategy using NVIDIA® NeMo™ and a transformer-based language model, NVIDIA Megatron. Attendees will also learn to optimize an LLM using NVIDIA TensorRT™, an SDK for high-performance deep learning inference, guardrail prompts and responses from the LLM model using NeMo Guardrails, and deploy the AI pipeline using NVIDIA Triton™ Inference Server, an open-source software that standardizes AI model deployment and execution across every workload.
This content contains three Labs, plus an introductory notebook and two lab activities notebooks:
- Overview of End-To-End LLM bootcamp
- Lab 1: Megatron-GPT
- Lab 2: TensorRT-LLM and Triton Deployment with LLama-2-7B Model
- Lab 3: NeMo Guardrails
- Lab Activity 1: Question Answering task
- Lab Activity 2: P-tuning/Prompt tuning task
The tools and frameworks used in the Bootcamp material are as follows:
The total Bootcamp material would take approximately 8 hours and 45 minutes. We recommend dividing the material's teaching into two days, covering Lab 1 in one session and the rest in the next session.
To deploy the Labs, please refer to the Deployment guide presented here
This material originates from the OpenHackathons Github repository. Check out additional materials here
Don't forget to check out additional Open Hackathons Resources and join our OpenACC and Hackathons Slack Channel to share your experience and get more help from the community.
Copyright © 2023 OpenACC-Standard.org. This material is released by OpenACC-Standard.org, in collaboration with NVIDIA Corporation, under the Creative Commons Attribution 4.0 International (CC BY 4.0). These materials may include references to hardware and software developed by other entities; all applicable licensing and copyrights apply.