Pinned Loading

1. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

2. ray-project/ray ray-project/ray Public

   Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

   Python 32.2k 5.5k

3. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 23.4k 3.3k

4. triton-inference-server/server triton-inference-server/server Public

   The Triton Inference Server provides an optimized cloud and edge inferencing solution.

   Python 7.8k 1.4k

5. NVIDIA/TensorRT-LLM NVIDIA/TensorRT-LLM Public

   TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

   C++ 7.6k 831

6. triton-inference-server/model_navigator triton-inference-server/model_navigator Public

   Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.

   Python 169 24