SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

tensorrt int8 量化yolov5 onnx模型

RepVGG TensorRT int8 量化，实测推理不到1ms一帧！

a simple pipline of int8 quantization based on tensorrt.

👀 Apply YOLOv8 exported with ONNX or TensorRT(FP16, INT8) to the Real-time camera

TensorRT Int8 Python version sample. TensorRT Int8 Python 实现例子。TensorRT Int8 Pythonの例です

A LLaMA2-7b chatbot with memory running on CPU, and optimized using smooth quantization, 4-bit quantization or Intel® Extension For PyTorch with bfloat16.

quantization example for pqt & qat

MT-Yolov6 TensorRT Inference with Python.

mxnet GluonCV quantization binary ternary models

development quantization framework