Official PyTorch implementation of "GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance" (ICML 2025)
-
Updated
Apr 13, 2026 - Python
#
llm-quantization
Here are 20 public repositories matching this topic...
Open quantization tooling for TurboQuant-style low-bit LLM releases, stock GGUF deployment, and Apple Silicon runtime experiments.
apple toolkit tooling tensor quantization apple-silicon llm llm-inference gguf llm-quantization turboquant
-
Updated
May 5, 2026 - Python
[CAAI AIR'24] Minimize Quantization Output Error with Bias Compensation
post-training-quantization llm-compression output-error-optimization bias-compensation llm-quantization
-
Updated
Mar 12, 2025 - Python
A more deep research about TurboQuant algorithms
-
Updated
Apr 6, 2026 - Python
A high-performance, memory-efficient healthcare framework that deploys fine-tuned Large Language Models (LLMs) on edge devices. Multi-agent system to provide personalized diagnostic reasoning, health education, and dietary planning.
-
Updated
Sep 7, 2025 - Jupyter Notebook
Ternary Quantization for LLMs: Implement balanced ternary (T3_K) weights for 2.63-bit quantization—the first working solution for modern large language models.
-
Updated
Nov 29, 2025 - C++
Shift-based post-training quantization analysis for LLMs (ShiftQuant paper)
-
Updated
Mar 28, 2026 - Python
Production-grade LLM quantization, benchmarking, and edge deployment toolkit. Supports bitsandbytes INT8/INT4, GPTQ (Hessian calibration), AWQ (activation-aware), and GGUF (Q2_K–Q8_0). Four-dimensional benchmarking: perplexity, TPS/TTFT, VRAM profiling, and LLM-as-Judge quality scoring. RTX 5090 Blackwell sm_120 ready.
python benchmarking transformers pytorch nvidia-cuda model-compression onnx edge-ai huggingface streamlit blackwell post-training-quantization awq llama-cpp gptq bitsandbytes ollama gguf llm-quantization rtx-5090
-
Updated
Jun 14, 2026 - Python
LLM quantization project built around `llama.cpp` + `Ollama` + `GGUF`
-
Updated
Mar 22, 2026 - Python
Let me make GGUF files quickly
-
Updated
Jun 4, 2025 - Python
Paired capability-level GGUF quantization fragility benchmark across Qwen2.5-3B and SmolLM2 1.7B.
-
Updated
Jun 25, 2026 - Python
pytorch quantization dequantization 2-bit hugging-face hugging-face-hub llm-quantization torch-quantization
-
Updated
Jun 26, 2024 - Jupyter Notebook
Local & lightweight LLM inference runtime in C++ with support for GGUF & quantization
open-source library opensource cpp17 mit-license cpp-library cpp-lib llm cpp-module llm-inference llm-local llm-tools llm-framework gguf llm-library llm-quantization llm-integration lightweight-llm
-
Updated
Feb 27, 2026
GWIQ-Atlas: is a brain-atlasing and model-interpretability suite that combines per-layer census, compliance behaviour tracing, SAE features, and quantization analyses for LLMs.
python transformers quantization sae sparse-autoencoder brain-atlas huggingface activation-analysis mechanistic-interpretability llm-analysis llm-quantization llm-interpretability model-atlas feature-census
-
Updated
Jun 30, 2026 - Python
PentaNet extends BitNet's ternary quantization to pentanary {-2,-1,0,+1,+2}, improving perplexity by 6.4% at 124M params while preserving zero-multiplier arithmetic.
-
Updated
Apr 17, 2026 - Python
A high-performance inference engine optimized for deploying quantized LLMs on edge devices. Focuses on SIMD optimizations and memory management.
-
Updated
Apr 9, 2026 - C++
OpenVINO Model Manager — desktop GUI for Intel Arc
linux-gui openvino intel-gpu nncf pyqt6 intel-arc llm-tools optimum-intel llm-quantization llm-conversion
-
Updated
Jun 22, 2026 - Python
Implemented and fine-tuned BERT for a custom sequence classification task, leveraging LoRA adapters for efficient parameter updates and 4-bit quantization to optimize performance and resource utilization.
nlp lora quantization data-augmentation nlp-augmentation llm qlora llm-fine-tuning peft-fine-tuning-llm llm-quantization
-
Updated
Dec 30, 2024 - Jupyter Notebook
Implementation of advanced Natural Language Processing architectures and optimization techniques, built from scratch. The projects focus on understanding the internal mechanics of Transformers, LLM efficiency through quantization, and scaling via Mixture-of-Experts (MoE).
load-balancing mixture-of-experts transformer-architecture positional-encoding llm-fine-tuning llm-quantization
-
Updated
Jan 8, 2026 - Python
Improve this page
Add a description, image, and links to the llm-quantization topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-quantization topic, visit your repo's landing page and select "manage topics."