AutoCompress is a hardware-guided compression framework that uses differentiable architecture search and serving-engine cost models to produce heterogeneous LLM architectures meeting latency, throughput, and memory constraints while preserving accuracy.
LLM pruning must preserve quality while meeting hardware- and workload-specific constraints on latency, throughput, and memory. Existing pruning workflows often target parameter counts, requiring practitioners to manually select model sizes that meet latency, throughput, and GPU memory limits. However, parameter count alone does not predict these metrics, which depend on the model architecture, workload, hardware, and serving engine. We introduce AutoCompress, a hardware-guided compression framework that directly incorporates such constraints into differentiable architecture search. AutoCompress jointly optimizes global and per-layer structure to produce heterogeneous architectures, guided…
LLM pruninghardware-guided compressiondifferentiable architecture searchheterogeneous architecturesglobal and per-layer structureserving-engine cost modelsend-to-end latencythroughputGPU memory limitslong-context local coding
- Upvotes
- 6
- Authors
- 7
- Published
- Nov 5, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.