On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer
Prefill-to-Autoregressive Cross-model Transfer (PACT), a form of latent communication between language models, combines a larger model's prompt representations with a smaller model's efficient generat
- Upvotes8