runanywhereai/trnrt
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
- Stars1099
- LanguagePython
- LicenseApache-2.0
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
TrnRT is an open inference engine for large language models on AWS Trainium. It serves an OpenAI-compatible API from one trn2.48xlarge, runs decode as NKI megakernels (about one kernel per layer half), and keeps a prefix cache across device and host memory. GLM-5.3-Flash is the first supported model; more open models are on the way.
How we built it, with the AgentX results: Serving GLM-5.3-Flash on one Trainium2 box with NKI layer megakernels.
Description and README excerpt from RunanywhereAI/trnrt on GitHub ↗.
runanywhereai/trnrt was first seen on GitHub · Rising today on Nov 11, 2026. Since then it has appeared in 93 checks on 3 different days. Its best position was #10 of 30, reached on Nov 11, 07:15 UTC. It dropped off the list after Nov 13, 06:30 UTC.
Position on GitHub · Rising today at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by GitHub.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 13, 06:30 UTC, after 3 days of trending.
#10 on GitHub · Rising today, on Nov 11, 07:15 UTC. Trendoday checks this list every 60 minutes.
It was first seen on Nov 11, 2026 and has been on the list on 3 days (93 checks) since.
Mostly Python; it is licensed under Apache-2.0. It has 1099 stars and 9 forks.