About this paper
Challenging repository-level benchmarks remain limited for many programming language ecosystems, where filtering scarce candidates with ambiguous descriptions or defective tests can discard valuable engineering problems. We introduce Longtail-SWE, a benchmark of 202 issue-resolution tasks across 13 languages with limited coverage in existing repository-level benchmarks: Clojure, Dart, Elixir, Erlang, F#, Haskell, Julia, Kotlin, Lua, OCaml, R, Scala, and Swift. Its construction uses a solver--judge--repairer pipeline: a solver attempts each candidate task, and a judge reviews the task together with the solver's trajectory and test results to guide revisions to tests and task descriptions.…
- Upvotes
- 13
- Authors
- 3
- Published
- Nov 6, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
Longtail-SWE: High-Quality SWE Tasks for Underrepresented Languages was first seen on Hugging Face · Daily papers on Nov 10, 2026. Since then it has appeared in 14 checks on 2 different days. Its best position was #5 of 30, reached on Nov 11, 05:28 UTC. It dropped off the list after Nov 11, 17:28 UTC.