GEPA
Reflective Prompt Evolution Can Outperform RL
ICLR 2026 Oral (Stanford/Berkeley/UT Austin). Treats execution traces as readable signal, evolving prompts via NL reflection + Pareto selection — 35× fewer rollouts than RL.
On My Radar
Week of Jun 8, 20268 finds
Tools
- karpathy/autoresearchKarpathy's experiment in automating research loops with LLMs. Worth a look for harness-design ideas.Watch Later
- lm-evaluation-harnessEleutherAI's de-facto standard harness for benchmarking LLMs across hundreds of tasks. The reference tool when you need reproducible eval numbers.To Test
- GEPA (reference implementation)The official MIT-licensed Python impl of the GEPA optimizer — evolve prompts, code, and arbitrary text artifacts against your own eval.To Test
- DeepEvalPopular open-source LLM eval framework — pytest-style assertions, RAG/agent metrics, and built-in GEPA-style prompt optimization.To Test
Libs
- inkeep/agentsOpen-source framework for building and running agents. Relevant reference point for agent-harness architecture.To Test
- DSPy — GEPA optimizerDSPy's first-class GEPA integration. The most ergonomic way to run GEPA over a declarative LLM program with a defined metric.To Test
- LLMLinguaMicrosoft's prompt-compression toolkit — compresses prompts up to 20× with minimal performance loss. The serious counterpoint to anecdotal SPR-style compression.To Test