Why We No Longer Evaluate on SWE-bench Verified
Scale AI has dropped SWE-bench Verified, citing contamination and flawed tests. Its analysis found training leakage inflates scores and recommends SWE-bench Pro instead.
Topic
Data Contamination
Topic
Topic
Topic
Scale AI has dropped SWE-bench Verified, citing contamination and flawed tests. Its analysis found training leakage inflates scores and recommends SWE-bench Pro instead.