Show HN: Parsimony – an open-source benchmark for how much code AI models add

Hacker News: Front Page ·
Startups

Existing benchmarks measure solve rate and token output/cost, but don't punish excess code, which makes a codebase harder to work with in the long term. I wanted to measure this since it matters most for long-term projects. Method: it takes the patches that benchmarks already publish (DeepSWE, SWE-bench Verified, SWE-bench Live) and counts "coding units": syntax elements like statements, calls,…

Read original article on Hacker News: Front Page →

Related news