Bad benchmarks and evals: Senior SWE-Bench, napkin math, and winter tires

2 comments

• stephantul54 minutes ago
  The (relevant) segue into tires elevates this post so much. I don’t understand why, but it does
  • vintagedave24 minutes ago
    Maybe that it's an example of the same pattern in a real-world, non-digital area? It makes it feel more grounded.
• alexpran2 days ago
  [flagged]