I let an AI build and ship a product. Its tests passed 6/6, then failed 5 of 8 real cases.

Its test suite passed six out of six. Then I pointed it at thirteen real crash reports from my own server and it got three of eight. I asked Claude to build and ship a commercial product with as little help from me as possible — write the code, write the tests, set up the storefront, publish it.