The Fatal Flaw
When AI assumes you have all day
I gave eight AIs the same web development task of creating a simple one-page website, using only HTML, CSS, and JavaScript. Each of them received the same prompt, stipulating they had to meet certain constraints, including: dynamic screen size adaption, international accessibility standards, and human-readable code. They also had to have certain content included: photos, testimonials, date and time details of an event, and a registration form. The scenario was fictional but based on real projects I’ve done.
My test scenario was deliberately specific. A knowledgeable but time-strapped volunteer needed a usable website for a fabric swap meet, something that would live for a few weeks or months. They didn’t have time to learn new tools, install libraries, or debug complex setups. They needed the AI to produce something they could actually use.
This is realistic. Most people who reach for AI to build a web page aren’t looking for a technical masterpiece. They’re looking for “good enough” so they can focus on what actually matters: the content. For whatever reasons, they can’t just go to Wix or Wordpress or a similar service and scrape something together from a template.
But several of the AIs I tested didn’t understand that. They produced solutions that assumed the human user had unlimited time and expertise.
The Failures
Createxyz failed completely, I put a summary about their output in my previous article.
Replit also failed here, though dressed up as a benefit. Yes, Replit can produce sophisticated solutions without showing you the code. But that’s also its fatal flaw for this scenario. You need to pay to inspect the code at all. My imaginary volunteer on a tight budget couldn’t easily verify whether accessibility criteria were actually met. Critical features are gated behind a paywall. Want to upload actual photos? That’s behind a paywall too. Want to know what JavaScript framework it’s using? Replit will tell you it chose React, which is complete overkill for a simple form page that will exist for eight weeks.
A deeper problem is that Replit assumes you either don’t know how to read code, or don’t want to learn. Some users love that. I understand the appeal. But it also means you’re locked into trusting someone else’s judgment, with no way to verify, edit, or understand what you’re actually paying for. I’ve seen third-party vendors pull this same move. They make vague claims about their code, then charge extra to fix problems that were never “new scope” at all. Replit’s approach recreates that dynamic with AI.
Meta also failed this criterion, though less dramatically. At least they produced separate HTML and CSS files in the same folder, making it theoretically editable. But the output was minimal to the point of inflexibility. All the sections were centred, and each was a different width (whatever the minimum for the content was). Each section was encased in a <section> element, and each section had an ID assigned, yet these IDs had no CSS attached to them. The volunteer would have had to do quite a bit of CSS work to get something presentable, with no comments to guide them.
Who Got It Right
Gemini and ChatGPT were the only ones who clearly understood the assignment. They produced clean, well-commented code. A casual web creator could pick it up and edit it without a computer science degree. If they needed to swap out a photo filename or update testimonial text, they could do it with reasonable ease.
Copilot and Poe produced code in the right format, but skipped comments entirely. Claude included some comments, but not as thoroughly as Gemini and ChatGPT.
Why This Matters
Here’s what bothers me about tools like Replit: they seem to assume that if you’re using them, you either can’t or won’t learn to read code. But it comes with a hidden cost.
When you can’t inspect the code, you can’t catch bugs as easily. You can’t make simple content edits — not code, but content — without burning credits or waiting for another AI request. You can’t learn anything about how your own website works. And you’re trapped in a relationship where you have to trust the vendor’s judgment completely.
The frazzled volunteer in my scenario didn’t need React. They didn’t need a polyfills file or separate mobile and desktop codebases. They needed HTML, CSS, and minimal JavaScript that they could actually understand and modify if needed.
A good AI solution produces code that invites collaboration, instead of demanding total faith. It uses comments liberally so the human it’s collaborating with can follow what it wrote. It doesn’t import seventeen libraries when three lines of CSS would do.
Most importantly, it respects the human’s time. And that means not forcing them to learn an entire ecosystem just to change some placeholder text.
What’s your experience been? Have you run into AI-generated code that was technically impressive but practically useless? Let me know in the comments.
Next time, I’ll dive into the one thing nearly every AI struggled with: accessibility.
This is part of the Make vs Buy series, exploring whether it’s better to build things yourself or pay for ready-made solutions.