Most “best AI tools” lists only cover what people kept. We think the tools we tried and cancelled are more useful to talk about, because the reasons we dropped them are the same reasons a demo can be misleading.

An AI meeting note-taker with a polished demo

The pitch was clean: join the call, generate a summary, extract action items automatically. In practice, it struggled with the actual texture of client calls — crosstalk, people trailing off mid-sentence, technical terms specific to a client’s industry. The summaries needed almost as much editing as writing notes ourselves would have taken. We switched to a cheaper transcription tool and doing the summarization ourselves through Claude with a prompt tuned to our own meeting format, which produced better results for less money.

The lesson: a demo built around a clean, scripted conversation doesn’t tell you how a tool handles a real one.

An “AI website builder” aimed at agencies

This one generated a decent-looking site from a short brief in under a minute — genuinely impressive as a demo. The problem showed up once we needed to actually customize anything beyond the template’s assumptions: the generated code was difficult to extend cleanly, and modifying it by hand often took longer than building the same site from a solid starting template ourselves. It’s a good tool for someone who needs a single static page fast and won’t touch it again. It wasn’t a good tool for client work that needs to evolve over months.

The lesson: “fast to generate” and “fast to maintain” are different claims, and most demos only show you the first one.

A general-purpose AI “workflow automation” platform with its own proprietary logic layer

We wanted to like this one — it promised to replace several of our n8n workflows with a friendlier interface. What we found instead was a system that worked well for the exact examples in its own documentation and became noticeably harder to debug the moment our actual workflow diverged from those examples. Since we already run n8n self-hosted, replacing something transparent and debuggable with something more opaque wasn’t a good trade, even with a nicer UI.

The lesson: a nicer interface on top of a more opaque system is a real cost, not just a preference — you feel it the first time something breaks and you can’t easily see why.

What canceling these taught us about evaluating new tools

We now run any new AI tool against messy, real inputs from an actual current project before deciding — not the tool’s own sample data, and not a quiet week. If it only performs well on clean inputs, that’s the actual finding, not a caveat to work around later.