Learn by Playing, Lead by Knowing: Why Experience Is the Real AI Skill

During Tuesday's Duda webinar, What We're Actually Building With Duda Vibe, Dylan from Olive Street Design dropped a comment into the chat that summed up the last two years better than most conference keynotes:

"I can just have AI do it myself" is the new "my nephew has Photoshop, I'll just have him do it".

Everyone on the panel laughed because everyone on the panel has lived it. There was mention of clients sending over a homepage they've redesigned in Claude, then asking what to do with it. The tool produced something. What it couldn't produce was the judgement to know whether it looked good, would work or what should happen next.

Picking up from last week

Last week I wrote about what happens when the tool learned from us: if AI is trained on human craft, who answers for what it makes? This week takes the practical side of that question: what AI is actually bringing to the table in web, app and SaaS tool design. If the tool learned from us, what do we still bring that it can't?

We learn by playing

Every designer using a tool learned by doing, whatever the medium or output, and whether the end result is crafted, engineered or architected. A graphic designer learns spacing by setting type, an engineer learns tolerances by making parts that don't fit, an architect learns how a space works by drawing and redrawing it, and a developer learns a framework by breaking it. You try it, get it wrong, and try again. The same is true of every new technology I've adopted over the years. As an early adopter I've experienced my share of pushback, but my fascination with and appreciation for design processes and tools, old and new, has never wavered.

During Tuesday's webinar, Nat Rosasco of Olive Street Design explained how a bit of experimenting turned into a whole new CRM. The agency has 18 people and around 1,300 clients split across three very different client types. They had tried QuickBooks, Monday, HubSpot, Pipedrive, Zoho and Salesforce, and none of them fitted without workarounds and custom fields. So one evening Nat opened Duda Vibe and simply described the business. About 40 minutes of conversation produced a plan. Roughly 20 to 25 hours later, they had a working CRM that they now expect to run the business on. Built from scratch the traditional way, Nat estimated it would have cost six figures.

Brian Lewis plays in a more structured way. He duplicates a build before trying something new, and if it doesn't work out, which he puts at roughly a 50-50 chance, he throws the copy away and tries again. That's play with a safety net, and it's exactly how you should approach it.

Gerry White, the SEO consultant on the panel, has been playing in a different direction, using Vibe to build his own SEO dashboards rather than client websites. Same tool, completely different output, because he brought a different set of problems to it.

I've been playing too. In August I tested Duda Vibe against Google Stitch, Bolt.new and Lovable on a real charity build for Outside Thinkers, and wrote up what happened. Duda Vibe gave the best result of the four by a clear margin, but it used my entire month's AI credits in a single pass, and even the best output needed a designer's hands before it was fit to hand over. To actually deliver, I built the site in Duda's Editor 2.0, with Copilot doing the structural first pass and me finishing the design myself. That saved three to four hours on the scaffold and left credits to spare. Play showed me what the tools could do. Experience told me what the client needed.

Nat's advice to anyone who hasn't started was simple: jump in, don't overthink it, and pick one thing that would help you day to day.

Experience is the multiplier

Here's where it gets interesting. Play gets you started, but experience decides what you get out of it.

Gerry White mentioned a video he'd seen making the point that a developer who vibe codes is dramatically more efficient, while someone with no background who vibe codes will probably produce something pretty terrible. That's not snobbery. It's about knowing what the tool can't tell you.

Gerry gave a good example. He hadn't managed to get his Vibe dashboards reading Google Analytics directly, so he exported the data to a Google Sheet and had Vibe read from that. He has workarounds like that for a lot of things, as he put it, because he's done all that stuff before. Someone starting fresh would hit the same wall and conclude it simply doesn't work.

That's the value of old-school experience. It isn't about doing things the long way forever. It's knowing where the walls are, knowing what good looks like, and knowing when something that looks finished isn't. That's tacit knowledge, the kind you only build by doing, and it's the part no prompt can supply.

A word on the anti-AI stance

On LinkedIn I've seen plenty of creative agencies push back on AI. I understand where that comes from. If you've spent years mastering graphic design or development, watching a tool do part of that in seconds is unsettling, and the fear is real.

But I think the stance mixes up two different things: the talent and craft, and the admin that surrounds it.

When a client sends me a logo, if the full set isn't supplied, I need it in every size, format and colour variation: vector files, favicon, social, print, full colour, mono, reversed and so on. That's not design. It's production admin. Before, I'd either pass it to someone who does production graphics every day, or do it slowly myself because I don't do it often. Now AI can do it in minutes. Nat made the same point on the panel. Why would you ask your best designer to spend an hour of her time on that when she could be doing something that actually needs her?

Refusing to use AI for the laborious tasks doesn't protect the craft. It just means your most talented people spend less of their time on it.

This is the part I care most about. Using AI well doesn't mean handing over control. It means being the conductor. In AI terms, that's keeping a human in the loop: a person who sets the brief, checks what comes back and makes the call at each stage, not just someone who signs off the finished result.

Prep before you prompt. It's like cooking. You get all your ingredients and spices on the counter before you start, not halfway through. Before any vibe coding test I run, I prepare prompt sheets first, including what each tool needs to know. People who've built things the long way can see what's coming, and that foresight shows in the result. I've written before about why process comes before the prompt.

Get better at prompting, and keep getting better. Prompts are briefs. The same prompt run ten times can give you eight different results, as Brian and I agreed on the panel. I've had Claude follow a benchmarking process perfectly five times and then forget a step on the sixth. That isn't a reason to give up. It's a reason to stay in charge, check the output and refine the brief. Your prompts usually aren't the problem. Your process is.

Know what the tool won't volunteer. Gerry raised the question every agency should be asking about anything with user logins: how secure is it? He's had the experience of changing a number in a URL and suddenly being able to see everyone else's records. As he pointed out, even a simple instruction in the prompt to make security a priority improves the result, but you have to know to ask for it. The tool won't raise it on your behalf. The conductor has to.

Nat quoted a friend who describes AI as an executive intern. It's a useful framing. An intern can be brilliant, but only if someone knows what to ask of them and checks the work.

What it means commercially

One question I asked the panel was how they're pricing this kind of work. Brian's answer was standard time. He's seen agencies rush to lower their rates because the work is faster, and his view is that you don't need to. You invested the time to learn the tool and master it, and the risk of things not working as they did yesterday hasn't gone away.

I'm still working my own approach out. There's a real cost in tokens and credits on the other side of it. But I agree with the principle. The time you save is best put back into value for the client, whether that's more strategy, more testing, or the work you never had time for before. For my retained clients, that's already happening: when something takes less time, the hours go somewhere more useful, and I'm open with them about it.

Bring your experience with you

We're at an odd moment, and it pulls in two directions. On one side, the anti-AI posts keep coming. On the other, there's a flood of AI slop: sites, content and designs that were quick to make and look it. They're two reactions to the same shift, and neither changes the fact that AI isn't going away. Nat reckons most agencies still aren't really using it, and the ones leaning in thoughtfully are pulling ahead.

My view sits in the balance between the two. Play with it, but don't leave your experience at the door. It's what keeps the work from becoming slop, it's what makes the tool worth using, and it's what your clients are really paying for.

If you'd like help working out where AI fits in your process, and where it doesn't, just email me.