Quality Architect: The Future QA Role, According to Karim Jouini

An episode for testers and automation engineers wondering where their job is heading, and for QA leads rethinking their team. To get to the point: the Kinect story at 09:25, the moment Rosie and Karim settle on “quality architect” at 22:49, and the skills the role needs at 28:59.

For Karim Jouini, AI doesn’t remove the QA job: it removes its repetitive tasks. Writing Selenium or Playwright scripts, running manual campaigns, hunting down where a bug comes from: these tasks, which he estimates make up to 90% of a tester’s work today, will largely move to agents. Yet testing becomes more important, because delivery keeps accelerating.

The role that emerges, Rosie Sherry and he call the quality architect: the person who defines “what good looks like”, puts in place the architecture, tooling and guardrails, and upskills the whole team, while responsibility for quality is shared across product, engineering and the business. The role requires four skills:

• management, because everyone becomes a manager of agents;
• systems thinking;
• understanding the whole loop, from market need to delivery;
• product-oriented quality metrics.

The Kinect: when a product is harder to test than to build

Karim’s love for testing started at Microsoft, on one project: the Kinect, the Xbox camera that let you play without a controller, around 2010. He wasn’t a developer on it but a beta tester, one of about a thousand employees chosen for their diversity: size, skin colour, family setup, living room layout.

Every day, testers received tasks: play football or golf in front of the camera, then rate the experience. The sessions were recorded, and every night, the millions of collected videos were replayed against the new version of the algorithm to check that it did better. Even defining “better” was hard: if Karim is bad at golf, he’ll feel the camera isn’t recognising his swing when the fault is his. How do you calibrate perceived quality without rewarding bad players?

A hiring culture turned upside down

Karim’s other memory is less positive: a culture where, when hiring, excellent engineers were steered into development and average engineers into testing. He pushed back, all the way to HR, to rethink how testers were hired and evaluated. His conviction: excellent testers transform the velocity of an entire organisation, because developers move faster when every change is well evaluated and they can trust the system.

Why quality held Expensya back, and led to Thunders

At Expensya, an AI-based expense management platform (60 countries, 2 million users), Karim didn’t have Microsoft’s money to “throw hundreds of people at the problem”. In a fintech operating in 60 countries, regulation changes two to three times a week, and customers (utilities, rail and defence companies, half the banks in Europe) accept no regressions. The team tried everything: a large QA team, lots of automation, around a million lines of Playwright. The problems remained: a cost that slowed innovation, and a growing gap between what the business needed and what teams built and tested.

After selling Expensya, Karim and his co-founder Jihed Othmani surveyed around a hundred CTOs, CPOs and QA leads. 92% named quality assurance, in one form or another, as their biggest bottleneck. Their thesis: generative AI makes it possible to get the quality of manual testing back, with agents instead of humans, and therefore at scale.

AI eats tasks, not jobs: the lesson from ops

Rosie notes that job titles are shifting everywhere: her community lead also does product, and she calls herself a “community architect”. Karim answers with a transformation everyone has forgotten: ops.

In his group at Microsoft, first part of Xbox and then of Azure, around 250 people worked, including some twenty ops engineers who deployed to production, provisioned machines and secured the infrastructure. Eight years later, only one was left. No one was “cut”: 80 to 90% of the classic tasks moved to the hyperscalers, the rest to developers who became DevOps. Technology ate tasks, not jobs. The remaining ops engineer wasn’t there by luck: he was the one who understood the system in depth, and could fix it when things went really wrong.

For Karim, testing is going through the same shift, but in 5 to 10 years instead of 15 to 20. If your job is 100% writing automation scripts, you’re in trouble. If it’s thinking about and architecting quality, AI is an opportunity: you were losing 80% of your time writing scripts, and now you can delegate it. Longer term, he sees a dominant role emerging, the technical product manager, and already sees teams merging QA and business analysts, where the QA is the person who knows the product best.

What is a quality architect?

A quality architect is the expert who defines what “good quality” means for a product, then puts in place the test architecture, tooling, guardrails and metrics that let the whole team (product, engineering, business and AI agents) reach that bar. They no longer carry test execution alone: they design the system that makes it possible. Karim uses test architect and quality architect interchangeably.

That’s where the conversation lands. As with DevOps, quality becomes everyone’s job: product, developer or business analyst, “quality is part of your job. It’s not the thing that you do when you have time.” But they all rely on architects who put the right tooling and guardrails in place, make the right investments and define what good looks like. In large teams, it’s a role in its own right; in small ones, a skill carried by other roles, a product manager for instance. Rosie connects it to quality coaching, which would be one part of it.

Karim details four skills.

1. Management, including of agents

The first skill is a soft one, and Karim considers it the most underrated: learning management. Every individual contributor becomes a manager, of humans or of agents. And managing agents looks a lot like managing a team: think the plan through before building, break it down, hand it out, coordinate, check the right thing is being built the right way, measure the right indicators. It’s, he says, a way of practising shift-left.

2. Systems thinking

Rosie raises the risk of “the glaze”: skimming over too many AI outputs without really reading them. Karim sees a need for systems thinking. It isn’t a gift: “system thinking is a hard skill, and one that you can learn.” He’s seen brilliant developers build an excellent solution without realising it would explode in production. At Microsoft scale, “whatever comma that you put is multiplied by hundreds of millions.” Systems thinkers know they are one piece of a large machine, and factor that machine into their thinking.

3. Understanding the loop

A quality architect has to think in terms of velocity. The job isn’t only to test what goes out to market, but to do it in a way that’s efficient for everyone. So they need to understand the whole loop, from market need to market delivery, and stay up to date on tooling.

4. Product-oriented quality metrics

Karim recommends that every quality team look at Airbnb’s “11-star experience” framework: what does a 5-star, a 6-star, a 10-star experience look like, the one users will remember and talk about? Many QA teams don’t use frameworks like this because they’re absorbed by running campaigns. Hence his conviction that product and quality will merge: “quality will never be amazing if you only test whatever gets shipped to you.” You have to influence the thinking from the start.

What will disappear from the QA tester’s job

Rosie asks it directly. Karim’s answers:

• Hand-writing Selenium and Playwright scripts, “at least the way we do it today”. Overseeing agents that write them, why not.
• Manual testing as we practise it today, even though many disagree. Tomorrow’s manual tester is the equivalent of a QA lead managing agent QAs: deciding what to test, what coverage to aim for, which dimensions to cover (accessibility, regulation…), then handing execution to agents.
• The detective work of debugging: finding where a bug comes from is something AI excels at, because it has access to huge amounts of data. At Thunders, where we test our own product with Thunders, AI debugs “at an incredible speed”.

In total, maybe 90% of a tester’s current tasks, he estimates. But testing becomes more important, because code is being generated faster and faster. Karim cites a recent statistic that production defects per change request have tripled: teams skim through changes produced too fast. For the broader debate, read our analysis of how AI is reshaping QA roles.

Adopting AI without throwing everything away: migration, waitism and change management

Should you delete your million lines of Playwright and start from scratch? No, says Karim: there won’t be successful AI projects that can’t bring the best of what you had before. At Thunders, our customers migrate their tests in a few days, because converting code into natural language is easy: “going from rigid to smart is quite easy. The other way around is harder, and sometimes impossible.”

The waitism trap

The real danger, in his view, is what the French call attentisme, “waitism” as he translates it for Rosie: models change every week, so people wait to see who wins. But it’s an arms race, and a competitor that adopts AI can innovate you out of business. When a prospect prefers to wait for the “right” AI testing solution, Karim answers that 90% of what you learn will carry over, just as Selenium skills transfer to Playwright. What matters is the mindset shift: from doing testing to delegating testing.

Change management

No big bang. With our customers, mostly large enterprises, we deploy one team, gather feedback, then three teams three months later, adjusting along the way. Karim compares the change to switching from cycling to driving to work: you gain a lot, but you lose things, like parking anywhere, and no improvement to the car will bring them back. Teams need to be helped through that, with change-management specialists and partner integrators acting as forward-deployed engineers. At PwC, for example, SAP deployments rely mostly on SAP acceptance testing that used to involve 200 to 300 testers; Thunders shortens those projects by acting as an “army of testers”.

Above all: don’t buy AI to fire your team. “Teams are not good at implementing stuff to get fired.” Karim advises setting shared goals on release speed (what happens to your business if you ship every week instead of every month?) and on production bugs, rather than on cost. Rosie shares her practice of “20% AI time”, one or two hours a day to explore; at Thunders, our marketing and sales teams have “AI Wednesday”, with a roadmap of tasks to automate ranked by impact and risk.

• AI eats tasks, not jobs: as with ops, most of a tester’s tasks will be absorbed, and the role will move towards design.
• The quality architect defines “good” and builds the system (tools, guardrails, metrics) that lets the whole team reach it.
• Learn management: managing agents takes the same reflexes as managing people: plan, break down, coordinate, measure.
• Systems thinking can be learned, and it’s becoming essential given the volume of what AI produces.
• Measure quality from the product side, with frameworks like Airbnb’s 11 stars, and get involved from the design stage.
• Don’t wait for the perfect solution: start small, with one team, and goals on speed and quality rather than cost cutting.

Into the MoTaverse is the podcast of Ministry of Testing, one of the largest communities of software testing professionals, also known as the MoTaverse. Its founder, Rosie Sherry, talks with testers, quality leaders and people across the ecosystem about how quality gets built, and increasingly how AI is changing the craft. We are proud to be a MoTaverse partner. Watch the episode on ministryoftesting.com.