TLDR
Software quality assurance aims to prevent defects by controlling development, testing and delivery processes. An audit assesses that system rather than the software itself: reference standards, quality assurance plan, test coverage, traceability, compliance and defect management. It identifies gaps, rates their severity and leads to a prioritised action plan.
To identify gaps and prioritise actions, start by auditing your quality system.
Introduction
Software quality assurance covers the processes and controls intended to guarantee that software meets its defined quality requirements throughout its development lifecycle.
Having automated tests, a test plan and defect tracking is not enough to prove that QA is effective. Risk coverage, requirements traceability and the evidence behind each control still have to be verified.
That is the purpose of the software quality audit: confronting planned practices with actual practices to identify the gaps and the priority actions. It replaces neither testing, which detects defects, nor debugging, which traces their cause and then fixes them.
What is a software quality audit?
A software quality audit assesses the processes, responsibilities, practices and evidence that make up the quality assurance system. Unlike testing, it does not look for defects in the software directly.
A quality audit can be run by an internal auditor independent of the activities under review, by a customer, by a specialist firm or by a third-party body. It relies on a reference standard and on verifiable evidence.
It produces a completed grid, gaps rated by severity and an audit report that serves as the basis for the action plan.
The types of software quality audit
An audit can be classified by its object or by the relationship between the auditor and the organisation being audited.
In regulated sectors, IEC 62304 governs the lifecycle of medical device software, in addition to the quality management requirements of ISO 13485. In automotive, ISO 26262 and ASPICE cover functional safety and process assessment respectively.
ISO/IEC 25010 provides a model of characteristics for evaluating the quality of a software product.
Five signals that trigger an audit
1. Regressions keep coming back
Features that were already validated break after a release. These regressions can reveal uneven coverage, fragile regression tests, or controls unable to intercept certain defects before they reach production.
2. Delivery timelines drift
Validation phases stretch out and defects surface late. These delays can signal tests that run too slowly, unstable environments, or defect management that no longer keeps pace with development.
3. The team or the supplier changes
A change of team exposes dependencies on individual knowledge. The quality audit then checks whether processes, responsibilities, requirements and evidence are documented well enough to maintain continuity despite the change of people.
4. A certification or a due diligence is coming
A certification, an acquisition or a due diligence requires verifiable material. The audit identifies compliance gaps in advance and gathers the evidence needed to demonstrate process control.
5. The product is growing fast
Applications, teams and releases change scale. The audit determines whether the scope of the QA system, its resources and its controls are keeping up with that growth, or whether the maturity of the practices has become insufficient.
The 7-step method
A usable audit frames the scope, gathers the evidence, qualifies the gaps and turns them into actions that are tracked.
Step 1: define the scope and the reference standards
Define the scope: applications, environments, teams and period. The review covers the stages included in that scope, from requirements through to maintenance.
Then choose the reference standards: IEEE 730 for software quality assurance, IEEE 1028 for reviews and audits, ISO 9001 for quality management, ISO/IEC 25010 for product quality, CMMI and TMMi for process maturity.
The audit grid ties each criterion to an expected piece of evidence and to a possible gap. In regulated sectors, where a standard requires a documented activity, the absence of evidence can constitute a nonconformity.
Step 2: audit the quality assurance plan
The software quality assurance plan (SQAP) sets out:
- objectives and scope;
- roles and responsibilities;
- processes and reference standards;
- verification and validation;
- defect and change management;
- documentation, measurement and reporting.
The SQAP organises quality assurance for the project. The test plan defines the testing itself: scope, levels, responsibilities, environments, schedule, and entry and exit criteria.
The quality manager or QA lead owns the SQAP. The audit checks its revision date, its history and its consistency with actual practice. An outdated SQAP no longer describes the real system.
Step 3: audit the testing setup
The audit examines unit tests, integration, system and acceptance tests as well as end-to-end tests, along with coverage of code, requirements, risks and user journeys.
Performance tests, security tests, accessibility tests and browser and mobile compatibility tests are reviewed against the product’s requirements.
The audit also assesses whether it makes sense to move from manual testing to automation. Repeatable checks lend themselves to automation; manual and exploratory testing remain useful wherever human analysis is needed.
The audit measures regression suite runtime, automation rate, maintenance cost, flaky or disabled tests, and the stability of environments and data. Code quality is examined only where it affects testability or maintenance.
Three questions to ask during the audit
- Which risks are covered by no relevant test at all?
- Which tests are unstable, disabled or automated for no good reason?
- How much effort does the team spend maintaining the suite?
These findings show whether the tooling genuinely supports the test strategy.
Audit question: do the tests cover the right risks, at the right level and with an execution mode that fits?
Step 4: audit traceability and defects
Traceability follows the chain:
requirement → test case → result → defect → fix → retest
A third party must be able to reproduce the test from its preconditions, data, steps and expected results. The audit also checks the severity, priority, time to fix and reopening rate of defects. A root cause analysis is expected for critical or recurring defects.
Audit question: can every defect be traced, its priority justified and the causes of critical defects shown to be addressed?
Step 5: audit integration into the lifecycle and the team
In CI/CD, quality gates block a release when a defined criterion fails. Continuous testing in a pipeline makes it possible to run the checks as changes come in. Their CI/CD integration moves validation closer to development and delivery.
The audit also examines quality culture: code review, TDD and the Definition of Done. The quality assurance engineer contributes to test strategy, results analysis and the improvement of practices.
Audit question: are the quality controls built into the lifecycle and able to block a release that fails the defined criteria?
Step 6: quantify the cost of poor quality
The cost of quality separates prevention and appraisal spending from the costs tied to failures.
Detecting a defect early limits rework, whereas late detection can pull in more teams and delay the release. Invisible costs include lost productivity and service interruptions, among others.
On 19 July 2024, a faulty CrowdStrike update caused crashes on Windows systems. The company subsequently strengthened its testing, its validation controls and its staged rollout.
Audit question: are defects detected early enough to limit their cost and their impact on users and operations?
Step 7: report back, prioritise and follow up
The audit report separates the finding, which rests on evidence, from the recommendation. Gaps are rated by severity, then placed on an impact/effort matrix.
Each action gets an owner, a due date and an indicator. The action plan can track requirements coverage, test pass rate, critical defects, reopenings, regression suite runtime, time to fix and production incidents.
These indicators drive decisions. A rise in critical defects after release should lead to a review of the controls or of the criteria for going to production.
Audit question: do the KPIs make it possible to verify that the actions actually reduce defects, delays and incidents?
What an audit finds, nine times out of ten
An audit often reveals less a lack of tests than a gap between the system as planned and the way it actually runs.
These findings have to be weighed against the context of the QA teams, the criticality of the product and the applicable requirements. An audit delivers what a simple checklist cannot: evidence and prioritisation. It is what separates a minor gap from a finding likely to affect quality, compliance or delivery.
Deliverables, duration and people to involve
The audit produces four deliverables: a findings report, a completed audit grid, a prioritised action plan and a dashboard for tracking the gaps.
Its duration depends on scope. A few days can be enough for one application and one team; several weeks may be needed for a full portfolio. The number of environments, documentation maturity, regulatory constraints and how accessible the evidence is all shift that duration.
For an audit focused on a single application, the time commitment can be estimated as follows:
These figures cover the time each participant is called on for, not the total duration of the audit. They should be adjusted to the size of the scope, the number of teams and the availability of evidence.
Tools
Tools replace neither the reference standard nor the audit method. They make it easier to collect evidence, maintain traceability and analyse the system. As test volume grows, scaling automation also means keeping execution and maintenance under control.
Test management and ALM solutions centralise requirements, test cases, results and defects. The choice of QA testing tools depends among other things on scope and on the technologies in use. For web applications, web automation frameworks structure the execution and maintenance of automated scenarios.
Automation and maintenance tooling makes it possible to run tests repeatably and to track their stability. In that context, Thunders notably allows you to create self-adapting tests to limit the maintenance caused by changes to the application.
Traceability and reporting features connect requirements, tests and defects, then provide the indicators needed to track the action plan.
Tooling choices should stay consistent with the team’s maturity and with the audit findings. An extra tool does not, on its own, fix an incomplete process, broken traceability or poorly defined validation criteria.
Software quality audit checklist
1. Scope and reference standards
- Define the applications, environments, teams and periods to audit.
- Select the standards and frameworks that apply to your context.
- Tie every criterion in the audit grid to a piece of evidence.
2. Quality assurance plan
- Check the revision date of the quality assurance plan.
- Compare documented processes with the practices actually applied.
- Verify the responsibilities and evidence set out in the SQAP.
3. Testing setup
- Split the tests by level and by product risk.
- Identify unstable, disabled or expensive-to-maintain tests.
- Check coverage of the main non-functional requirements.
4. Traceability and defects
- Link every requirement to its tests and associated results.
- Verify the reproducibility and severity of recorded defects.
- Measure time to fix and reopening rates.
5. Development lifecycle and team
- Check that quality controls are built into the lifecycle.
- Review the blocking rules applied in CI/CD.
- Examine code reviews and the Definition of Done.
6. Cost of poor quality
- Identify the costs tied to internal and external defects.
- Include delays, incidents and team time in the calculation.
- Connect the main costs to the gaps that cause them.
7. Reporting and follow-up
- Rate every gap by risk and severity.
- Turn recommendations into assigned, dated actions.
- Track their completion with indicators defined in advance.
