An Overview Of QA & QC

Quality Assurance (QA) and Quality Control (QC) are two well embedded aspects of quality management in the development of delivery and maintenance of software. Although QA and QC are closely interrelated, they have separate definitions:

QA provides the confidence (and certainty?) that the quality requirements will be fulfilled both for the management of the enterprise and for the customers of the enterprise and other stakeholders such as government agencies, and regulators.

QC is in essence a subset of QA which focuses on inspection elements of QA.

QC is likely to be minor or sometime irrelevant aspect of QA for service organisations, because they have no deliverable physical product. At best it is confined to documents in the form of designs, contracts and agreement, manuals or other documentation which covers the delivery of a service.

Inspection is the process of measuring, examining and testing the features of a product or service to ensure they meet the product or service specification. Auditing is a documented, planned and independent assessment of whether the product or service specifications are being met. Audits may be made of the QA system, processes, products and services. Audits extend to so-called compliance audits to ensure compliance with standards, specifications, contractual teams and regulatory requirements.

How satisfactory QA and QC are can usually only be determined in the negative when customer service delivery fails, often in high profile situations such as product withdrawals, and systems failure or successful cyberattacks on operating systems.

What Impact Has The Deployment Of AI Had On QA & QC?

I would suggest it is too soon to make judgement on the effects of AI which is subject to such a velocity of change for its effects to be measured exhaustively. However, proponents of AI would assert that AI has a significant impact on quality control through machine learning algorithms that analyse large amounts of data. That analysis allegedly enables enterprises to detect patterns and predict problems and consequently improve product quality and reducing costs. They include:

• Improving Standards Across Industries - AI QA can enhance industries that require strict QC, such as pharmaceutical or food production. In pharmaceuticals, AI helps companies follow regulations and improve the consistency of their products. In food production, AI ensures that safety and quality standards are consistently met.
• Automated Measurement and Inspection - AI-powered systems enable automation of the process of measuring and inspecting products, which helps businesses identify defects more efficiently. Rapid identification reduces errors and accelerates approval process so that high-quality products reach customers faster.
• Speed and Efficiency - AI processes large amounts of data much faster than humans, resulting in quicker more efficient checks, which reduces manual effort and improves productivity resulting in more efficient operation.
• AI in Manufacturing - AI can improve manufacturing processes, where saving time is critical in industries with high-volume production. AI systems provide precise quality checks, enabling businesses to maintain high standards but concurrently increasing production speed.
• Cloud-Based Quality Management Systems - Cloud-based quality management systems (QMS) are being introduced to manage QC processes. QMS provide easy access to data. QMS enables teams to collaborate universally as well as scaling operations, manage demand, and maintain data security.

Adoption of AI for QC means that the QC’s technology tools become even more essential but correspondingly more dependent on AI technology, which is rapidly changing, many would suggest without proper oversight. AI-enabled systems can improve accuracy, speed, and efficiency, which make key segments of modern business operations dependent on AI.

Quis Custodet Ipsos Custodes?

If it is accepted that AI is revolutionising QC by providing faster, more accurate processes to ensure product quality, how are QA and QC being deployed to ensure that AI itself is being developed and operates with the requisite level of oversight?

QA's role is to validate the ‘usefulness’ of this data and assess its ability to fulfil the intended purpose. QA engineers evolve scenarios to measure algorithm performance, observe data behaviour, which should ensure delivery of accurate and consistent predictive results from the AI.

By contrast with traditional software development, QA involvement does not stop after initial testing. Engineers repeat the process over some period, depending on the thoroughness of the project and available resources.

QA for AI projects is very complex field with a number of key challenges, which include:

• Developing AI Model Data sets Developing AI models requires vast amounts of data to capture the multiplicity of scenarios. Insubstantial breadth of data representation lead, which to biased models perform inadequately under some scenarios. AI systems require data that reflect the complex relationships and their subtleties. Intricate decision-making tasks can prove challenging to extraction of levels of detail.
• Complex Algorithms - QA teams require a thorough understanding of complex algorithms, encompassing their interactions and information-processing mechanisms. Effectively navigating this complexity is crucial for developing testing strategies that comprehensively assess AI systems' functionality, accuracy, and reliability.
• Supervised and Unsupervised Systems - Supervised systems rely on data which has been assigned specific meaningful contexts or categories. Unsupervised systems operate on data with no tags or contexts. They require different testing methodologies. QA teams have to test the subtle nuanced differences between supervised and unsupervised data to align with each system type's specific requirements and characteristics.
• Integration of Third-Party Components - AI systems incorporate third-party components such as cloud services, external data sources and application planning interfaces .QA serves the role meeting the challenges to ensure the functionality and compatibility of integrated third party components.
• Transparent decision making - QA teams require the ability to confirm and evidence that decisions made by AI systems can be effectively communicated and understood by developers, users, regulatory bodies and other third parties. Careful design is necessary for transparent communication.
• Changeable Learning Speed - It is slowly being recognised that AI systems are evolving in an unpredicted and unpredictable way poses challenges for QA. The rate at which models learn and adapt can vary based on factors such as the amount of available data, the complexity of the task, or changes in the input. QA processes need to account for this variability.
• Changeable learning speeds directly impact system behaviour. Rapid learning may result in quick adaptation to new patterns but can also lead to overfitting. Slower learning hinders the system's ability to capture evolving patterns in the data.
• Calibrating simplicity and complexity in AI models - AI model must not be too simple if they are capture the underlying patterns in the data. Simplicity results in poor performance- a failure n inability to represent the complexities of real-world scenarios adequately. Over complex models showcase low error rates but can perform poorly on validation or unseen datasets. They may demonstrate high accuracy within the training set but fail to generalize to different scenarios. QA teams should find the right balance between model complexity and simplicity.
• Risks of Using Pre-Trained Models - Leveraging pre-trained models in AI introduces risks necessitating careful consideration during QA. For example, QA teams must evaluate the compatibility of these models with specific use cases, assess the transferability of knowledge to new domains, scrutinize for inherited biases, and navigate challenges in fine-tuning to prevent overfitting.
• Concept Drift - This a malady to which older and in time legacy AI systems will be particularly prone. Concept drift occurs when the statistical properties of the variable or input features in the data change over time. The drift can be gradual or abrupt, affecting the performance of AI models data distribution. Continuous monitoring and adaptation strategies are necessary to address shifts in data distribution.

The Future QA & QC For AI

AI must not be allowed to assume the mantle of infallibility, especially as it is currently driving the velocity of change, and can outstrip the capabilities of QA and QC. To that end the robust QA and QC processes outlined above must continue to be applied to development and delivery of AI, even if they slow some aspects of development and delivery.

I would question the current capacity of resources to maintain effective oversight of AI, especially given its rapid evolution. The key question is whether AI will be allowed to run ahead of QA and QC, or whether AI will be restrained to the pace and capacity of QA and QC resources.

Commercial confidentiality on failures of AI should be limited to serve the public interest, in order to ensure that errors deriving from reliance on AI may be publicly acknowledged and rectified.

Legislation is required to ensure a free market in AI, such that its delivery is not monopolised by the global “magnificent seven” who have their own agendas to pursue, not only of a commercial nature.

Bob McDowall

June 2025