Audit evidence and nonconformity

ISO 19011 defines audit evidence as records, statements of fact or other information that are relevant to the audit criteria and verifiable. Evidence is gathered by sampling, through interviews, observation and review of documents and records, and its reliability depends on its source: an independent system log usually weighs more than a verbal statement. Comparing evidence with the criteria produces audit findings, which may show conformity, nonconformity or opportunities for improvement.

A nonconformity is the non-fulfilment of a requirement of the standard, the law or the organisation's own policy. Certification bodies working under ISO/IEC 17021-1 usually grade them as major, when they undermine the system's ability to achieve its intended results, or minor. Under clause 10.2 of ISO/IEC 42001 and ISO/IEC 27701, the organisation must react, analyse the root cause, take corrective action and keep documented information about the results.

In AI audits, typical evidence includes model cards, data lineage, test results, monitoring logs and approval records. Example: an auditor samples five AI systems and finds two without a documented impact assessment, recording a nonconformity.