Explainability (XAI)

Explainability, often abbreviated XAI, asks whether an output can be accounted for in terms a given audience can understand and use. NIST IR 8312 proposes four principles: the system delivers an explanation, the explanation is meaningful to its recipient, it accurately reflects how the system produced the output, and the system operates within its knowledge limits. Techniques range from inherently interpretable models to post-hoc methods such as feature attributions and counterfactual explanations.

Explainability differs from transparency, which concerns disclosing that AI is used and how the system works in general. Legal hooks include the GDPR right to meaningful information about the logic involved in automated decisions, LGPD Art. 20 on reviewing automated decisions, and the EU AI Act right to an explanation of individual decisions taken with high-risk systems. It also enables effective human oversight.

Example: a lender declining a credit application tells the applicant that high existing debt relative to income was the main factor, and what would change the result.