Explainable AI is the practice of showing a person why an AI system produced a specific output, with evidence that person can understand and check. It is often shortened to XAI, for explainable artificial intelligence. The definition is simple when a model scores a loan application.