Finance cannot operate on unexplained confidence
A confident answer is not the same as a controlled process. Controllers need to know which invoice, purchase order, bank line, policy, and approval produced an outcome. When the evidence is incomplete or contradictory, the correct behavior is often to stop—not to generate a more persuasive answer.
What a glass-box system exposes
The system should make its working state legible to the people responsible for the books. That includes source links, matching logic, confidence, policy checks, changes over time, and the exact point where human judgment was requested.
- Every number linked to its source
- Plain-language reasons for each proposed action
- Visible authority limits and approval gates
- A complete history of agent and human actions
- A safe pause when the system does not know
Explainability is operational, not decorative
A paragraph generated after a decision is not enough. Explainability must be embedded in the workflow so a reviewer can inspect the evidence before approval, compare changes, and reconstruct the decision later.
The practical buying test
Give a vendor a messy, real example: a duplicate-looking invoice, a vendor name variation, a missing PO line, or a bank-detail change. Ask the system to show what it knows, what it inferred, what policy it applied, and why it stopped or proceeded.