This is particularly important because many people are prone to blindly trusting AI explanations, based on a false assumption that they understand the process that produced them. To avoid the problem of misplaced human trust, the explanations provided by AI models must clearly show how the model works, as well as its limitations, in a format that is clear and transparent.

“Everyone expects that there will be new regulations, but for now, trust is the key driver,” says Meeri Haataja, CEO of Saidot, a provider of enterprise AI risk-assessment platforms. “Companies are looking at AI governance and responsibility because they see that it’s essential for their stakeholders.”

As the IBM report and other surveys reveal, consumers are more likely to give their business to companies that are transparent about how they use AI models. Organizations can face public relations headaches and potential legal liability when opaque AI systems produce biased outcomes.

Companies and government entities are deploying AI applications to make increasingly impactful decisions. In recent years, we’ve seen harmful consequences from machine bias in health care, criminal justice, and consumer lending.

In a heavily regulated industry like financial services, companies must be able to explain that their decisions aren’t biased against, for instance, women or minority borrowers. Similarly, credit agencies now rely on machine learning to determine credit scores. These agencies face increasing pressure to explain the logic behind their algorithms.

The European Union’s General Data Protection Regulation (GDPR) requires all businesses that gather data to explain how their automated systems make decisions. The EU has proposed additional regulations that require stiffer standards for transparency, accuracy, and responsible use that only explainable AI can make possible.

Singapore has also been a leader in this field. In 2018, the Monetary Authority of Singapore released its FEAT framework—an acronym for fairness, ethics, accountability, and transparency—as a blueprint for how financial services companies should manage AI and data analytics.