As machines continue to play a larger role in making decisions that impact a person’s life, it will become more important for these machines to explain the process made to come to these decisions.

“We have to try and introduce or enable trust in these systems,” says Dr. Christina J. Colclough, who runs The Why Not Lab and is an advocate for global workers’ rights in the digital age. “And the way that we can do that is by having demands for transparency, fairness, and auditability, so that humans don’t feel that they are controlled by this algorithmic system, which knows more about me than I do.”

In essence, companies must open up the “black box” to let consumers and regulators know what’s going on under the hood and determine if AI’s recommendations are fair, accurate, reliable and in a person or society’s best interest.

“Just like with anything that is powerful, when people don’t see it and they don’t understand it, they end up fearing it and therefore avoiding it,” adds Shivananda. “It is important to make sure that as we build AI-based experiences with the use of technology, we need to build explainability into the process. A customer should be able to see why something happened on a product or an experience and platforms need to be able to explain why any choice was made.”

This becomes increasingly important in areas such as healthcare and autonomous driving, where an AI system’s decisions could be a matter of life and death. Many companies, including Google and IBM have already made commitments in this area.
The extent to which we approach explainability will vary in an effort to balance the need for privacy and security and accuracy – if we reveal too much, can the system be gamed or compromised? Without some effort to address explainability, however, developers will be leaving themselves open to questions and criticism.

Explainability, then, helps place humans in the decision-making process. First, by understanding how decisions are made, and then by providing the necessary context for users to further refine and optimize their AI experience. As a result, users can feel more confident not just in how AI may be advising them on more routine day-to-day activities, but also in more serious matters, such as critical healthcare or other areas where an autonomous decision will have far-reaching effects on a person’s wellbeing.

As AI advances it will become more ubiquitous while constantly learning to seamlessly add value to a user’s life. Using local context and external sources of knowledge, this “purpose-driven AI” balances a user’s competing needs and interests and is able to take creative approaches to influence user behaviours, all in the service of the user’s higher purpose.

The technological and, more importantly, ethical considerations to achieve this, however, strike at the very core of what it is to be a human being.
One only has to look at themselves to understand the complexity in asking users to adopt an AI-driven life – we have a persona for work, home, friends, people in our professional network, at the grocery store, in a job interview, on vacation, and on it goes. AI must be mindful of the boundaries within each of these personas and do so in a way that helps the user, but doesn’t necessarily begin to alter or influence their life in a significant way – by crossing work and home life, or providing recommendations for overtly commercial or perhaps subtle nefarious ways.

“How do we maintain our human rights, but also what I call our right to be human?” asks Dr. Colclough. “How do we avoid the commodification of people, so they’re not just seen as numerous data points and algorithmic influences, but the human you are – with your beauties, your bad sides, your good sides? How do you remain relevant and wanted and prioritized in this very digitalized world?”

You could consider all the internal and external inputs and outputs of data like a supply or value chain. Streams of data – contextual, emotional, factual, personal, commercial, cultural – are delivered on single or interconnected roads from myriad sources to AI-enabled products that now surround your life at work, home and play. The level of trust we’re asking of consumers to essentially let algorithms, and the developers behind them, run their lives is immense.

The widespread adoption of consumer technology is tied to a number of factors, including access, price, perceived benefits and perhaps most importantly, ease of use. From personal computers, smartphones and the internet, consumers flock to new technology when it’s clear they don’t need a degree in computer science to operate them.

Trust in a company’s offering, and by extension the company itself, relies on addressing the fact that today’s consumers expect a product to be reliable, intuitive, and simple to operate. It’s table stakes.

The evolution of AI-enabled products and services into our lives introduces a new challenge in consumer adoption. If AI is to be truly ubiquitous in our lives, we need to evolve from the current, intrusive way we interact with our technology – keyboard, mouse, clicks, searches, power cords, and anything else that ties us to machine or place. Voice and conversational AI, and a backend system that is accurately pulling together and analyzing all of your personal data and habits from a variety of sources, are now key to a superior AI experience.

According to a study by Gartner, 70 percent of respondents feel comfortable with AI analyzing their vital signs, and identification of voice and facial features to keep transactions secure. However, 52 percent of respondents do not want AI to analyze their facial expressions to understand how they feel, and 63 percent do not want AI to take an always-on listening approach to get to know them better. It can then be assumed there are similar reservations about brain-computer interfaces.

“People will want to make sure that their data is being treated properly and not being used for any sort of malicious or ill-conceived intent,” says Poggi. “ And people are obviously very sensitive with their personal data. We’re really crossing a bridge that we have not crossed before, because it’s much more personal in nature. The data we’re asking for is not just your name, address, and social security number, (but) now looking for your face or listening to your voice, which gets obviously a little bit more intimate.

“I think what’s going to become a key challenge is how do we face the need to get massive amounts of personal data in order to build high quality AI engines and at the same time, treating that data with a high degree of respect for the individual.”
So how do developers and designers bridge this gap?

Conversational AI should, in theory, create a more personal 1:1 experience than how we currently interact with technology. Bad design, minimal transparency, and generally poor communication that isn’t aware of the user (demographics, region, culture), demonstrates that you haven’t taken the time to anticipate your customer’s needs and wants. As a result, consumers will likely turn away, or provide substandard data to maximize the AI experience.

“We need to have as much data about people and about their experiences in order to have highly effective AI engines that are able to produce interesting results for us,” adds Poggi. “For instance, we need biometric data. We need voice data. We need visual data for facial recognition. All of these sensors have to provide us the data in a highly credible way that’s repeatable and robust so that we’re not making bad decisions off of that data. I think that the evolution of the quality of those input devices is going to be a key enabler to serve successful AI propagation in a humanistic way.”

For centuries, humans have been the masters of the tools and machines created to make them more productive at work and happier at home. Today, we are living in a unique moment in history where this relationship is changing. Machines now have the capability of being a much more dynamic part of our lives.

AI products and systems need data, personal data, to learn and become more effective in understanding a user. If users are reluctant to share this information, or “game it” in an effort to protect themselves from giving up too much, then they will never realize the full potential of AI in their lives. A company’s best laid plans for creating and delivering a truly superior AI product may live or die based on a most human characteristic – the ability to trust, or not.

Transparency will be key in this regard. At the highest levels companies will need to have a very honest conversation about what they do, what they don’t do and why they do it. Consumers will need to understand the philosophy behind a company’s actions, its frame of reference when developing the algorithms it is asking us to trust, and what recourse they have when things go wrong.

Consumers will need to know what data is being collected, who is collecting it, where it is staying and how it is being used and reused. Companies will need to own up and admit mistakes, which will happen, and clearly state how they will fix these missteps.

If we want to make AI-driven products and services serve people and the planet, governments and regulators must look at the whole ecosystem and its impact on people, and then put demands to what these systems are intended to do and hold companies accountable, with data privacy and security at the forefront.

“We’re not going to be able to market AI as being instantly trustable or to prove from a technological perspective that it is trustable in all cases,” says Foster. “Trust is going to have to be earned over time.”
AIX design that brings developers together with policymakers and especially end-users is the first step in achieving that long-term trust for the AI industry.