This article is a good read. However I feel that the author is kind of missing the point. When we say AI is not “explainable” we are not simply talking about whether we can draw a straight line linking “causes” in the form of data attributes to the final “action” recommended by AI, via a set of probabilistic computations.
“Explainable” also implies a notion of “accountable“.
When we say human decisions are “explainable” were are at least in part actually saying that humans can be held accountable for their decisions. So if a human takes a weird racist decision, and we ask him “Can you explain yourself?”, we don’t really just want to know how he has developed racial stereotypes due to his particular life experience and figure out if perhaps those stereotypes are baseless. We really are asking him “Are you accountable for your actions? Do you own up to them? And if they are considered against the law, can you atone for them? Can you be punished?”
This is the sense in which AI actions are still a black box. Because we cannot obviously punish a machine or hold it accountable. And it seems we cannot hold the creators of AI algorithms accountable either, because they can shrug and say “We only taught the machine HOW to learn, but the actual learning happened from data and we did not manually inculcate any biased rules into the algorithm.”
Even if AI is black box in this sense today, it doe snot HAVE to be that way. We can design AI learning techniques that has specific overrides that prevent it from learning bad things , such as racial stereotypes and also encourage it to learn good things, such as empathy for the unlucky or the downtrodden. In such an AI system, the accountability will definitely lie with the creators of the AI systems, i.e. the humans.
This is no different than how good parents need to inculcate culture and empathy into their children as they grow up, and also avoid loading their minds with their own presumptive biases and world views. Alas this ideal of child rearing is not achieved that often in practice!
Anand's musings 