In this series I am writing several blog articles about Artificial Intelligence. Today’s article concentrates on one of the most critical issues facing the use of the AI/ML …..validation.
Although I have a Ph.D. in Cognitive Psychology, my B.S. and M.S. are in Physics and Mathematics. My professors used to always say that you should never work a math or physics problem unless you know the correct answer beforehand. Otherwise how would you know if you worked it correctly? (remember that elusive 2*pi that was always lost somewhere?). After many years of working in the field of Artificial Intelligence/Machine Learning (AI/ML) (almost 40 years) I have found this particular rule to be true. As long as you are working a math or physics problem validation of the result is relatively easy. Unless it’s a problem that has never been worked out before, there is a reference somewhere where you can check your answer. When we work with AI/ML, however we are usually working with semantic ideas and concepts where there is no reference book like the Handbook of Chemistry and Physics or Arfkin’s Mathematical methods of Physics where you can go to check the validity of your answer.
For most of my career when the General in the audience asked the inevitable question, “how do you validate the answer from your AI/ML application?”, the answer was always …..we use experts!
Validation of semantic or mixed algorithms is a difficult problem because you must first define what you mean by the word “validation”. The level of validation needed usually is dependent on the criticality of the application. Using our smart light bulb example from one of the previous blog articles, if the light bulb tells you its about to burn out and it actually could have lasted a few more years, not much is lost. However, if you are relying on an armed unmanned vehicle to make the correct decision regarding whether to launch a missile at a target, there is much more at stake and therefore the level of validation and probably the method would need to be different.
Validation is often the forgotten requirement in DoD solicitations. This happens because it is so difficult to solve that many buyers leave it out hoping that it can be solved later. The DoD has both a legal and moral responsibility to make sure that there is an adequate validation approach for the AI/ML applications it buys. The classic fall-back position is to insert a supervised learning loop in which the answer given by the AI/ML application is first approved by a human in the loop (subject matter expert) before it is applied. If one is going to use real human intelligence as the validation method during execution of the application, then why bother with AI/ML?
The only way to keep an AI/ML application autonomous is to provide the expert validation beforehand. And this can only be done by using finite states in which the answers to specific situations, scenarios, (states) are pre-defined by experts. Applications like this are called finite state machines and because the correct answers to states are pre-defined many do not consider this to be AI/ML. The truth, however is that the human brain actually stores information and the context associated with it in a similar manner. Knowledge within the brain is stored with collected bits and pieces of context that may modify the understanding of the knowledge but not completely define it because the person has not experienced or learned everything associated with a particular piece of knowledge. An expert will have more context than a novice but still may not have complete understanding of the solution to a problem, therefore your validation is only as good as the “experts” that you have access to.
My point is that DoD Program managers may want AI/ML in their systems but they must be very careful to consider the level of validation needed/required beforehand or they will end up being sold a very complex and expensive piece of software that can’t be trusted when a simple solution would have worked just as well.
This leads us to our next blog topic; Complexity. AI/ML software and algorithms can be extremely complex which translates to high cost…but is it necessary? My next blog will look at why AI/ML solutions don’t need to be complex or expensive…no matter what the big companies tell you!
Andy Bevilacqua, Ph.D.
Cognitive Optical Psychophysicist