What Deep Learning Is Good For
And When It Shouldn’t be Used
“If Deep Learning is not followed by Deep Reflection — the amount of gained wisdom will remain …. pretty shallow” — Oleg Feldgajer
BROAD STROKES GIVEAWAY
Not all applications of Artificial Intelligence (AI) are created equal. Some problems require the answer to be 100% deterministic, not approximated. For example, you wouldn’t want neural network to “approximate” your monthly salary and receive different paychecks each month?
On the other hand, in many cases, the amount of data that needs to be processed is so huge, that even 90% classification accuracy — is EXTREMELY welcome! Especially, when the final review of the results is done by a highly qualified, and experienced, professional.
I’m not making such statements lightly. Exactly 20 years ago, I was running deep learning, neural network company — INM (International Neural Machines Inc.)
Steve Blank made it abundantly clear: “all startups are essentially a temporary organization formed to search for a repeatable and scalable business model” — and INM wasn’t any different.
By 1996, we developed and successfully tested our pattern recognition platform on images and speech, as well as tons of financial data. Encouraged by initial successes in detecting Visa fraud and money laundering — we yearned for an opportunity to work on advanced medical diagnosis. This is how I met, Nathan Hermony, head of Elscint’s nuclear medicine division.
Elscint Ltd. was at that time a world leader in the advanced medical imaging equipment. They successfully competed against such industry giants as General Electric, Siemens and Philips. Little did I know, that I’m also about to receive an introduction to the cut-throat world of Nuclear Medicine!
Nuclear medicine is a medical specialty that uses radioactive tracers to assess bodily functions and to diagnose and treat disease. Specially designed cameras allow doctors to track the path of these radioactive tracers. Single Photon Emission Computed Tomography or (SPECT) and Positron Emission Tomography or (PET) scans — are the two most common imaging modalities in nuclear medicine.
PROBLEM
Nathan had a serious problem on his hands. Historically, Elscint excelled in SPECT imaging. At the time we met, PET appeared to be eclipsing SPECT for cardiac imaging and oncology. The success of PET was attributed to its inherently better image resolution. In cardiac scanning, for example, it has generally been reported that PET offers a resolution of 5 to 7 mm, compared with a cardiac SPECT resolution of 12 to 15 mm.
Elscint revenues where exceeding $300MM+ and the company was negotiating multi-million dollar sales with a number of hospitals in Ontario, Canada. INM’s advanced pattern recognition capabilities could have become the last straw that breaks the camel’s back.
Well, I was so proud of lending a nuclear medicine client — that I offered Elscint to meet with hospital officials, and sell, sell, sell! Nathan was very pleased with my offer, but instead of rubbing his hands — he began to sweat, profusely. Soon, I was about to discover his predicaments.
It took approximately 4 weeks, before my neural network presentation took place at Mount Sinai Hospital, in Toronto. Top notch myocardial perfusion (heart disease) specialists, hospital administrators and IT personnel came to the auditorium. They were genuinely intrigued and impressed — by the possibilities of deep learning technologies. Few days later, as INM was setting up the database, I received an urgent phone call from Nathan.
Apparently, a much larger RFP has surfaced from The Western Hospital in Toronto — and this was it! Elscint simply had to win this bid! And Mount Sinai Hospital — had to wait.
I met with Dr. Burns at The Western Hospital, immediately. He was one of the most respected cardiac specialists around the world — and one of the most enthusiastic researchers I ever met. Not only did he help me understand the extent of available database — I also learned a great deal about some of the critical factors leading to a successful diagnosis.
Few weeks past, the data was received, and ….. you guessed it: Nathan calls again! This time, we were asked to work with Ottawa Heart Institute at Ottawa’s Civic Hospital. Now, this was Elscint’s top priority!
As they say: “fool me once, shame on you, fool me twice, shame on me” — so I wasn’t going to turn myself into a pretzel, and blindly comply. Instead of rushing to Ottawa, I asked Nathan: “what is the root cause for such an erratic behavior?” After a while, it became clear to me: Elscint was dashing to close the biggest deals before the next show of Radiological Society of North America (RSNA). Some of Elscint’s competitors were expected to introduce a new equipment at the show — with better features than Elscint could offer that year.
Granted, it took some convincing to do, but after a few heated exchanges — Elscint finally agreed that a good business model should go beyond offering new FEATURES. Although new features can be portrayed as an improved CUSTOMER VALUE — such can be easily replicated by competitors. Since the highest benefits attributed to any brand can be realized at the END VALUE — focusing on escaping the competition all together, offers the highest payback. And we agreed to give neural networks a try — without further delays!
3 months later, INM took the podium at one of the nuclear medicine users group conference — organized by Elscint. Just before our presentation, a medical team from Kansas presented their findings linked to a myocardial perfusion analysis. They used conventional statistical methods and reported detection accuracy of 62%.
We were next. Using exactly the same SPECT machines, INM reached 92% of average accuracy — over 10 different tests. All the results were neatly tabulated and our neural networks processed 10x as many input variables, as the Kansas team did!
Needless to say, Nathan Hermony was all smiles! He never requested INM to pitch in front of another hospital, again. Six months later, on Nov 25, 1998 — Elscint was successfully acquired by GE Medical.
THE BOTTOM LINE
What I learned from that experience, was how suitable neural networks are for processing huge amounts of healthcare data. Without any prior medical training, we could quickly extract a huge number of unstructured input parameters — and train the system to recognize the disease. All we asked the doctors for — was to properly label healthy vs. unhealthy hearts.
At some point, during the training phase, our neural networks plateaued — and reached the best generalization level they could. No amount of additional training improved the overall accuracy. In fact, one should be careful to prevent overtraining of neural networks on training sets. This could cause performance degradation on unseen data.
Yes, deep learning is not perfect, but it’s extremely fast and efficient. And since recognition accuracy is not 100% — a final decision should be always reserved for a qualified professional.
Oleg Feldgajer is President & CEO of Canada Green ESCO Inc. Oleg is positioning the company to become a leader in financing AI enhanced green energy projects and ventures. CGE’s mission is to guide DISRUPTIVE businesses in ENERGY & TRANSPORTATION toward profitable business models. Oleg is passionate about such mission, and firmly believes that without AI based innovation, we will all prematurely choke on polluted air and dirty water. CGE delivers 100% financing (levered and unlevered) to its clients — and utilizes large equity pools, and non-recourse debt. Oleg offers creative, fresh ideas to open-minded businesses — that embrace both: logic AND opportunistic intuition. CGE stands against mediocrity & its modus operandi is quite simple: If CGE is not invited to join your BOD, or Advisory Board — we failed!
