VisionPro Deep Learning Shows Early Promise in Identifying Images of Covid-19 in Chest X-Rays
When the Covid-19 pandemic started raising alarms around the world, the deep learning experts at Cognex started wondering if their technology could help healthcare professionals mount an effective defense.
The alarm proved justified: As of mid-September 2020, Covid-19 had infected nearly 30 million people worldwide and claimed almost a million lives. Clinicians everywhere faced similar challenges: Depending on laboratory tests to confirm Covid cases was time-consuming, potentially delaying diagnosis and treatment. X-rays and other medical imaging technologies could provide rapid confirmation of a Covid diagnosis, but it was easy to misinterpret the meaning of these images.
Cognex’s deep learning team looked at these challenges and realized that a software package they built for automating and optimizing production lines might provide a solution to the medical-imaging component of pandemic response.
The Value of Combining Deep Learning and Medical Imaging
Medical images like X-rays are critical to confirming a COVID-19 diagnosis — giving doctors and radiologists visual evidence that laboratory tests are accurate. Moreover, deep learning software can lighten the workload of clinicians by analyzing thousands of medical images and identifying anomalies that refute or support a diagnosis.
There is just one roadblock: The most popular open-source deep learning tools are difficult to use and require substantial programming expertise. It is not practical to expect healthcare workers, such as doctors, radiologists, and other clinicians, to master these tools.
A team of AI experts at Cognex set out to surmount this obstacle with a base question: Could Cognex’s industrial automation software provide an easy-to-use alternative that can match the performance of the world’s top open-source deep learning tools?
The opening test of this hypothesis showed strong potential. According to research by a five-person team of Cognex deep learning experts, the company’s state-of-the-art machine vision software equaled or surpassed the accuracy of the world’s leading open-source deep learning tools.
The study, titled “Identification of images of COVID-19 from Chest X-rays using Deep Learning: Comparing Cognex VisionPro Deep Learning 1.0 Software with Open Source Convolutional Neural Networks,” has drawn attention from major research publishers. The co-authors were Arjun Sarkar, Joerg Vandenhirtz, Jozsef Nagy, David Bacsa and Mitchell Riley, all of whom work on Cognex’s Life Sciences team.
“We were surprised to learn that it’s easy for the software to differentiate between the pathologies that show up on X-rays,” said Vandenhirtz, Cognex’s Senior AI Expert for Life Sciences. "It is nearly impossible for humans to figure out differences in X-ray images with different pathologies. Five radiologists can give five different opinions on these kinds of images.”
Vandenhirtz coordinated the study to help extend the company’s premium machine-vision technology into the healthcare and life sciences space. The global coronavirus pandemic provided the urgency, while COVIDx, a large data set of COVID-19 chest X-rays provided test imagery for the study. He hired Sarkar, a master’s degree candidate studying biomedical engineering at the University of Applied Sciences in Aachen, Germany, to conduct the experiment and summarize his findings in the research report.
Sarkar had a strong background using TensorFlow, the leading deep-learning platform from Google, Vandenhirtz said. TensorFlow requires programmers to build their models in a text-based terminal interface. VisionPro Deep Learning, by contrast, has a user-friendly GUI that requires no programming experience.
Building on a Study of COVID and Deep Learning
Cognex’s study built on the findings of a research study at the University of Waterloo in Ontario, Canada. The study, titled “COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images,” collected nearly 14,000 chest X-rays in a data set called COVIDx. Co-authors Linda Wang and Alexander Wong used open-source deep learning packages to build COVID-Net, a sophisticated neural network that analyzed the X-rays and learned to identify lungs that had tell-tale signs of COVID-19.
A group of University of Waterloo researchers launched a startup called DarwinAI to develop commercial deep learning software to tap the value of resources such has COVID.Net, which holds immense promise but still faces a fundamental usability challenge.
“Right now, it's just a very technical kind of implementation that data scientists could leverage but certainly not a radiologist or healthcare worker, so it needs to be wrapped in a proper application UI that's fairly easy to use and that somebody who's not deeply technical can leverage,” DarwinAI CEO Sheldon Fernandez said in an interview with CDNet.
The Cognex researchers understood the implications of these limitations. VisionPro Deep Learning was developed for Cognex’s clients in manufacturing. Its developers designed it specifically so that factory managers and technicians could use deep learning to analyze images on their production lines to maintain quality control and keep defective and damaged products out of the marketplace.
In an automotive factory, for instance, Cognex’s machine vision cameras take digital pictures of parts like fenders and engine blocks. VisionPro Deep Learning scans these images for scratches, dents and other anomalies that human inspectors often miss. Flagging these flaws in advance keeps production lines more productive and strengthens product quality. It can also be used to classify parts or defects, as well as locate parts and verify assemblies. These types of inspection tasks are often still done manually or not all because they essentially require the involvement of human judgement.
How VisionPro Deep Learning Performed on the COVIDx Data Set
A measurement called F-score assesses the overall accuracy of a deep learning system, which attempts to accurately predict patterns and anomalies on digital images. Cognex’s researchers analyzed nearly 14,000 X-ray images in the COVID-Net data set. The images were divided into three categories: normal, non-COVID-19 pneumonia and COVID-19.
As this table comparing multiple deep learning packages shows, COVID-Net generated strong predictive results, ranging from 92.6% on normal images to 94.7% for COVID-19 images. VisionPro Deep Learning did even better, with 95.6% on normal X-rays and 97.0% on COVID-19 X-rays.
Of course, this is just one study. Though the Cognex team used industry standard techniques for research and statistical analysis, it remains to be seen if other researchers can replicate the results.
Vandenhirtz said the company’s primary short-term interest is telling the global medical community about the capabilities of this kind of software, which also has been showing promising results on CT (computed tomography) scans. It also could prove useful in areas like ophthalmology, which rely on pictures of the retina, or digital pathology, which uses microscopic images of histologic slides.
For all their capabilities, deep learning algorithms cannot fully replace the wisdom of human clinicians, Vandenhirtz said. But, like the stethoscope or blood pressure cuff, it is a useful tool to help healthcare professionals perform their job at a high-level.
In this context Cognex VisionPro Deep Learning software provides a useful heatmap feature, which highlights areas in the image that are important for the classification. Yellow to red colored areas are important, while areas colored green to blue had no importance for the decision algorithm.
In the real world, this heatmap feature allows the tool to not only give a recommendation for the potential diagnosis (i.e. Covid-19 positive or negative) but also identifies the areas where it detected the corresponding disease symptoms. This is important because it helps radiologists zero in on a specific region of the image for them to verify or counter the AI diagnosis, thus preventing it from making the right decision for the wrong reason.
“We don’t believe, at least in the short to medium term, that AI will be capable of making a diagnosis,” he concluded. “VisionPro Deep Learning can make recommendations, but ultimately the radiologist has to decide what the image means.”
AI will not replace the radiologists, he added, but it will replace the radiologists who are not using AI.