VisionPro Deep Learning Helps Radiologists Identify Images of COVID-19 or Pneumonia
The novel coronavirus disease — named COVID-19 by the World Health Organization — is caused by a new coronavirus class known as the SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2). This single-stranded RNA (ribonucleic acid) virus can cause severe respiratory infections, potentially leading to hospitalization and death. Nearly 55 million people have been infected worldwide with 1.35 million deaths.
Today, scientists are working diligently on therapeutics and vaccines to protect the general population from getting COVID-19. Until their efforts bear effective fruit, one of the best solutions has been detecting the virus in its early stages and then isolating infected people through quarantine, preventing spread of the disease. The real-time reverse transcription-polymerase chain reaction test using nasopharyngeal swabs measures RNA levels in the body and has been used for the most accurate COVID-19 diagnosis. However, the test takes hours to conduct and backlogs can lead to even longer wait times. A better and more accurate method for diagnosing COVID-19 is through X-rays and computed tomography (CT) scans.
In the summer of 2020, a team of medical researchers applied Cognex’s VisionPro Deep Learning (DL) software to the problem of detecting the coronavirus by analyzing chest X-rays, with positive results. In a sequel paper, the team compares the efficacy of applying VisionPro DL software to identify COVID-19 indications in CT scans. The paper also explored how to program the software even faster and easier, again with significantly positive results.
X-rays, CT Scans, and COVID-19
Medical images such as X-rays can give doctors and radiologists visual evidence that COVID-19 laboratory tests are accurate. Moreover, deep learning's neural networks, which are trained similar to how a child learns via examples, can lighten the workload of clinicians by analyzing thousands of medical images and identifying anomalies that refute or support a diagnosis.
There’s just one roadblock: The most popular open-source deep learning tools are difficult to use and require substantial programming expertise. It is not practical to expect health-care workers such as doctors, radiologists, and other clinicians to master these tools.
This summer, a team of artificial intelligence (AI) experts at Cognex set out to surmount this obstacle with a base hypothesis: Could Cognex’s industrial automation software provide an easy-to-use alternative to the world’s top open-source deep learning tools, one that matches their performance? The study, titled “Identification of images of COVID-19 from Chest X-rays using Deep Learning: Comparing Cognex VisionPro Deep Learning 1.0 Software with Open Source Convolutional Neural Networks,” compared the Cognex VisionPro DL computer neural network (CNN) to several prominent open-source CNNs for X-ray evaluation, including VGG19, ResNet, DenseNet, and Inception. Since passing peer review, the paper, authored by Arjun Sarkar, Joerg Vandenhirtz, Jozsef Nagy, David Bacsa and Mitchell Riley, all of whom work on Cognex’s Life Sciences team, has drawn attention from several major research publishers.
“We were surprised to learn that it’s easy for the software to differentiate between the pathologies that show up on X-rays,” said Vandenhirtz, Cognex’s Senior AI Expert for Life Sciences. “It’s is nearly impossible for humans to figure out differences in X-ray images with different pathologies. Five radiologists can give five different opinions on these kinds of images.”
Study 1: VisionPro DL Stands Out, Stands Above
Cognex’s study built on the findings of researchers at the University of Waterloo in Ontario, Canada, titled “COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images.” Using nearly 14,000 chest X-rays in a data set called COVIDx. Co-authors Linda Wang and Alexander Wong used open-source DL packages to build COVID-Net, a sophisticated neural network that analyzed the X-rays and learned to identify lungs that had tell-tale signs of COVID-19.
A measurement called the F-score assesses the overall accuracy of a deep learning system, which attempts to accurately predict patterns and anomalies on digital rel="noopener noreferrer" images. Essentially, the F-score is the percentage of correct predictions generated by the deep learning system.
Cognex’s researchers analyzed nearly 14,000 X-ray images in the COVID-Net data set. The images were divided into three categories: normal, non-COVID-19 pneumonia, and COVID-19. As this table comparing multiple DL packages shows, COVID-Net generated strong predictive results, ranging from 92.6% on normal images to 94.7% for COVID-19 images. VisionPro Deep Learning 1.0 did even better, with 95.6% on normal X-rays and 97.0% on COVID-19 X-rays.
Study 2: VisionPro Deep Learning Widens Its Lead With CT Scans
A more recent Cognex paper developed by the same research team looks beyond chest X-rays to CT scans. While many studies have demonstrated success in detecting images of COVID-19 using deep learning with CT scans and X-rays, most of the deep learning architectures need extensive programming because they do not offer a graphical user interface (GUI) to program the system. It is difficult for radiologists who lack knowledge in deep learning or programming to use these programs, let alone train them.
“A major problem with adopting deep learning software is that a standard package, such as TensorFlow, requires programmers to build their models in a text-based terminal interface,” continued Vandenhirtz. “VisionPro Deep Learning, by contrast, has a user-friendly GUI that requires no programming experience. If you can learn Microsoft Office, then you can learn VisionPro DL.” Vandenhirtz added that, Arjun Sarkar, lead researcher on the project, had never worked with VisionPro DL before joining Cognex. Within two months, Sarkar learned the program, conducted the research, and wrote up the findings. A conventional DL study might require men-years to build a network, develop models, and train algorithms. VisionPro DL dramatically reduces that time frame.
With efficacy and ease-of-use as two critical considerations for subsequent investigations, Cognex’s latest investigation looked at VisionPro DL’s success in identifying normal COVID-19 and non-COVID pneumonia, as well as how much training is necessary to achieve high F-scores. The subsequent paper, “Detection of COVID-19 from Chest Computed Tomography (CT) images using Deep Learning: Comparing COGNEX VisionPro Deep Learning 1.0 Software with Open Source Convolutional Neural Networks,” uses a chest CT image data set from Linda Wang’s team at the Vision and Image Processing Lab of the University of Waterloo, which includes more than 100,000 expertly tagged images. In addition to the CNNs in total we benchmarked Cognex’s VisionPro Deep Learning against other state-of-the-art CNNs, including the University of Waterloo’s CNN architectures COVID-Net-CT-A and COVID-Net-CT-B, as well as Google’s latest CNN architecture Xception.
As the table below illustrates, Cognex’s VisionPro Deep Learning 1.0 performed slightly better than all other CNN network architectures with F-Scores > 99.4 in all three classes (normal, non-COVID-Pneumonia and COVID-19). This initial investigation broke the original 100,000 image CT scan data set into two groups: a training group of 61,783 images and a “test” group of 21,191 images, which were analyzed after training by each CNN.
To gain more insights into how many images are necessary to “train” an existing X-ray CNN to evaluate normal, COVID-19, and pneumonia conditions, Cognex started over, training the Cognex CNN on 26,338 images instead of more than 61,000. As the table below shows, F-scores for each CNN were compared. Cognex’s VisionPro Deep Learning dominated the other CNN architectures with F-Scores > 99.1 for all three classes of images (normal, COVID-19, pneumonia), while all other CNNs dropped to the high F-Scores ranging from the high eighties to mid-nineties, especially in the two relevant pathologic classes: pneumonia and COVID-19.
Deep Learning Gives Radiologists Powerful Diagnostic Tool
While the findings of Cognex’s first two studies still require verification by other medical researchers, initial results are promising. Furthermore, the software has not been approved yet for medical use.
Vandenhirtz said the company’s primary short-term interest is telling the global medical community about the capabilities of this kind of software. It also could prove useful in areas such as ophthalmology, which rely on pictures of the internal mechanisms of the eye.
For all their capabilities, deep learning algorithms cannot fully replace the wisdom of human clinicians, Vandenhirtz said. But, like the stethoscope or blood pressure cuff, it is a useful tool to help health-care professionals perform their job at a high level.
“We don’t believe, at least in the short-to-medium term, that AI will be capable of making a diagnosis,” he concluded. “VisionPro DL can make recommendations, but ultimately the radiologist has to decide what the image means.”