Deep Learning Identifies Images of Covid-19 in Lung X-Rays

neural network brain image over lung x-rays

The recent success in applying Cognex deep learning software to the identification of images of COVID-19 in lung X-rays – beating several other deep learning-based solutions developed by life science research groups from around the world – was peer-reviewed and published by the leading artificial intelligence journal, SN Computer Science, a Spring publication. “Identification of Images of COVID-19 from Chest X-rays Using Deep Learning: Comparing Cognex VisionPro Deep Learning 1.0 Software with Open-Source Convolutional Neural Networks”1 was published in March 2021 after several months of scientific peer review.

cat scans of lungs in VisionPro Deep Learning software environment

Heat maps of lung scans

The paper was developed on a public chest X-ray dataset provide by the University of Waterloo. As part of the experiment that was funded by Cognex, the Cognex Life Sciences team applied VisionPro software to the problem of identifying images of COVID-19 by analyzing chest X-rays with COVID-19 positive results vs. chest X-rays from healthy patients or patients with non-COVID-19 pneumonia. In a sequel paper, the team compares the efficacy of applying VisionPro software to identify images of COVID-19 in CT scans. The paper also explores how to program the software even faster and easier, again with significantly positive results.   

X-Rays, CT Scans, and COVID-19

Medical images such as X-rays can give doctors and radiologists visual evidence that COVID-19 laboratory tests are accurate. Moreover, deep learning-powered software, which is programmed similar to how a child learns rather than through complex mathematics, can lighten the workload of clinicians by analyzing thousands of medical images and identifying anomalies that refute or support a diagnosis. 

An obstacle can be that the most popular open-source deep learning tools can require substantial programming expertise to use. It is not practical to expect healthcare workers such as doctors, radiologists, and other clinicians to master these tools. 

This summer, a team of artificial intelligence (AI) experts at Cognex set out to surmount this obstacle with a base hypothesis: Could Cognex’s industrial automation software provide an easy-to-use alternative to the world’s top open-source deep learning tools, one that matches their performance?  The study, titled “Identification of images of COVID-19 from Chest X-rays using Deep Learning: Comparing Cognex VisionPro Deep Learning 1.0 Software with Open-Source Convolutional Neural Networks,” compared the Cognex VisionPro computer neural network (CNN) to several prominent open-source CNNs for X-ray evaluation, including VGG19, ResNet, DenseNet, Inception and also COVID-NET an AI-created CNN which has been especially tailored towards the detection of COVID-19 in chest X-Rays, developed by the University of Waterloo. 

Since passing peer review, the paper, authored by Arjun Sarkar, Joerg Vandenhirtz, Jozsef Nagy, David Bacsa and Mitchell Riley, all of whom work on Cognex’s Life Sciences team, has drawn attention from several major research publishers. 

“We were surprised to learn that it’s easy for the software to differentiate between the pathologies that show up on X-rays,” said Vandenhirtz, Cognex’s Senior AI Expert for Life Sciences. “It is nearly impossible for humans to figure out differences in X-ray images with different pathologies. Five radiologists can give five different opinions on these kinds of images.” 

Study 1: VisionPro Deep Learning Stands Out, Stands Above

Cognex’s study built on the findings of researchers at the University of Waterloo in Ontario, Canada, titled “COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images.” Using nearly 14,000 chest X-rays in a data set called COVID-X. Co-authors Linda Wang and Alexander Wong used artificial intelligence to build COVID-Net, a sophisticated neural network that analyzed the X-rays and learned to identify lungs that had tell-tale signs of COVID-19.

A rel="noopener noreferrer" measurement called the F-score assesses the overall accuracy of a deep learning system, which attempts to accurately predict patterns and anomalies on digital images. Essentially, the F-score is the ratio of correct vs. wrong predictions generated by the deep learning system. 

Cognex researchers trained their deep learning tools on nearly 14,000 X-ray images in the COVID-X data set. The images were divided into three categories: normal, non-COVID-19 pneumonia, and COVID-19. As this table comparing multiple deep learning packages shows, COVID-Net generated strong predictive results, with F-Scores ranging from 92.6% on normal images to 94.7% for COVID-19 images. VisionPro 1.0 did even better, with F-Scores of 95.6% on normal X-rays and 97.0% on COVID-19 X-rays.

 

Study 2: VisionPro Deep Learning Widens Its Lead with CT Scans 

A more recent Cognex paper developed by the same research team looks beyond chest X-rays to chest CT scans. While many studies have demonstrated success in detecting images of COVID-19 using deep learning with CT scans and X-rays, most of the deep learning architectures need extensive programming because they do not offer a graphical user interface (GUI) to train the system. It can be difficult for radiologists who lack knowledge in deep learning or programming to use these programs, let alone train them.  

cat scans of lungs in VisionPro Deep Learning software environment

Heat maps of lung scans

“A major problem with adopting deep learning software is that a standard package, such as TensorFlow, requires programmers to build their models in a text-based terminal interface,” continued Vandenhirtz. “VisionPro, by contrast, has a user-friendly GUI that requires no programming experience. If you can learn Microsoft Office, then you can learn VisionPro.” Vandenhirtz added that, Arjun Sarkar, lead researcher on the project, had never worked with VisionPro before joining Cognex. Within two months, Sarkar learned the program, conducted the research, and wrote up the findings. A conventional deep learning study might require man-years to build a network, develop models, and train algorithms. VisionPro dramatically reduces that time frame. 

With efficacy and ease-of-use as two critical considerations for subsequent investigations, Cognex’s latest investigation looked at VisionPro’s success in identifying images of COVID-19 and non-COVID pneumonia, as well as how much training is necessary to achieve high F-scores. The subsequent paper, “Identification of images of COVID-19 from Chest Computed Tomography (CT) scans using Deep Learning: Comparing COGNEX VisionPro Deep Learning 1.0 Software with Open-Source Convolutional Neural Networks,” uses a chest CT image data set from Linda Wang’s team at the Vision and Image Processing Lab of the University of Waterloo, which includes more than 100,000 expertly tagged images. In addition to the CNNs, Cognex’s VisionPro was benchmarked against other state-of-the-art CNNs, including the University of Waterloo’s CNN architectures COVID-Net-CT-A and COVID-Net-CT-B, as well as Google’s latest CNN architecture Xception.

As the table below illustrates, Cognex’s VisionPro performed slightly better than all other CNN network architectures with F-Scores >99.4 in all three classes (normal, non-COVID-Pneumonia and COVID-19). This initial investigation broke the original 100,000 image CT scan data set into two groups: a training group of 61,783 images and a “test” group of 21,191 images, which were analyzed after training by each CNN. 
 

f-score bar char results for cat scans

To gain more insights into how many images are necessary to “train” an existing X-ray CNN to evaluate normal, COVID-19, and pneumonia conditions, Cognex started over, training the Cognex CNN on 26,338 images instead of more than 61,000. As the table below shows, F-scores for each CNN were compared. Cognex’s VisionPro dominated the other CNN architectures with F-Scores >99.1 for all three classes of images (normal, COVID-19, pneumonia), while all other CNNs dropped to F-Scores ranging from the high eighties to mid-nineties, especially in the two relevant pathologic classes: pneumonia and COVID-19.

f-score bar char results for cat scans

Deep Learning Gives Radiologists Powerful Diagnostic Tool

While the findings of Cognex’s first two studies still require verification by other medical researchers, initial results are promising. Furthermore, the software has not been approved yet for medical use.   

Vandenhirtz said the company’s primary short-term interest is telling the global medical community about the capabilities of this kind of software. It also could prove useful in areas such as ophthalmology, which rely on pictures of the internal mechanisms of the eye. 

For all their capabilities, deep learning algorithms cannot fully replace the wisdom of human clinicians, Vandenhirtz said. But, like the stethoscope or blood pressure cuff, it is a useful tool to help health-care professionals perform their job at a high level. 

“We don’t believe, at least in the short-to-medium term, that AI will be capable of making a diagnosis,” he concluded. “VisionPro can make recommendations, but ultimately the radiologist has to decide what the image means.” 

Access the SN Computer Science article here: Identification of Images of COVID-19 from Chest X-rays Using Deep Learning: Comparing COGNEX VisionPro Deep Learning 1.0™ Software with Open-Source Convolutional Neural Networks.

1Springer article citation: Sarkar, A., Vandenhirtz, J., Nagy, J. et al. Identification of Images of COVID-19 from Chest X-rays Using Deep Learning: Comparing COGNEX VisionPro Deep Learning 1.0™ Software with Open Source Convolutional Neural Networks. SN COMPUT. SCI. 2, 130 (2021). https://doi.org/10.1007/s42979-021-00496-w

More Posts on

GET ACCESS TO SUPPORT & TRAINING FOR PRODUCTS & MORE

Join MyCognex