Yiqiao Yin | Seeing Deep into the Lungs with Deep Learning

Nov 9, 2022 | Engineering and Tech, Health and Medicine

X-rays and other forms of medical imaging let doctors peer into the body, revealing the internal structure of organs and tissues without invasive surgery. Doctors use the results to identify abnormalities such as broken bones, diagnose diseases such as cancer, or even monitor the health of a foetus within the womb. Although this technology is remarkable, the images aren’t useful in isolation. Experts must analyse the resulting data and parse what is healthy or unhealthy from the noise. Yiqiao Yin, Jaiden Schraut, Leon Liu and Jonathan Gong have created new machine learning technologies to support that crucial interpretation, focusing on X-rays and lung health.

A Window into the Body

By 2010, over 5 billion medical imaging studies had been conducted, including X-ray images revealing the shapes of bone fractures, the presence of breast tumours, the location of kidney stones, the health of the lungs, and much more.

Skin, muscle, bone – and just about every part of your body – is opaque to visible light. This means that these tissues absorb light with wavelengths of between 380 and 700 nanometres: the visible part of the electromagnetic spectrum. However, if we go down to light with wavelengths of just 10 nanometres and below, we find X-rays. The lower the wavelength of light, the more energy it contains, and so X-rays carry much more energy than visible light.

These X-rays tend to pass straight through soft tissues, barely interacting with them, just as visible light passes through glass. However, harder tissues contain heavier elements such as calcium, which can absorb X-rays. This means X-rays will pass through soft tissue, but bones, teeth and other hard tissues will be opaque, blocking X-rays just as your hand blocks visible light.

Decoding the Images

X-ray imaging makes it easy to spot a broken bone, and even to diagnose what kind of fracture has occurred – is it a partial hairline fracture, or a break all the way through? However, not all uses of X-rays are as simple as spotting cracks in bone.

Yiqiao Yin and his colleagues, Jaiden Schraut, Leon Liu and Jonathan Gong, focus on chest X-rays for lung health, which require careful interpretation. Machine learning can support this interpretation by automating some aspects of the data processing, but this requires a delicately constructed artificial intelligence system, as well as the trust of both the doctor and patient.

Yin and his colleagues have developed new cutting-edge algorithms to process the raw data for doctors, while providing them with more context and additional information than existing solutions.

While X-rays of broken bones tend to be 2D images, in more complex health issues, multiple images can be stitched together to create a 3D representation. This process gives more detail, and allows doctors to navigate the 3D space to make more precise conclusions. However, this can mean that it is even more complex to make those conclusions.

One important step is segmentation, in which the pixels of the images are divided according to their brightness, texture, colour, or other criteria. This can be done manually by radiologists, but this is incredibly time-consuming, and introduces the potential for human error in interpreting the data.

This is especially challenging in the lungs, which evolved to be incredibly complex to maximise surface area for gas exchange. This means that, for example, it can be difficult to distinguish the damage done to the lungs by pneumonia versus COVID-19, which each have different treatments. Recent research has produced models that can automate the process of segmentation, and Yin’s team has further developed this approach by building a hybrid model that both segments and classifies the data.

Inside the Algorithm

Neural networks have become a popular tool for processing lung scans. These are a powerful class of algorithms that are inspired by the structure of the brain, consisting of ‘neurons’ and connections between those neurons. You can input data in one end of the network, and it will flow through to an output layer, just as information enters your eye and travels through the neurons in your brain. This processes the raw data, generating the image we see in our mind.

However, each connection must be fine-tuned for the model to work. This is hugely complex, and can’t be done manually. Instead, Yin and his colleagues trained and tested their neural network using the data and results from 21,165 chest X-ray images.

Ironically in this case, a perennial drawback of neural networks is their lack of transparency. They provide incredibly accurate results, but it is almost impossible to figure out exactly how they came to that conclusion. Yin and his team tackled this by combining the classification and segmentation processes.

While one part of their model builds a segmented 3D image, another part identifies any pixels or areas that were particularly useful in the neural network’s recommended diagnosis. These can be combined to produce a heatmap on top of the 3D image, automatically highlighting those areas of the lungs that distinguish between different diagnoses.

This means doctors get a recommended classification (for example, the model might recommend a diagnosis of pneumonia induced lung cancer), while also presenting a 3D heatmap for them to explore. This boosts the trust in the model, since doctors can see the underlying data displayed clearly, and patients can be lead through an explanation of how the conclusion was reached, and where exactly the problems lie.

The team tested their final model using more real-life examples from the COVID-19 Radiography database, and their final model achieved an accuracy of 95%, meaning 19 out of 20 of their diagnoses matched the data. This is a remarkable achievement, and provides doctors with an excellent tool to interpret and explain the diagnosis.

Improving Medical Diagnostics

The more accurately we can diagnose patients’ diseases, the more effective our treatments are. For instance, distinguishing bacterial pneumonia from COVID-19 is the difference between a simple course of antibiotics and two weeks of worsening symptoms.

Furthermore, although this combination of automated segmentation and classification is relatively novel, this approach isn’t restricted to just chest X-rays: it could be applied to many essential medical exams, such as CT scans to diagnose thyroid cancer, MRI to identify stroke in the brain, ultrasound to measure the health of a foetus, and many others.

The more we improve our non-invasive scanning technologies, the more patients can be spared mis-diagnoses, delays, or even exploratory surgery, all of which can cause unnecessary harm. Yin’s technology advances our ability to image, diagnose, and ultimately ensure patients receive the treatment they need.






Yiqiao Yin
Senior Data Scientist
Princeton, NJ

Yiqiao Yin studied machine learning and applied statistics at Columbia University, obtaining an MA in 2019 for his work on predicting relapse in breast cancer patients, achieving an accuracy of 92% compared to the industry standard of 60–70%. In 2020, Yin became a data scientist at Bayer Crop Science in New York, designing and prototyping new algorithms and techniques for the Harvest Analytic Division. Since 2020, Yin has also worked part-time as the Head of Curriculum Development at Veritas AI in New York, developing e-learning and interactive teaching materials for a pre-college AI program. This year, he joined LabCorp in New Jersey, leading projects that tackle drug development AI challenges using tools such as convolutional neural networks, long-short term memory, and recurrent neural networks. He’s worked on technologies that assist pathologists and radiologists with prognoses, and continues to develop state-of-the-art tools that help deliver safe and effective healthcare.


E: Eagle0504@Gmail.com

W: https://www.yiqiao-yin.com/


Jaiden Schraut

Leon Liu

Jonathan Gong


JX Shraut, L Liu, J Gong, Y Yin, A multi-output network with U-net enhanced class activation map and robust classification performance for medical imaging analysis, Discover Artificial Intelligence, 2023, 3, 1. https://doi.org/10.1007/s44163-022-00045-1


We encourage all formats of sharing and republishing of our articles. Whether you want to host on your website, publication or blog, we welcome this. Find out more

Creative Commons Licence (CC BY 4.0)

This work is licensed under a Creative Commons Attribution 4.0 International License. Creative Commons License

What does this mean?

Share: You can copy and redistribute the material in any medium or format

Adapt: You can change, and build upon the material for any purpose, even commercially.

Credit: You must give appropriate credit, provide a link to the license, and indicate if changes were made.




Professor Thomas Feuerstein | A Paradox Explained: Why a Super Selective β1-blocker Works in Acute Heart Failure

Professor Thomas Feuerstein | A Paradox Explained: Why a Super Selective β1-blocker Works in Acute Heart Failure

β1-adrenoceptors are found in the heart where they bind neurotransmitters/hormones such as noradrenaline and adrenaline. The binding of these β1-adrenoceptors agonists activates a response in the heart muscle that helps regulate the heart’s beat and contractile force. Drugs that block the action of these receptors are an established treatment for those suffering from left ventricular dysfunction due to chronic heart failure. However, their use in the acute setting is controversial. Professor Thomas Feuerstein of the University Hospital Freiburg in Germany and Dr Günther Krumpl of the Medical Research Network in Vienna are challenging these sceptical attitudes through mathematical modelling.

Dr Xu Hannah Zhang | p38γ – It’s More than Just a Kinase

Dr Xu Hannah Zhang | p38γ – It’s More than Just a Kinase

Kinases take energy from adenosine triphosphate molecules to fuel other molecules in performing vital biological processes. Dr Xu Hannah Zhang at City of Hope, Los Angeles, has worked with colleagues to better understand the p38 family of kinases, and in particular, how the p38γ isoform plays a role in cancer. Her work has shown – for the first time – that p38γ is much more than just a kinase, and her recent studies point to new avenues in the search for cutaneous T-cell lymphoma therapeutics.

Dr Daisuke Minakata – Sunshine and Organic Molecules in Water

Dr Daisuke Minakata – Sunshine and Organic Molecules in Water

Organic molecules dissolved in rivers, lakes, seas and oceans are essential to plant and animal life. Some of these molecules are also degraded and enter a complex cycle of carbon, nitrogen and sulphur containing compounds. Surprisingly, scientists currently have a limited understanding of the fate of these molecules. Dr Daisuke Minakata and his colleagues from Michigan Technological University are involved in an ambitious programme to overcome this critical knowledge gap.

Professor Christian Laforsch | Professor Andreas Greiner – Microplastics: Solutions for a Persistent Pollutant

Professor Christian Laforsch | Professor Andreas Greiner – Microplastics: Solutions for a Persistent Pollutant

Plastics have revolutionised human existence. Medicine, technology, agriculture and construction all rely on highly durable plastic materials. However, the enduring legacy of plastics extends far beyond our cities and towns. Everywhere we look, from the deepest parts of the oceans to alpine glaciers, we find tiny fragments called microplastics. Recently, the collaborative research centre, ‘CRC 1357 Microplastic’, at the University of Bayreuth was granted a second funding phase by the German Science Foundation, to continue their intensive research into microplastics. The CRC 1357 team studies the formation and behaviour of microplastics in the environment and their long-term effects on soils, plants, organisms, and ecosystem processes. Through their research, the University of Bayreuth will be able to contribute to ground-breaking recommendations for policy-makers, industry and society.