Determining Top Fully Connected Layer's Hidden Neuron Count for Transfer Learning, Using Knowledge Distillation: a Case Study on Chest X-Ray Classification of Pneumonia and COVID-19.
Deep convolutional neural network (CNN)-assisted classification of images is one of the most discussed topics in recent years. Continuously innovation of neural network architectures is making it more correct and efficient every day. But training a neural network from scratch is very time-consuming...
| Publicado en: | Journal of Digital Imaging Vol. 34; no. 6; pp. 1349 - 1359 |
|---|---|
| Autor principal: | |
| Formato: | diagnostic images equations & formulas research tables/charts Journal Article |
| Publicado: |
Springer Nature
Dec2021
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=154097224&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 154097224 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 08971889 DOQ jtl: Journal of Digital Imaging issn: 08971889 maglogo: N pubinfo: dt: Dec2021 vid: 34 iid: 6 pid: 237 pub: Springer Nature place: New York, New York artinfo: ui: 154097224 152702867 154097224 154097224 10.1007/s10278-021-00518-2 154097224 ppf: 1349 ppct: 10 formats: fmt: @attributes: type: P tig: atl: Determining Top Fully Connected Layer's Hidden Neuron Count for Transfer Learning, Using Knowledge Distillation: a Case Study on Chest X-Ray Classification of Pneumonia and COVID-19. aug: au: Ghosh, Ritwick affil: Department of Mining Engineering, Indian Institute of Engineering Science and Technology, P.O, Shibpur, 711103, Botanical Garden, Howrah, West Bengal, India sug: subj: Digital Imaging Classification Neural Networks (Computer) Methods Transfer (Psychology) Learning Methods Machine Learning Utilization Radiography, Thoracic Classification Pneumonia Radiography COVID-19 Radiography Neurons Human Knowledge Models, Educational Diagnostic Imaging Time Factors ab: Deep convolutional neural network (CNN)-assisted classification of images is one of the most discussed topics in recent years. Continuously innovation of neural network architectures is making it more correct and efficient every day. But training a neural network from scratch is very time-consuming and requires a lot of sophisticated computational equipment and power. So, using some pre-trained neural network as feature extractor for any image classification task or "transfer learning" is a very popular approach that saves time and computational power for practical use of CNNs. In this paper, an efficient way of building full model from any pre-trained model with high accuracy and low memory is proposed using knowledge distillation. Using the distilled knowledge of the last layer of pre-trained networks passes through fully connected layers with different hidden layers, followed by Softmax layer. The accuracies of student networks are mildly lesser than the whole models, but accuracy of student models clearly indicates the accuracy of the real network. In this way, the best number of hidden layers for dense layer for that pre-trained network with best accuracy and no-overfitting can be found with less time. Here, VGG16 and VGG19 (pre-trained upon "ImageNet" dataset) is tested upon chest X-rays (pneumonia and COVID-19). For finding the best total number of hidden layers, it saves nearly 44 min for VGG19 and 36 min and 37 s for VGG16 feature extractor. pubtype: Academic Journal doctype: diagnostic images equations & formulas research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|