Scaling Up a Multispectral Resnet-50 to 128 GPUs

Rocco Sedona; Gabriele Cavallaro; Jenia Jitsev; Alexandre Strube; Morris Riedel; Matthias Book

doi:10.1109/IGARSS39084.2020.9324237

Scaling Up a Multispectral Resnet-50 to 128 GPUs

Rocco Sedona, Gabriele Cavallaro, Jenia Jitsev, Alexandre Strube, Morris Riedel, Matthias Book

Faculty of Industrial Engineering, Mechanical Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Similarly to other scientific domains, Deep Learning (DL) holds great promises to fulfil the challenging needs of Remote Sensing (RS) applications. However, the increase in volume, variety and complexity of acquisitions that are carried out on a daily basis by Earth Observation (EO) missions generates new processing and storage challenges within operational processing pipelines. The aim of this work is to show that High-Performance Computing (HPC) systems can speed up the training time of Convolutional Neural Networks (CNNs). Particular attention is put on the monitoring of the classification accuracy that usually degrades when using large batch sizes. The experimental results of this work show that the training of the model scales up to a batch size of 8,000, obtaining classification performances in terms of accuracy in line with those using smaller batch sizes.

Original language	English
Title of host publication	2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1058-1061
Number of pages	4
ISBN (Electronic)	9781728163741
DOIs	https://doi.org/10.1109/IGARSS39084.2020.9324237
Publication status	Published - 26 Sept 2020
Event	2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Virtual, Waikoloa, United States Duration: 26 Sept 2020 → 2 Oct 2020

Publication series

Name	International Geoscience and Remote Sensing Symposium (IGARSS)

Conference

Conference	2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020
Country/Territory	United States
City	Virtual, Waikoloa
Period	26/09/20 → 2/10/20

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Other keywords

classification
convolutional neural network
Distributed deep learning
high performance computing
residual neural network
sentinel-2

Access to Document

10.1109/IGARSS39084.2020.9324237

Cite this

Sedona, R., Cavallaro, G., Jitsev, J., Strube, A., Riedel, M., & Book, M. (2020). Scaling Up a Multispectral Resnet-50 to 128 GPUs. In 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Proceedings (pp. 1058-1061). Article 9324237 (International Geoscience and Remote Sensing Symposium (IGARSS)). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IGARSS39084.2020.9324237

@inproceedings{51fb2c35e789479686d85c4c0a2d4488,

title = "Scaling Up a Multispectral Resnet-50 to 128 GPUs",

abstract = "Similarly to other scientific domains, Deep Learning (DL) holds great promises to fulfil the challenging needs of Remote Sensing (RS) applications. However, the increase in volume, variety and complexity of acquisitions that are carried out on a daily basis by Earth Observation (EO) missions generates new processing and storage challenges within operational processing pipelines. The aim of this work is to show that High-Performance Computing (HPC) systems can speed up the training time of Convolutional Neural Networks (CNNs). Particular attention is put on the monitoring of the classification accuracy that usually degrades when using large batch sizes. The experimental results of this work show that the training of the model scales up to a batch size of 8,000, obtaining classification performances in terms of accuracy in line with those using smaller batch sizes.",

keywords = "classification, convolutional neural network, Distributed deep learning, high performance computing, residual neural network, sentinel-2",

author = "Rocco Sedona and Gabriele Cavallaro and Jenia Jitsev and Alexandre Strube and Morris Riedel and Matthias Book",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 ; Conference date: 26-09-2020 Through 02-10-2020",

year = "2020",

month = sep,

day = "26",

doi = "10.1109/IGARSS39084.2020.9324237",

language = "English",

series = "International Geoscience and Remote Sensing Symposium (IGARSS)",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1058--1061",

booktitle = "2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Proceedings",

address = "United States",

}

Sedona, R, Cavallaro, G, Jitsev, J, Strube, A, Riedel, M & Book, M 2020, Scaling Up a Multispectral Resnet-50 to 128 GPUs. in 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Proceedings., 9324237, International Geoscience and Remote Sensing Symposium (IGARSS), Institute of Electrical and Electronics Engineers Inc., pp. 1058-1061, 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020, Virtual, Waikoloa, United States, 26/09/20. https://doi.org/10.1109/IGARSS39084.2020.9324237

Scaling Up a Multispectral Resnet-50 to 128 GPUs. / Sedona, Rocco; Cavallaro, Gabriele; Jitsev, Jenia et al.
2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2020. p. 1058-1061 9324237 (International Geoscience and Remote Sensing Symposium (IGARSS)).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Scaling Up a Multispectral Resnet-50 to 128 GPUs

AU - Sedona, Rocco

AU - Cavallaro, Gabriele

AU - Jitsev, Jenia

AU - Strube, Alexandre

AU - Riedel, Morris

AU - Book, Matthias

PY - 2020/9/26

Y1 - 2020/9/26

N2 - Similarly to other scientific domains, Deep Learning (DL) holds great promises to fulfil the challenging needs of Remote Sensing (RS) applications. However, the increase in volume, variety and complexity of acquisitions that are carried out on a daily basis by Earth Observation (EO) missions generates new processing and storage challenges within operational processing pipelines. The aim of this work is to show that High-Performance Computing (HPC) systems can speed up the training time of Convolutional Neural Networks (CNNs). Particular attention is put on the monitoring of the classification accuracy that usually degrades when using large batch sizes. The experimental results of this work show that the training of the model scales up to a batch size of 8,000, obtaining classification performances in terms of accuracy in line with those using smaller batch sizes.

AB - Similarly to other scientific domains, Deep Learning (DL) holds great promises to fulfil the challenging needs of Remote Sensing (RS) applications. However, the increase in volume, variety and complexity of acquisitions that are carried out on a daily basis by Earth Observation (EO) missions generates new processing and storage challenges within operational processing pipelines. The aim of this work is to show that High-Performance Computing (HPC) systems can speed up the training time of Convolutional Neural Networks (CNNs). Particular attention is put on the monitoring of the classification accuracy that usually degrades when using large batch sizes. The experimental results of this work show that the training of the model scales up to a batch size of 8,000, obtaining classification performances in terms of accuracy in line with those using smaller batch sizes.

KW - classification

KW - convolutional neural network

KW - Distributed deep learning

KW - high performance computing

KW - residual neural network

KW - sentinel-2

UR - http://www.scopus.com/inward/record.url?scp=85101965031&partnerID=8YFLogxK

U2 - 10.1109/IGARSS39084.2020.9324237

DO - 10.1109/IGARSS39084.2020.9324237

M3 - Conference contribution

AN - SCOPUS:85101965031

T3 - International Geoscience and Remote Sensing Symposium (IGARSS)

SP - 1058

EP - 1061

BT - 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020

Y2 - 26 September 2020 through 2 October 2020

ER -

Sedona R, Cavallaro G, Jitsev J, Strube A, Riedel M , Book M. Scaling Up a Multispectral Resnet-50 to 128 GPUs. In 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2020. p. 1058-1061. 9324237. (International Geoscience and Remote Sensing Symposium (IGARSS)). doi: 10.1109/IGARSS39084.2020.9324237

Scaling Up a Multispectral Resnet-50 to 128 GPUs

Abstract

Publication series

Conference

Bibliographical note

Other keywords

Access to Document

Fingerprint

Cite this