Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN

Yankun Lang; Li Wang; Pew Thian Yap; Chunfeng Lian; Hannah Deng; Kim Han Thung; Deqiang Xiao; Peng Yuan; Steve G.F. Shen; Jaime Gateno; Tianshu Kuang; David M. Alfi; James J. Xia; Dinggang Shen

doi:10.1007/978-3-030-35817-4_16

Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN

Yankun Lang, Li Wang, Pew Thian Yap, Chunfeng Lian, Hannah Deng, Kim Han Thung, Deqiang Xiao, Peng Yuan, Steve G.F. Shen, Jaime Gateno, Tianshu Kuang, David M. Alfi, James J. Xia, Dinggang Shen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

4 Scopus citations

Abstract

Craniomaxillofacial (CMF) landmark localization is an important step for characterizing jaw deformities and designing surgical plans. However, due to the complexity of facial structure and the deformities of CMF patients, it is still difficult to accurately localize a large scale of landmarks simultaneously. In this work, we propose a three-stage coarse-to-fine deep learning method for digitizing 105 anatomical craniomaxillofacial landmarks on cone-beam computed tomography (CBCT) images. The first stage outputs a coarse location of each landmark from a low-resolution image, which is gradually refined in the next two stages using the corresponding higher resolution images. Our method is implemented using Mask R-CNN, by also incorporating a new loss function that learns the geometrical relationships between the landmarks in the form of a root/leaf structure. We evaluate our approach on 49 CBCT scans of patients and achieve an average detection error of 1.75 ± 0.91 mm. Experimental results show that our approach overperforms the related methods in the term of accuracy.

Original language	English (US)
Title of host publication	Graph Learning in Medical Imaging - 1st International Workshop, GLMI 2019, held in Conjunction with MICCAI 2019, Proceedings
Editors	Daoqiang Zhang, Luping Zhou, Biao Jie, Mingxia Liu
Publisher	Springer
Pages	130-137
Number of pages	8
ISBN (Print)	9783030358167
DOIs	https://doi.org/10.1007/978-3-030-35817-4_16
State	Published - 2019
Event	1st International Workshop on Graph Learning in Medical Imaging, GLMI 2019 held in conjunction with the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019 - Shenzhen, China Duration: Oct 17 2019 → Oct 17 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11849 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	1st International Workshop on Graph Learning in Medical Imaging, GLMI 2019 held in conjunction with the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019
Country/Territory	China
City	Shenzhen
Period	10/17/19 → 10/17/19

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-35817-4_16

Cite this

Lang, Y., Wang, L., Yap, P. T., Lian, C., Deng, H., Thung, K. H., Xiao, D., Yuan, P., Shen, S. G. F., Gateno, J., Kuang, T., Alfi, D. M., Xia, J. J., & Shen, D. (2019). Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN. In D. Zhang, L. Zhou, B. Jie, & M. Liu (Eds.), Graph Learning in Medical Imaging - 1st International Workshop, GLMI 2019, held in Conjunction with MICCAI 2019, Proceedings (pp. 130-137). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11849 LNCS). Springer. https://doi.org/10.1007/978-3-030-35817-4_16

Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN. / Lang, Yankun; Wang, Li; Yap, Pew Thian et al.
Graph Learning in Medical Imaging - 1st International Workshop, GLMI 2019, held in Conjunction with MICCAI 2019, Proceedings. ed. / Daoqiang Zhang; Luping Zhou; Biao Jie; Mingxia Liu. Springer, 2019. p. 130-137 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11849 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Lang, Y, Wang, L, Yap, PT, Lian, C, Deng, H, Thung, KH, Xiao, D, Yuan, P, Shen, SGF, Gateno, J, Kuang, T, Alfi, DM, Xia, JJ & Shen, D 2019, Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN. in D Zhang, L Zhou, B Jie & M Liu (eds), Graph Learning in Medical Imaging - 1st International Workshop, GLMI 2019, held in Conjunction with MICCAI 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11849 LNCS, Springer, pp. 130-137, 1st International Workshop on Graph Learning in Medical Imaging, GLMI 2019 held in conjunction with the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019, Shenzhen, China, 10/17/19. https://doi.org/10.1007/978-3-030-35817-4_16

Lang Y, Wang L, Yap PT, Lian C, Deng H, Thung KH et al. Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN. In Zhang D, Zhou L, Jie B, Liu M, editors, Graph Learning in Medical Imaging - 1st International Workshop, GLMI 2019, held in Conjunction with MICCAI 2019, Proceedings. Springer. 2019. p. 130-137. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-35817-4_16

Lang, Yankun ; Wang, Li ; Yap, Pew Thian et al. / Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN. Graph Learning in Medical Imaging - 1st International Workshop, GLMI 2019, held in Conjunction with MICCAI 2019, Proceedings. editor / Daoqiang Zhang ; Luping Zhou ; Biao Jie ; Mingxia Liu. Springer, 2019. pp. 130-137 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{bb6ca856a8dc444d9a84bfd76156d27d,

title = "Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN",

abstract = "Craniomaxillofacial (CMF) landmark localization is an important step for characterizing jaw deformities and designing surgical plans. However, due to the complexity of facial structure and the deformities of CMF patients, it is still difficult to accurately localize a large scale of landmarks simultaneously. In this work, we propose a three-stage coarse-to-fine deep learning method for digitizing 105 anatomical craniomaxillofacial landmarks on cone-beam computed tomography (CBCT) images. The first stage outputs a coarse location of each landmark from a low-resolution image, which is gradually refined in the next two stages using the corresponding higher resolution images. Our method is implemented using Mask R-CNN, by also incorporating a new loss function that learns the geometrical relationships between the landmarks in the form of a root/leaf structure. We evaluate our approach on 49 CBCT scans of patients and achieve an average detection error of 1.75 ± 0.91 mm. Experimental results show that our approach overperforms the related methods in the term of accuracy.",

author = "Yankun Lang and Li Wang and Yap, {Pew Thian} and Chunfeng Lian and Hannah Deng and Thung, {Kim Han} and Deqiang Xiao and Peng Yuan and Shen, {Steve G.F.} and Jaime Gateno and Tianshu Kuang and Alfi, {David M.} and Xia, {James J.} and Dinggang Shen",

note = "Funding Information: This work was supported in part by NIDCR grants (DE022676 and DE027251). Funding Information: This work was supported in part by NIDCR grants (DE022676 Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 1st International Workshop on Graph Learning in Medical Imaging, GLMI 2019 held in conjunction with the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019 ; Conference date: 17-10-2019 Through 17-10-2019",

year = "2019",

doi = "10.1007/978-3-030-35817-4_16",

language = "English (US)",

isbn = "9783030358167",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "130--137",

editor = "Daoqiang Zhang and Luping Zhou and Biao Jie and Mingxia Liu",

booktitle = "Graph Learning in Medical Imaging - 1st International Workshop, GLMI 2019, held in Conjunction with MICCAI 2019, Proceedings",

}

TY - GEN

T1 - Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN

AU - Lang, Yankun

AU - Wang, Li

AU - Yap, Pew Thian

AU - Lian, Chunfeng

AU - Deng, Hannah

AU - Thung, Kim Han

AU - Xiao, Deqiang

AU - Yuan, Peng

AU - Shen, Steve G.F.

AU - Gateno, Jaime

AU - Kuang, Tianshu

AU - Alfi, David M.

AU - Xia, James J.

AU - Shen, Dinggang

N1 - Funding Information: This work was supported in part by NIDCR grants (DE022676 and DE027251). Funding Information: This work was supported in part by NIDCR grants (DE022676 Publisher Copyright: © 2019, Springer Nature Switzerland AG.

PY - 2019

Y1 - 2019

N2 - Craniomaxillofacial (CMF) landmark localization is an important step for characterizing jaw deformities and designing surgical plans. However, due to the complexity of facial structure and the deformities of CMF patients, it is still difficult to accurately localize a large scale of landmarks simultaneously. In this work, we propose a three-stage coarse-to-fine deep learning method for digitizing 105 anatomical craniomaxillofacial landmarks on cone-beam computed tomography (CBCT) images. The first stage outputs a coarse location of each landmark from a low-resolution image, which is gradually refined in the next two stages using the corresponding higher resolution images. Our method is implemented using Mask R-CNN, by also incorporating a new loss function that learns the geometrical relationships between the landmarks in the form of a root/leaf structure. We evaluate our approach on 49 CBCT scans of patients and achieve an average detection error of 1.75 ± 0.91 mm. Experimental results show that our approach overperforms the related methods in the term of accuracy.

AB - Craniomaxillofacial (CMF) landmark localization is an important step for characterizing jaw deformities and designing surgical plans. However, due to the complexity of facial structure and the deformities of CMF patients, it is still difficult to accurately localize a large scale of landmarks simultaneously. In this work, we propose a three-stage coarse-to-fine deep learning method for digitizing 105 anatomical craniomaxillofacial landmarks on cone-beam computed tomography (CBCT) images. The first stage outputs a coarse location of each landmark from a low-resolution image, which is gradually refined in the next two stages using the corresponding higher resolution images. Our method is implemented using Mask R-CNN, by also incorporating a new loss function that learns the geometrical relationships between the landmarks in the form of a root/leaf structure. We evaluate our approach on 49 CBCT scans of patients and achieve an average detection error of 1.75 ± 0.91 mm. Experimental results show that our approach overperforms the related methods in the term of accuracy.

UR - http://www.scopus.com/inward/record.url?scp=85076281234&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85076281234&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-35817-4_16

DO - 10.1007/978-3-030-35817-4_16

M3 - Conference contribution

AN - SCOPUS:85076281234

SN - 9783030358167

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 130

EP - 137

BT - Graph Learning in Medical Imaging - 1st International Workshop, GLMI 2019, held in Conjunction with MICCAI 2019, Proceedings

A2 - Zhang, Daoqiang

A2 - Zhou, Luping

A2 - Jie, Biao

A2 - Liu, Mingxia

PB - Springer

T2 - 1st International Workshop on Graph Learning in Medical Imaging, GLMI 2019 held in conjunction with the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019

Y2 - 17 October 2019 through 17 October 2019

ER -

Automatic Detection of Craniomaxillofacial Anatomical Landmarks on CBCT Images Using 3D Mask R-CNN

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this