Surgical Planning

Automatic Multiparametric Magnetic Resonance Imaging-Based Prostate Lesions Assessment with Unsupervised Domain Adaptation

Multiparametric magnetic resonance imaging (mpMRI) has emerged as a valuable diagnostic tool in prostate lesion assessment. However, training convolutional neural networks (CNNs) inevitably involves magnetic resonance(MR) images from multiple cohorts. There always exists variation in scanning protocol among cohorts, inducing significant changes in data distribution between source and target domains. This challenge has greatly limited clinical adoption on a large scale. Here, a coarse mask-guided deep domain adaptation network (CMD²A-Net) is proposed to develop a fully automated framework for prostate lesion detection and classification (PLDC). No category or mask label is required from the target domain. A coarse segmentation module is trained to cover the possible lesion-related regions, so that attention maps can be generated to dedicate the local feature extraction of lesions within those regions. Experiments are performed on 512 mpMRI sets from datasets of PROSTATEx (330 sets) and two cohorts, A (74 sets) and B (108 sets). Using ensemble learning, CMD²A-Net accomplishes an AUC of 0.921 in cohort A and 0.913 in cohort B, demonstrating its transferability from a large-scale public dataset PROSTATEx to small-scale target domains. Results from an ablation study also support its effectiveness in classification between benign and malignant lesions, compared to the state-of-the-art models.

Depth Estimation for Real-time 3D Annotation in Transnasal Surgery

Surgical annotation promotes effective communication between medical personnel during surgical procedures. However, existing approaches to 2D annotations are mostly static with respect to a display. In this work, we propose a method to achieve 3D annotations that anchor rigidly and stably to target structures upon camera movement in a transnasal endoscopic surgery setting. This is accomplished through intra-operative endoscope tracking and monocular depth estimation. A virtual endoscopic environment is utilized to train a supervised depth estimation network. An adversarial network transfers the style from the real endoscopic view to a synthetic-like view for input into the depth estimation network, wherein frame-wise depth can be obtained in real time. Accuracy in terms of estimated depth and stability of the proposed 3D annotation system is evaluated qualitatively and quantitatively.

The training data and test data can be downloaded here.

Advanced pre-operative imaging modalities such as ultrasonography (US), computed tomography (CT) and magnetic resonance imaging (MRI) can provide anatomy and pathology information, which is indispensable and crucial for a detailed diagnosis and treatment plan. 3D anatomical models can be readily visualized for simulation of surgical intervention. Our group is interested in patient-specific surgical planning for computer-assisted surgery. Optimization of cardiac mapping strategy for radiofrequency catheter ablation is one of the examples. We also adopt timely 3D-printing technologies to fabricate patient-specific models using materials that mechanically mimic soft tissues and challenging anatomical features. Such tangible 3D-printed models not only can give better insight for surgical planning, but also allow simulation of actual procedure, as well as validation of virtually optimized surgery plan in pre-clinical settings, eventually enhancing the quality of surgical planning for safer and more efficacious interventions.

PUBLICATIONS

[1] J. Dai, X. Wang, Y. Li, Z. Liu, Y. L. Ng, J. Xiao, J.K.M. Fan, J. Lam, Q. Dou, V. Vardhanabhuti, K.W. Kwok, “Automatic mpMRI-based prostate lesions assessment with unsupervised domain adaptation,” Advanced Intelligent Systems (AISY) (Accepted)

[2] H.S. Tong, Y.L. Ng, Z. Liu, J.D.L. Ho, P.L. Chan, J.Y.K. Chan, K.W. Kwok, “Real-to-Virtual Domain Transfer-based Depth Estimation for Real-time 3D Annotation in Transnasal Surgery: A Study of Annotation Accuracy and Stability”, International Journal for Computer Assisted Radiology and Surgery (IJCARS), 16(5):731-739, 2021 Detail

[3] Y. Feng, Z. Guo, Z. Dong, X.Y. Zhou, K.W. Kwok, S. Ernst, S.L. Lee, "An Efficient Cardiac Mapping Strategy for Radiofrequency Catheter Ablation with Active Learning," International Journal of Computer Assisted Radiology and Surgery (IJCARS), vol. 12, no. 7, pp. 1199-1207, 2017. Detail

[4] K.C.Y. So, Y. Fan, L. Sze, K.W. Kwok, A.K.Y. Chan, G.S.H. Cheung, A.P.W. Lee, "Using Multi-material 3D Printing for Personalized Planning of Complex Structural Heart Disease Intervention," Journal of the American College of Cardiology (JACC): Cardiovascular Intervention, vol. 10, no. 11, 2017 Detail

[5] Y. Fan, K.W. Kwok, Y. Zhang, G. S.H. Cheung, A. K.-Y. Chan, and A.P.W. Lee, "Three-dimensional printing for planning occlusion procedure for a double-lobed left atrial appendage," Circulation: Cardiovascular Interventions, vol. 9, no. 3, p. e003561, 2016 Detail

[6] M.C.W. Leong, K.H. Lee, B.P.Y. Kwan, Y.L. Ng, Z. Liu, N. Navab, W. Luk, K.W. Kwok, “Performance-aware Programming for Intraoperative Intensity-based Image Registration on Graphics Processing Units”, International Journal for Computer Assisted Radiology and Surgery (IJCARS), vol. 16, no. 3, 375-386, 2021 Detail

HIGH PERFORMANCE IMAGE REGISTRATION TOOL

Intensity-based image registration is a powerful tool to resolve image misalignments. The Diffeomorphic Log-Demons is a well-established, Demons-based registration method. This open-sourced experimental tool is a CUDA-enabled implementation of the algorithm. We employ performance-aware programming techniques to ensure full device utilization. Preliminary results show this tool can speed up the computation by two orders of magnitudes.

The source files of this registration tool can be downloaded here.