Join the Machine Learning in Medical Imaging Consortium (MaLMIC) for a networking opportunity
Monthly Virtual Forum Series on Zoom
Friday, May 21, 2021
3:30 to 5:30 p.m. EDT
Benjamin Haibe-Kains, Chaya Moskowitz and Matt Hemsley talked about reproducibility and good practices in machine learning research. This was followed by discussion focused on lessons learned, opportunities to collaborate and sharing of resources.
Monthly Virtual Forum Series on Zoom
Friday, May 21, 2021
3:30 to 5:30 p.m. EDT
Benjamin Haibe-Kains, Chaya Moskowitz and Matt Hemsley talked about reproducibility and good practices in machine learning research. This was followed by discussion focused on lessons learned, opportunities to collaborate and sharing of resources.
Improving reproducibility in machine learning research
Benjamin Haibe-Kains, PhD
Associate Professor, University of Toronto
Talk summary: While biostatistics and machine learning are essential to analyze biomedical data, researchers are facing multiple challenges around research reproducibility and transparency. It is essential for independent researchers to be able to scrutinize and reproduce the results of a study using its materials, and build upon them in future studies.
Computational reproducibility is achievable when the data can easily be shared and the required computational resources are relatively common. However, the complexity of current algorithms and their implementation, the need for specific computer hardware and the use of sensitive biomedical data represent major obstacles in healthy-related research. In this talk, Benjamin described the various aspects of a typical biomedical study that are necessary for reproducibility and the platforms that exist for sharing these materials with the scientific community.
Design, conduct, and reporting of radiomic analyses: Let’s not reinvent the wheel
Chaya Moskowitz, PhD
Associate Attending Biostatistician, Memorial Sloan Kettering Cancer Center
Talk summary: The number of published papers reporting on radiomic analyses has been growing exponentially. Very few of these paper produce results that are translated into clinical practice at least in part because of methodological flaws in the work. Although complex methods for producing radiomic signatures are increasingly available and user-friendly and progress has been made in radiomic biomarker taxonomy and standardization, fundamental elements of study design, rigorous statistical analysis, and quality of reporting methods and results are frequently overlooked. In this talk, Chaya highlighted common pitfalls encountered in radiomic studies that could be avoided by knowledge of existing methods and adherence to existing reporting standards.
Deep learning with uncertainty quantification in MRI-guided radiation therapy
Matt Hemsley, BSc
PhD Student, Sunnybrook Research Institute
Talk summary: Deep learning methods are able to match or surpass the performance of classical methods as well as human experts in a growing number of medical imaging related tasks. However, most deep learning methods are unable to quantify uncertainty, hindering clinical translation. In this talk, Matt presented an overview of techniques used to model uncertainty on network output, demonstrated the connection between uncertainty quantification and reproducibility, and presented examples where uncertainty estimates were obtained from networks that performed clinical tasks related to real-time MRI-guided adaptive radiation therapy.