Flywheel’s Neuroinformatics Platform: Translating scientific findings into clinical applications

 

Lerma-Usabiaga led a group from Stanford University and the University of California, San Francisco (UCSF) in developing a framework for translating magnetic resonance imaging (MRI) scientific findings into clinical practice.  Their system is based on the Flywheel neuroinformatics platform, including both the data and computational management tools. The methods framework explores replication, which is essential for valid science, and generalization, which is essential for clinical applications. 

The authors gathered nine data sets into the Flywheel neuroinformatics platform, grouping  them into three categories. Variations in data characteristics begin during acquisition, the authors note, due to calibration differences in MRI instruments between competing MRI vendors and other factors.

Neuroinformatics platform reproducibility

Lerma-Usabiaga, et al., (2019) use Flywheel to support their goal of computational reproducibility. They  note that computational reproducibility is supported by using open source containerized methods whose inputs, outputs, parameters and versioning (provenance) are  stored in the Flywheel neuroinformatics platform. Other scientists can reproduce the analysis by accessing the Flywheel system, if they are authorized by the Institutional Review Board (IRB).

The containers execute the largest and most complex jobs, while  Flywheel’s software development kit (SDK) facilitates data preparation, further statistical analysis, and visualization  (2019, p. 4). Script reproducibility is supported via storage and versioning in a GitHub repository (https://github.com/garikoitz/paper-reproducibility) while input data and the executed version is stored in the Flywheel neuroinformatics platform.

Neuroinformatics platform for scientific reproducibility

The specific application the authors explore is a potential  biomarker in the white matter that can be used to assess individual subjects, following the  “Precision Medicine” approach emerging in neuroimaging. The authors point to a need for quantitative and objective method frameworks addressing biomarker measurement validation which provides precision, as well as replication, and reproducibility. 

The paper addresses increasing variations in the clinical setting – neuroimaging instruments, populations, and measurement protocols – as well as the problem of algorithmic complexity. Lerma-Usabiaga, et al., (2019) point out that by using containers and storing the analysis history  the Flywheel neuroinformatics platform system implemented “closely align” (2019, p. 8) with the Poldrack et al., (2017) description of scientific reproducibility research tools

“…entire analysis workflow…completely automated in a workflow engine and packaged in a software container or virtual machine to ensure computational reproducibility.” (p. 124, 2017)

Finally, Lerma-Usabiaga, et al., 2019 note that the Flywheel platform is  extensibile, simplifying the analysis of new datasets using identical computational methods and measuring how compliance ranges fluctuate. A  process of continuous data aggregation should allow continuing improvement of the methods and better definition of the compliance range, helping translate scientific research from the lab to the clinic.

References

Lerma-Usabiaga, G., et al., 2019. Replication and generalization in applied neuroimaging. NeuroImage, a Journal of Brain Function. Volume 202, Article 116048, November 15, 2019.

Marcus, D., et al., 2011. Informatics and data mining tools and strategies for the human connectome project. Front. Neuroinf., 5 (2011), p. 4.

Poldrack, R. A., et al., 2017. Scanning the horizon: towards transparent and reproducible neuroimaging research. Nat. Rev. Neurosci. 18, 115-126.