Designing Custom Multi-Reader Studies is Streamlined with Robust Viewer Configuration Options

By February 1, 2022Feature Release

Multi-reader studies are a popular tool for radiology researchers as they look to label images to build and refine AI models, among other applications. But administering these unique studies is often complex, time-consuming, and data-intensive. One of the greatest challenges researchers face is designing reader workflows and providing an intuitive interface that helps them complete their assigned tasks. 

Many common tools used in multi-reader studies aren’t built to solve these problems—they’re built for a few rigid purposes, leaving the researcher to cobble together a workflow specific to their needs. While standard read protocols, such as the RECIST Protocol, are available for researchers to leverage, the ability to design their own custom guided workflows aimed at novel hypotheses is not. This means researchers invest significant upfront time creating data capture systems external to the viewer and training readers to ensure accurate and adequate data capture throughout the study.

Ideally, blind readers should be presented with a guided workflow inside the viewer that solicits the necessary information without extraneous tools, form questions, or distractions. Researchers should be able to design these custom workflows without coding knowledge and with the confidence they are staying in compliance.

Bringing Focus and Simplicity to Multi-Reader Studies 

Flywheel’s highly configurable interface for multi-reader studies is built to solve the challenges of these studies for both administrators and readers. 

On the setup side, the researcher has broad customization abilities, allowing them to create a data capture form built into the viewer that captures their readers’ interpretations with advanced logic for displaying sets of questions and requiring annotations based on form responses. For example, the logic can require an annotation with a predefined label as well as the annotation tool type to be used. In addition, the researcher is able to change nearly any aspect of the OHIF viewer to suit it to the readers’ tasks. Options include hiding unwanted tools, defining custom hanging protocols, and designating slice specific workflows to be completed. With these capabilities, researchers can create custom workflows to guide the reader easily through each step ensuring optimal data capture and reader experience.

Setting custom hanging protocol for a reader study

Researchers can set custom hanging protocols and define slice specific workflows. In this example, the hanging protocol is asymmetrical and mixed 2D and 2D MPR viewports to navigate in three planes. When the user is on a promoted slice, a border appears around the 2D viewport and the study summary form populates for that slice.

On the reader side, the streamlined interface solicits the necessary information and minimizes unnecessary tools or form fields. This purpose-built interface minimizes confusion and frustration, improving the user experience for readers and helping them complete their assigned tasks efficiently. Blinding is also maintained through a series of role-based permissions so that the reader does not have access to other reader data or additional information about the subject.

With these capabilities, Flywheel users at institutions of all sizes can more efficiently design and administer multi-reader studies. This opens new doors to collaboration outside of an organization’s walls, and can help speed discovery with a platform that is friendly to both researchers and readers. 

Custom logic for a multi-reader study determines the tool type and followup questions

Multi-reader study administrators can define advanced custom logic for requiring specific annotations and/or displaying additional fields based on form field responses. In this example, the user selects “Yes” for the presence of a retinal layer which switches the tool type to the open freehand ROI tool for them to complete the required annotation. A label of “IB-RPE” is automatically applied and the next confirmation question is displayed.