Some examples of what VoxelPrompt can do!

Led by Andrew Hoopes (ahoopes.github.io), also with Victor Butoi, John Guttag and myself.

At @csail.mit.edu and the The MGH/HST Martinos Center for Biomedical Imaging

This flexible framework excels across many neuroimaging tasks, enabling a wide range of clinical and research goals.

For a first demonstration, we focus on the large scope of neuroimage analysis. VoxelPrompt can characterize and segment hundreds of anatomical and disease regions-of-interest, support various morphological measurements (e.g. tumor growth over visits), and produce language-based characteristics.

A core technical contribution is jointly learning the language-based agent with the adaptive vision network, which leads to a powerful mechanism for flexibly addressing complex analytical targets.

VoxelPrompt uses a language agent that interprets user prompts, predicts step-by-step instructions (as code), and employs a powerful vision network to analyze any number of patient-specific scans at once.

Most practitioners lack compute resources or machine-learning expertise to build new models for their specific needs. This creates significant barriers to translating AI advancements into scientific and clinical workflows.

Clinicians and scientists face a broad range of analytical imaging aims, yet most AI medical imaging tools used in practice are designed for narrow, predefined computational goals and data modalities.

For @technologyreview.com I spoke with more than 10+ federal workers who occupy, or until recently occupied, scientific and technical positions across various agencies. They explained to me how much behind-the-scenes work the US government does to keep America’s engine of innovation humming. 2/