Diagnosing Diversity Collapse and Validating Mask-Conditioned Diffusion for Labeled Microtubule Microscopy

Abstract

Mask-conditioned diffusion models have become standard for generating labeled training data in biomedical imaging, but evaluating them is harder than building them. Standard metrics like FID and KID score images through ImageNet-trained features that do not transfer to microscopy, leaving them poorly calibrated for specialized domains; worse, a model can produce visually plausible images that still transfer poorly to downstream tasks. We identify a specific failure mode: late-training checkpoints trade texture diversity for a fixed color palette (r = −0.71 between color and diversity), collapsing to a high-similarity, low-diversity state that standard metrics do not penalize. We propose a diversity-aware diagnostic that reads a realism axis jointly with a diversity axis (intra-set similarity). This diagnostic selects a checkpoint that domain experts cannot distinguish from real recordings (2AFC accuracy 50.9% over 110 trials, p = 0.92) and yields useful downstream segmentation: a segmenter trained only on synthetically labeled data reaches a median skeleton-IoU statistically indistinguishable from one trained on real data, at lower cross-fold variance. A parametric baseline does not close this synthetic-to-real gap. We release DiffuMT on HuggingFace: a triplet dataset of (binary mask, real IRM microtubule microscopy image, mask-conditioned synthetic counterpart), the diagnostic implementation, and code to reproduce the downstream-utility validation.