We propose a new strategy for estimating the number of cellular objects that should be manually segmented for evaluating the segmentation accuracy of an algorithm. The strategy uses geometric and edge quality measurements that are directly related to segmentation outcomes and do not require highly accurate segmentation. Sample sizes are determined from standard deviations of cell feature measurements calculated from the entire image set rather than a small sample of that set. We estimate the confidence level that a sample size represents the whole population as well as an error associated with a particular sample of cell images. The use of this strategy may reduce the effort and time required for generating a reference data set for evaluating segmentation algorithm performance with images of biological cells. We demonstrate the usefulness of this methodology on a large and diverse data set for which reference data is available. We show that other techniques give rise to inconsistent results because the standard deviation of the data for the whole population is unknown, while our technique involves calculations that give consistently accurate sample sizes.
Adele P Peskin, Joe Chalfoun, John T Elliott, Karen Kafadar Estimating the Number of Manually Segmented Cellular Objects Required to Evaluate the Accuracy of a Segmentation Algorithm http://www.nist.gov/manuscript-publication-search.cfm?pub_id=913694