Crowdsourcing the creation of image segmentation algorithms for connectomics

To stimulate progress in automating the reconstruction of neural circuits,we organized the first international challenge on 2D segmentationof electron microscopic (EM) images of the brain. Participants submittedboundary maps predicted for a test set of images, and were scoredbased on their agreement with ground truth from human experts. Thewinning team had no prior experience with EM images, and employeda convolutional network. This ``deep learning'' approach has sincebecome accepted as a standard for segmentation of EM images. The challengehas continued to accept submissions, and the best so far has resultedfrom cooperation between two teams. The challenge has probably saturated,as algorithms cannot progress beyond limits set by ambiguities inherentin 2D scoring. Retrospective evaluation of the challenge scoring systemreveals that it was not sufficiently robust to variations in the widthsof neurite borders. We propose a solution to this problem, which shouldbe useful for a future 3D segmentation challenge.