Search

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. We observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings. This resource describes a collection of neurons from a variety of light microscopy-based datasets, which can serve as a gold standard for testing automated tracing algorithms, as shown by comparison of the performance of 35 algorithms.

Movie M2. Representative example of a CAR8 cell from the S2 subgroup.

Table ST1: Glossary of terms used in the paper Figure S1. Genetic modification and expansion of CAR+ T cells. Figure S2. In vitro expansion of CAR+ T cells. Figure S3. High-throughput cytotoxicity assay for monitoring T-cell target cell interactions in nanowell grids. Figure S4. CD19 expression on NALM-6 tumor cells or CD19+EL4 target cells as determined by immunofluoresecent staining. Figure S5. Composite micrographs illustrating representative examples of the interactions between single CAR+ T cells (E) and one or more NALM-6 tumor (T) cells. Figure S6. Increased probability of killing by individual CAR+ T cells at higher target cell densities. Figure S7. Donut plots summarizing the outcomes of the interaction between individual CAR8 cells and 1-3 CD19+-NALM-6 tumor cells. Figure S8. Timelapse Imaging Microscopy In Nanowell Grids (TIMING). Figure S9. At an E:T ratio of 1:1, identification of subgroups of killer CAR8 cells based on their motility and contact behavior with tumor cells. Figure S10. At an E:T of 1:1, the total duration of conjugation prior to NALM-6 tumor cell killing is no different for the CAR8 cells in the different subgroups. Figure S11. At an E:T ratio of 1:1, CAR8 cells in the S1 subgroup, demonstrate: (A) drop in motility, and (B) increased circularization upon conjugation to NALM-6 tumor cell. Figure S12. At an E:T ratio of 1:1, CAR8 cells in the different subgroups demonstrate different frequencies and kinetics of AICD subsequent to the interactions with NALM-6 cells. Figure S13. At an E:T ratio of 1:2-5, multi-killer CAR8 cells demonstrate no significant differences in their duration of conjugation prior to killing multiple NALM-6 tumor cells. Figure S14. Despite the increased crowding because of higher tumor cell density, multi-killer CAR8 cells displayed greater motility when conjugated to tumor cell in comparison to single-killer CAR8 cells that encountered only a single tumor cell. Figure S15. At an E:T ratio of 1:1, identification of subgroups of killer CAR4 cells based on their motility and contact behavior with tumor cells. Figure S16. At an E:T of 1:1, the total duration of conjugation prior to NALM-6 tumor cell killing is significantly longer for CAR4 cells in S2 subgroup in comparison to subgroups S1 and S3. Figure S17. At an E:T of 1:1, CAR4 cells in S2 subgroup induce apoptosis in tumor cells with delayed kinetics in comparison to CAR8 cells in the S2 subgroup. Figure S18. At an E:T ratio of 1:2-5, multi-killer CAR4 cells demonstrate increased circularization upon contact with one or more NALM-6 tumor cells. Figure S19. The ability of individual CAR4 cells to simultaneously conjugate to multiple NALM-6 tumor cells increases as the number of tumor cells within the nanowell increases. Figure S20. Comparison of the killing efficiency of individual single killer CAR+ T cells (E:T 1:1) with multi-killer CAR+ T cells (E:T 1:2-5) that killed multiple NALM-6 tumor cells.

T cells genetically modified to express a CD19-specific chimeric antigen receptor (CAR) for the investigational treatment of B-cell malignancies comprise a heterogeneous population, and their ability to persist and participate in serial killing of tumor cells is a predictor of therapeutic success. We implemented Timelapse Imaging Microscopy in Nanowell Grids (TIMING) to provide direct evidence that CD4+CAR+ T cells (CAR4 cells) can engage in multikilling via simultaneous conjugation to multiple tumor cells. Comparisons of the CAR4 cells and CD8+CAR+ T cells (CAR8 cells) demonstrate that, although CAR4 cells can participate in killing and multikilling, they do so at slower rates, likely due to the lower granzyme B content. Significantly, in both sets of T cells, a minor subpopulation of individual T cells identified by their high motility demonstrated efficient killing of single tumor cells. A comparison of the multikiller and single-killer CAR+ T cells revealed that the propensity and kinetics of T-cell apoptosis were modulated by the number of functional conjugations. T cells underwent rapid apoptosis, and at higher frequencies, when conjugated to single tumor cells in isolation, and this effect was more pronounced on CAR8 cells. Our results suggest that the ability of CAR+ T cells to participate in multikilling should be evaluated in the context of their ability to resist activation-induced cell death. We anticipate that TIMING may be used to rapidly determine the potency of T-cell populations and may facilitate the design and manufacture of next-generation CAR+ T cells with improved efficacy. Cancer Immunol Res; 3(5); 473–82. ©2015 AACR.See related commentary by June, p. 470

Movie M4. Representative example of a CAR8 cell that kills multiple tumor cells.

Movie M1. Representative example of a CAR8 cell from the S1 subgroup.

Movie M3. Representative example of a CAR8 cell from the S3 subgroup.

Movie M5. Representative example of a CAR4 cell that kills multiple tumor cells.

BigNeuron is an open community bench-testing platform combining the expertise of neuroscientists and computer scientists toward the goal of setting open standards for accurate and fast automatic neuron reconstruction. The project gathered a diverse set of image volumes across several species representative of the data obtained in most neuroscience laboratories interested in neuron reconstruction. Here we report generated gold standard manual annotations for a selected subset of the available imaging datasets and quantified reconstruction quality for 35 automatic reconstruction algorithms. Together with image quality features, the data were pooled in an interactive web application that allows users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and reconstruction data, and benchmarking of automatic reconstruction algorithms in user-defined data subsets. Our results show that image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. By benchmarking automatic reconstruction algorithms, we observed that diverse algorithms can provide complementary information toward obtaining accurate results and developed a novel algorithm to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms. Finally, to aid users in predicting the most accurate automatic reconstruction results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic reconstructions.