Objective: Characteristics of difficult-to-treat depression (DTD), including infrequent symptom remission and poor durability of benefit, compel reconsideration of the outcome metrics historically used to gauge the effectiveness of antidepressant interventions., Methods: Self-report and clinician assessments of depression symptom severity were obtained regularly over a 2-year period in a difficult-to-treat depression registry sample receiving treatment as usual (TAU), with or without vagus nerve stimulation (VNS). Alternative outcome metrics for characterizing symptom change were compared in effect size and discriminating power in distinguishing the vagus nerve stimulation + treatment as usual and treatment as usual treatment groups. We expected metrics based on remission status to produce weaker between-group separation than those based on the classifications of partial response or response and metrics that integrate information over time to produce greater separation than those based on single endpoint assessment., Results: Metrics based on remission status had smaller effect size and poorer discrimination in separating the treatment groups than metrics based on partial response or response classifications. Metrics that integrated information over the 2-year observation period had stronger performance characteristics than those based on symptom scores at single endpoint assessment. For both the clinician-rated and self-report depression ratings, the metrics with the strongest performance characteristics were the median percentage change in symptom scores over the observation period and the proportion of the observation period in partial response or better., Conclusion: In difficult-to-treat depression, integrative symptom severity-based and time-based measures are sensitive and informative outcomes for assessing between-group treatment effects, while metrics based on remission status are not., Competing Interests: Declaration of Conflicting InterestsThe author(s) declared the following potential conflicts of interest with respect to the research, authorship and/or publication of this article: Dr STA is a consultant to Genomind, LivaNova, Janssen, Neuronetics and Sage Therapeutics. He also receives research support from Compass Pathways and Neuronetics.Dr HAS serves as a scientific adviser and receives consulting fees from Cerebral Therapeutics Inc., Holmusk Technologies Inc., LivaNova PLC, MECTA Corporation, Neurolief Ltd, Neuronetics Inc., Parow Entheobiosciences LLC and SigmaStim LLC. He receives honoraria and royalties from Elsevier Inc. and Oxford University Press. He is the inventor on non-remunerative US patents for Focal Electrically-Administered Seizure Therapy (FEAST), titration in the current domain in ECT and the adjustment of current in ECT devices, each held by Balance Point LLC. He is also the originator of magnetic seizure therapy (MST).Dr MJ and Ms SB are employees of LivaNova USA PLC.Dr TG was an employee of LivaNova PLC when she helped draft the plan for the statistical analyses conducted in this study. She is currently an employee at Jazz Pharmaceuticals PLC.Dr MTB is a former employee and a current consultant of LivaNova USA PLC.Dr CRC has received research support from the American Foundation for Suicide Prevention, Assurex Health Inc., August Busch IV Foundation, Barnes-Jewish Hospital Foundation, Bristol-Myers Squibb, LivaNova PLC, National Institute of Mental Health, NeoSync, Stanley Medical Research Institute and the Taylor Family Institute for Innovative Psychiatric Research. He is a part-time employee at the John Cochran VA Medical Center in St. Louis.Dr KD has received honoraria for attending advisory boards, acting as a consultant or being a member of the speaker bureau for: Boehringer-Ingelheim, Gedeon-Richter, Johnson and Johnson, LivaNova, Lundbeck, Pfizer and Recordati.Dr AHY has received payment for lectures and advisory boards for the following companies: Allegan, AstraZeneca, Bionomics, COMPASS, Eli Lilly, Janssen, LivaNova, Lundbeck, Neurocentrx, Novartis, Sage, Servier, Sumitomo Dainippon Pharma and Sunovion, He is a consultant to Johnson & Johnson and LivaNova. He has received honoraria for attending advisory boards and presenting talks at meetings organized by LivaNova. He is a Principal Investigator on studies funded by COMPASS, LivaNova and Janssen, and Chief Investigator on a study funded by Novartis. He does not hold shares in pharmaceutical companies. Dr. McAllister-Williams has received fees from American Center for Psychiatry & Neurology United Arab Emirates, British Association for Psychopharmacology, European College of Neuropsychopharmacology, International Society for Affective Disorders, Janssen, LivaNova, Lundbeck, My Tomorrows, OCM Comunicaziona s.n.c., Pfizer, Qatar International Mental Health Conference, Sunovion, Syntropharma, UK Medical Research Council and Wiley; grant support from the National Institute for Health Research Efficacy and Mechanism Evaluation Panel and Health Technology Assessment Panel and non-financial support from COMPASS Pathways and Magstim.Dr AJR has received consulting fees from Compass Inc., Curbstone Consultant LLC, Emmes Corp., Evecxia Therapeutics, Inc., Holmusk Technologies, Inc., Johnson and Johnson (Janssen), LivaNova, MindStreet, Inc., Neurocrine Biosciences Inc., Otsuka-US; speaking fees from LivaNova, Johnson and Johnson (Janssen) and royalties from Wolters Kluwer Health, Guilford Press and the University of Texas Southwestern Medical Center, Dallas, TX (for the Inventory of Depressive Symptoms and its derivatives). He is also named co-inventor on two patents: U.S. Patent No. 7,795,033: Methods to Predict the Outcome of Treatment with Antidepressant Medication, Inventors: McMahon FJ, Laje G, Manji H, Rush AJ, Paddock S, Wilson AS; and U.S. Patent No. 7,906,283: Methods to Identify Patients at Risk of Developing Adverse Events During Treatment with Antidepressant Medication, Inventors: McMahon FJ, Laje G, Manji H, Rush AJ, Paddock S.