We review existing methods for quantifying fragmentation of accelerometry-measured physical activity and develop a unifying statistical framework that connects all methods by separately modeling switching between and duration of sedentary and active bouts. We analyzed the association of these metrics with mortality in National Health and Nutrition Examination Survey 2003–2006. We have identified that “Intradaily Variability” (1SD HR=1.32, 95%CI 1.18–1.48), “Active Bouts Fragmentation Index” (1SD HR=1.43, 95%CI 1.25–1.64), “Sedentary Bouts Gini Index” (1SD HR=1.15, 95%CI 1.06–1.25), and hazard rates of distribution function of durations of active and sedentary bouts (1SD HR=1.93, 1.40, 95% CI 1.59–2.34 and 1.23–1.59) independently predict mortality after adjusting for total sedentary time, total daily activity, age, sex, education, medical comorbidities, smoking status, alcohol consumption, and mobility limitations. Future studies should concentrate on developing metrics that jointly model the frequency of switching between the sedentary and active bouts as well as the bout durations.