The purpose of this paper is to provide a justification of punishment which can be endorsed by free will skeptics, and which can also be defended against the 'using persons as mere means' objection. Free will skeptics must reject retributivism, that is, the view that punishment is just because criminals deserve to suffer based on their actions. Retributivists often claim that theirs is the only justification on which punishment is constrained by desert, and suppose that non-retributive justifications must therefore endorse treating the people punished as mere means to social ends. Retributivists typically presuppose a monolithic conception of desert: they assume that action-based desert is the only kind of desert. But there are also personhood-based desert claims, that is, desert claims which depend not on facts about our actions, but instead on the more abstract fact that we are persons. Since personhood-based desert claims do not depend on facts about our actions, they do not depend on moral responsibility, so free will skeptics can appeal to them just as well as retributivists. What people deserve based on the mere fact of their personhood is to be treated as they would rationally consent to be treated if all they had in view was the mere fact of their personhood. We can work out the implications of this view for punishment by developing a 'hypothetical consent' justification in which we select principles of punishment in the Rawlsian original position, so long as we are careful not to smuggle in the retributivist assumption that it is under our control whether we end up as criminals or as law-abiding citizens once we raise the veil of ignorance. [ABSTRACT FROM AUTHOR]