We probe the origins of nonaffine deformation in discrete element simulations of densely packed granular systems under biaxial compression, using a new local measure of nonaffine micropolar deformation. This measure represents the deviation of particle motion from that dictated by a measure of micropolar strain and curvature, devised on the scale of a particle and its first ring of neighbors. Highly correlated mesoscopic structures emerge and contribute significantly to nonaffine deformation, an aspect of material behavior that is largely neglected in existing constitutive models. Distinct regimes of deformation were observed: a globally affine regime in which particles move in uncorrelated Brownian motion with some 'rattlers' present followed by a globally nonaffine regime involving rattlers, microbands, vortices, confined buckling of force chains, and persistent shear band(s). Structural development leading to and during persistent shear banding is elucidated. The highest levels of nonaffine deformation were observed during drops in the macroscopic stress ratio: here, nonaffine deformation is essentially confined to the shear band, where confined buckling of force chains is the intrinsic mechanism. Degrees of correlation between the local measures of nonaffine strain and curvature confirm the key role of particle rotations in shear bands. Probability density functions of local nonaffine deformation exhibit power law behavior cut off by an exponential tail, consistent with experiments. Self-diffusion anisotropy was observed inside the shear band. The degree of diffusion is greater tangential rather than normal to the shear band; particles in buckling force chains exhibit the highest degree of diffusion along the band. Selfdiffusion of particles after the peak stress ratio was found to be superdiffusive during each drop in stress ratio, and diffusive during each rise in stress ratio. The implications of these results for constitutive modeling are discussed, with particular attention paid to the significance of confined buckling of force chains. CE Database subject headings: Granular media; Discrete elements; Measurement; Deformation.