Land use change and its impacts on eco-environment attract great attentions. Earlier studies shed light on the impacts of urban build-up land and agriculture land. Yet, knowledge about the impacts of rural residential land on eco-environment, especially from different expansion patterns perspective, is relatively limited. Taking Baota District, a typical city in loess hilly and gully region during the period of 1990–2015 as a case study, this paper analyzed rural residential land and habitat quality changes based on land-use data (30-m spatial resolution), and further used direct/indirect measurement model to explore the impacts of different expansion patterns of rural residential land on eco-environment. Results showed that the growth rate of rural residential land decreased first and then increased during study period, and the pattern of edge expansion has the largest scale among the newly added residential land. Moreover, the eco-environment in northern area of Baota District had been significantly improved, especially the habitat quality of shrubs and grasslands. Unfortunately, this study found that the eco-environment around cities and townships deteriorated dramatically. Environment policy, land engineering projects and residential land changes were important driving factors on eco-environment changes. Furthermore, this study verified the difference in indirect impacts of rural residential land expansion patterns on eco-environment (infilling pattern