• Background is provided about the need for tools to support regional energy baselining and efficiency planning. • 296 homes in Pennsylvania are simulated in EnergyPlus using a batch simulation tool and energy audit records. • Simulation results are scaled and compared to building level electricity use data and utility sales data. • Analysis reveals discrepancies between simulated and actual energy use that varies by utility. • The role of building simulation tools for energy baselining and efficiency planning is discussed. Building simulation tools are seeing growing use in the residential sector due to a combination of increased interest in building energy performance, more user-friendly software packages, and decreasing computing power requirements. These tools are now being applied in batch simulations, where large numbers of homes are simulated at once to understand existing building stock energy consumption and potential energy efficiency improvements to make to those stocks. As these types of applications grow, attention is needed to ensure that simulation inputs and outputs are handled with an appropriate understanding of the uncertainties involved. In this paper, we use a batch simulation tool to translate energy audit records of 296 existing single-family homes in Pennsylvania into EnergyPlus building simulation models. We compare the results of these simulations to actual metered electricity use data from these homes, representative device-level data, and local utility data to better understand the abilities and limits of such modeling practices. Results show that while batch simulations are capable of producing accurate estimates of the energy consumption of a large number of homes, many more homes had their energy consumption overestimated than underestimated, resulting in a significant net overestimate for the sample as a whole. When results are scaled up to the utility level, simulation accuracy varies by utility with net overestimates ranging from around 15% to 25%. Comparisons of individual simulation results to metered consumption identify electric space heating systems as the source of much of this error. This highlights the importance of gathering accurate, detailed information on these systems to generate reliable simulation results. NREL has recently released a batch simulation tool called ResStock that takes a more statistically robust approach to simulating representative archetype homes as opposed to actual existing homes. The results of this study suggest that the type of probabilistic treatment of building characteristics underlying that tool will provide more meaningful results than trying to simulate actual buildings. With additional validation measures similar to those used in this study, ResStock can likely be used to reliably estimate baseline energy consumption and quantify efficiency savings opportunities and can serve as a valuable decision-making tool for state, local, and utility efficiency program designers. [ABSTRACT FROM AUTHOR]