A Deep-Learning Method for the Prediction of Socio-Economic Indicators from Street-View Imagery Using a Case Study from Brazil

Socioeconomic indicators are essential to help design and monitor the impact of public policies on society. Such indicators are usually obtained through census data collected at 10-year intervals, which are not only temporally coarse but expensive. Over recent years other ways of collecting data and producing these indicators have been explored, in particular using the new surveillance capabilities that remote observations can provide. The objective of this paper is to evaluate the assessment of socioeconomic indicators using street-view imagery, through a case study conducted in a region of Brazil, the Vale do Ribeira, one of the poorest semi-rural regions in Brazil. In this study we used socioeconomic indicators collected by the Brazilian Institute of Geography and Statistics (IBGE) and used Google Street View (GSV) images as our source of remote observations. A pre-trained convolutional neural network (CNN) was used to predict socio-economic indicators from GSV. To evaluate the performance of the classifier, we performed five-fold cross-validation between the predicted indicator and its true value. The best performance was obtained for the highest income class, with 80% of correct prediction. We conclude that the method has the potential to predict socioeconomic indicators across a large area with social challenges such as Vale do Ribeira, and that the network model is general enough to be used even when the imagery dataset is from semi-rural areas. This demonstrates the applicability of GSV datasets for similar settings and perhaps ensuring their replicability, which is a scientific requirement that requires further experimentation/evaluation.