With the development of Materials Genome Initiative (MGI) and data mining technology, machine learning (ML) has emerged as an important tool in the research of materials science. For the heat resistant alloys used in furnace tubes, the rapid prediction of the high-temperature properties is critical but difficult until now. In this work, the ML method based on the deep learning algorithm is developed to establish the direct correlation between microstructure inputs and output stress rupture properties of Fe-Cr-Ni based heat resistant alloys. Two simple convolutional neural networks (CNN) and the complex network with VGG16 architecture are implemented and evaluated. The simple CNN and VGG16 models are trained from scratch and pre-trained, respectively. Due to the relatively few training samples in the dataset, the data augmentation configuration and the improved architecture are effective to mitigate overfitting in simple CNN models. The result also shows that in the case of transfer learning, the features extracted from other datasets can be used directly to this new visual task. It is demonstrated that both the simple CNN and VGG16 models reach the high prediction accuracies (more than 90 %) of high-temperature properties with a wide range of microstructures. In addition, the good prediction performance achieved in the small dataset also reveals the deep learning approaches can be used to construct powerful vision models in engineering practice, where very limited data is the common situation.