Neural Architecture Search (NAS) has been widely applied to automate medical image diagnostics. However, traditional NAS methods require significant computational resources and time for performance evaluation. To address this, we introduce the GrMoNAS framework, designed to balance diagnostic accuracy and efficiency using proxy datasets for granularity transformation and multi-objective optimization algorithms. The approach initiates with a coarse granularity phase, wherein diverse candidate neural architectures undergo evaluation utilizing a reduced proxy dataset. This initial phase facilitates the swift and effective identification of architectures exhibiting promise. Subsequently, in the fine granularity phase, a comprehensive validation and optimization process is undertaken for these identified architectures. Concurrently, employing multi-objective optimization and Pareto frontier sorting aims to enhance both accuracy and computational efficiency simultaneously. Importantly, the GrMoNAS framework is particularly suitable for hospitals with limited computational resources. We evaluated GrMoNAS in a range of medical scenarios, such as COVID-19, Skin cancer, Lung, Colon, and Acute Lymphoblastic Leukemia diseases, comparing it against traditional models like VGG16, VGG19, and recent NAS approaches including GA-CNN, EBNAS, NEXception, and CovNAS. The results show that GrMoNAS achieves comparable or superior diagnostic precision, significantly enhancing diagnostic efficiency. Moreover, GrMoNAS effectively avoids local optima, indicating its significant potential for precision medical diagnosis., Competing Interests: Declaration of competing interest No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication. I would like to declare on behalf of my co-authors that the work described was original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed. We declare that no conflicts of interest, financial or otherwise, exist with the work submitted for publication. We further declare that no writing assistance, financial or otherwise, was used in the preparation of this work. We understand that the corresponding author has full responsibility for the content and integrity of the work submitted for publication and that the journal’s policies on competing interests, conflicts of interest, and transparency have been followed. We confirm that the work submitted for publication has not been published previously in any language or medium and is not currently under consideration for publication elsewhere. We further confirm that all authors meet the criteria for authorship credit and have agreed to the submission of this work for publication. Finally, we understand that the corresponding author has the authority to bind all other authors to this declaration and that we will comply with any future requests from the journal or its publisher for additional information or clarification regarding this declaration. This work will impact the field of image processing, including portrait processing and medical image classification. For example, when making medical diagnoses, which are often classified as sick or not, this technology can help doctors to make a preliminary diagnosis quickly and with a high degree of accuracy, which will make the work of radiologists much more efficient. The contribution of the algorithmic framework is divided into two main areas. We introduce granularity to facilitate the training of the network architecture and aim at validation accuracy and running time, and introduce a multi-objective optimization algorithm to avoid situations where training costs are reduced at the expense of accuracy. I hope this paper is suitable for “Computers in Biology and Medicine”., (Copyright © 2024 Elsevier Ltd. All rights reserved.)