Validity and Reliability of an Evaluation Index System for Consultation Competency of General Practitioners Practicing in Primary Care Settings

Background Currently, there is a lack of clinical competence evaluation tools applicable of general practitioners (GPs) practicing in rural settings in China, resulting in the lack of researches on the clinical competence evaluation of GPs in rural. Objective To explore reliability and validity of an evaluation index system for consultation competency of rural GPs developed previously, and provide an evaluation tool with high reliability and validity for scientific and objective assessment of consultation competence of GPs in rural settings. Methods Based on the evaluation system for consultation competency of rural GPs, a corresponding questionnaire was designed, and points were assigned to each option on a 5-point Likert scale. From September to December 2022, GPs or assistant general practitioners (AGPs) who were working in rural township health centers in Guangxi Province were recruited as the research subjects by using the purposive sampling and stratified sampling methods, the questionnaire was distributed through a national web-based survey platform-"WJX" to them. Cronbach's α coefficient, split-half coefficient, critical ratio (CR) and correlation coefficients were calculated based on questionnaire data. Confirmative factor analysis was employed to fit questionnaire data and assumed model, and calculate three categories of indicators, including preliminary fit criteria (PFC), overall model fit (OMF), and fit of internal structural model (FISM), to verify the degree of fit and structural validity of the measured data. Results A total of 600 questionnaires were distributed and 366 were validly collected, with an effective recovery rate of 61.0%, 86.1% were registered as GPs and 13.9% as AGPs, who came from five cities, including Nanning, Guilin, Wuzhou, Baise, and Guigang, and 204 township health centers in Guangxi Province. Cronbach'α coefficient for the whole questionnaire or for every section was higher than 0.700, and Guttman Split-Half coefficient was 0.931. The initial first-order model met the PFC well; except for goodness of fit index (GFI), adjusted goodness of fit index (AGFI), and normed fit index (NFI), other indicators related to the OMF reached for the best levels in the modified first-order model. Apart from R2 for 9 observable variables less than 0.5, the FISM for the modified first-order model showed good effects. The initial second-order model indicated as similar effects on the PFC as the initial first-order model; the modified second-order model shared similar OMF with the modified first-order model; the modified second-order model was inferior to the modified first-order model in terms of the FISM. Eventually, questionnaire data fitted the modified first-order model better. Conclusion The evaluation index system for consultation competency of rural GPs shows high reliability and validity, which can be used for research and practical work on the evaluation of consultation competence evaluation of GPs in rural settings.