Incomparable Subsequent Effects between Study and Control Fluids Might Cause Questionable Results in Randomized Controlled Trials

Dr. Wiedermann commented on our systematic review and meta-regression for the sources of heterogeneity in trials reporting HES 130/0.4 or 0.42 associated excess mortality in septic patients[1] that the pooled analysis of mortality, which showed neither benefit nor harm, might be influenced by trials of low-quality. Statistically, we agree with this conclusion if two of the recruited trials judged as the intermediate risk of bias are precluded. Sensitivity tests in a meta-analysis are important. But an issue need for further discussing is that whether the results from a sensitivity analysis should be considered as the confirmatory or just exploratory. The fundamental thought of meta-analysis is to synthesis all the evidence current available. The article search strategy and the study inclusion/exclusion criteria had both been prespecified without any knowledge of the pooled analysis of mortality. The overall results should be taken as primary analysis because all the studies were fulfilled the prespecified principle. Over-interpretation of any subgroup analysis may have raised potential risk even if only focus on the trials with low risk of bias, the reasons were followed below. (1) Quality of the reporting of a study does not equal to the quality of the study itself. A well conduct study may have a very low-quality on reporting.[2] In addition, the process for risk of bias assessment is a sort of subjective even two independent reviewers assessing separately. It is hard to demonstrate any solid result from sensitivity analysis unless the consensus on the risk of bias assessment focus on study conducting could be achieved. (2) The multiplicity issue is not very common mentioned in the meta-analysis. However, the false-positive rate does significantly increased accompany with the raised number of tests. Currently, the meta-analysis includes 11 trials in total.[1] If we assume 6 of them are a low risk of bias, but the question is they are not easy to identify. Theoretically, there will be 462 possible combinations (randomly choose 6 trials from 11) under this scenario. The corresponding type I error rate (at least to observe one false-positive subgroup result) would be almost 100% (=1-0.95^462). By this reason, it should use a more restricted significant level for each subgroup result. Further, the test for subgroup difference between trials with low versus high risk of bias is a kind of interaction test. It also has to face the multiplicity issue. More importantly, the uncertainty of chosen the threshold for the significant level for interaction test is still controversial. Thus, the significant results of subgroup should be interpreted with cautions.[3] There are many reasons for the heterogeneity of mortality as primary outcome under the same intervention in randomized controlled trials (RCTs).[4] Most importantly, bias associated with execution factors such as intervention or its subsequent effects may cause questionable results in RCTs.[5] The efficiency of fluid therapy as a subsequent effect was found incomparable between HES and control group in some of the recruited RCTs in this meta-regression. Interestedly, all of 3 out of 11 RCTs, which were calculated with more positive fluid balance in HES group (a colloid with more efficiency on plasma volume expansion than crystalloids in general) than in control group, similarly reported a high risk of HES on excess mortality in septic patients and with significance in Perner's trial.[6] Our analysis further suggested daily delta fluid balance being likely associated with mortality in septic patients receiving HES 130/130/0.4* (P = 0.079). Whereas, we failed to determine a dose-effect relationship of HES 130/0.4 or 0.42 with mortality (P = 0.298). These results indicated that fluid balance was an important factor confounding the results of these RCTs. This was the main finding of this meta-regression. In conclusion, information from bedside (such as incomparable subsequent effects in fluid therapy) should be considered with priority over statistics in interpreting the results of available RCTs in evaluating the impact of any clinical intervention on mortality in critically ill patients.