Search

Objectives: Automated phenotyping algorithms can reduce development time and operator dependence compared to manually developed algorithms. One such approach, PheNorm, has performed well for identifying chronic health conditions, but its performance for acute conditions is largely unknown. Herein, we implement and evaluate PheNorm applied to symptomatic COVID-19 disease to investigate its potential feasibility for rapid phenotyping of acute health conditions., Materials and Methods: PheNorm is a general-purpose automated approach to creating computable phenotype algorithms based on natural language processing, machine learning, and (low cost) silver-standard training labels. We applied PheNorm to cohorts of potential COVID-19 patients from 2 institutions and used gold-standard manual chart review data to investigate the impact on performance of alternative feature engineering options and implementing externally trained models without local retraining., Results: Models at each institution achieved AUC, sensitivity, and positive predictive value of 0.853, 0.879, 0.851 and 0.804, 0.976, and 0.885, respectively, at quantiles of model-predicted risk that maximize F1. We report performance metrics for all combinations of silver labels, feature engineering options, and models trained internally versus externally., Discussion: Phenotyping algorithms developed using PheNorm performed well at both institutions. Performance varied with different silver-standard labels and feature engineering options. Models developed locally at one site also worked well when implemented externally at the other site., Conclusion: PheNorm models successfully identified an acute health condition, symptomatic COVID-19. The simplicity of the PheNorm approach allows it to be applied at multiple study sites with substantially reduced overhead compared to traditional approaches., (© The Author(s) 2023. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For permissions, please email: journals.permissions@oup.com.)

Purpose: In distributed research network (DRN) settings, multiple imputation cannot be directly implemented because pooling individual-level data are often not feasible. The performance of multiple imputation in combination with meta-analysis is not well understood within DRNs., Methods: To evaluate the performance of imputation for missing baseline covariate data in combination with meta-analysis for time-to-event analysis within DRNs, we compared two parametric algorithms including one approximated linear imputation model (Approx), and one nonlinear substantive model compatible imputation model (SMC), as well as two non-parametric machine learning algorithms including random forest (RF), and classification and regression trees (CART), through simulation studies motivated by a real-world data set., Results: Under the setting with small effect sizes (i.e., log-Hazard ratios [logHR]) and homogeneous missingness mechanisms across sites, all imputation methods produced unbiased and more efficient estimates while the complete-case analysis could be biased and inefficient; and under heterogeneous missingness mechanisms, estimates with RF method could have higher efficiency. Estimates from the distributed imputation combined by meta-analysis were similar to those from the imputation using pooled data. When logHRs were large, the SMC imputation algorithm generally performed better than others., Conclusions: These findings suggest the validity and feasibility of imputation within DRNs in the presence of missing covariate data in time-to-event analysis under various settings. The performance of the four imputation algorithms varies with the effect sizes and level of missingness., (© 2022 John Wiley & Sons Ltd.)

Purpose: To evaluate the effect of diagnostic coding system transition on the identification of common conditions recorded in Taiwan's national claims database., Methods: Using the National Health Insurance Research Database, we estimated the 3-month prevalence of recorded diagnosis of 32 conditions based on the ICD-9-CM codes in 2014-2015 and the ICD-10-CM codes in 2016-2017. Two algorithms were assessed for ICD-10-CM: validated ICD-10 codes in the literature and codes translated from ICD-9-CM using an established mapping algorithm. We used segmented regression analysis on time-series data to examine changes in the 3-month prevalence (both level and trend) before and after the ICD-10-CM implementation., Results: Significant changes in the level were found in 19 and 11 conditions when using the ICD-10 codes from the literature and mapping algorithm, respectively. The conditions with inconsistent levels by both of the algorithms were valvular heart disease, peripheral vascular disease, mild liver disease, moderate to severe liver disease, metastatic cancer, rheumatoid arthritis and collagen vascular diseases, coagulopathy, blood loss anemia, deficiency anemia, alcohol abuse, and psychosis. Nine conditions had significant changes in the trend when using the ICD-10 codes from the literature or mapping algorithm., Conclusions: Less than half of the 32 conditions studied had a smooth transition between the ICD-9-CM and ICD-10-CM coding systems. Researchers should pay attention to the conditions where the coding definitions result in inconsistent time series estimates., (© 2021 John Wiley & Sons Ltd.)

A fundamental question in using real-world data for clinical and regulatory decision making is: How certain must we be that the algorithm used to capture an exposure, outcome, cohort-defining characteristic, or confounder is what we intend it to be? We provide a practical framework to help researchers and regulators assess and classify the fit-for-purposefulness of real-world data by study variable for a range of data sources. The three levels of certainty (optimal, sufficient, and probable) must be considered in the context of each study variable, the specific question being studied, the study design, and the decision at hand., (© 2020 The Authors Clinical Pharmacology & Therapeutics © 2020 American Society for Clinical Pharmacology and Therapeutics.)

Introduction: In May 2008, the Food and Drug Administration launched the Sentinel Initiative, a multi-year program for the establishment of a national electronic monitoring system for medical product safety that led, in 2016, to the launch of the full Sentinel System. Under the Mini-Sentinel pilot, several algorithms for identifying health outcomes of interest, including one for anaphylaxis, were developed and evaluated using data available from the Sentinel common data model., Purpose: To evaluate whether features extracted from unstructured narrative data using natural language processing (NLP) could be used to classify anaphylaxis cases., Methods: Using previously developed methods, we extracted features from unstructured narrative data using NLP and applied rule-based and similarity-based algorithms to identify anaphylaxis among 62 potential cases previously classified by human experts as anaphylaxis (N = 33), not anaphylaxis (N = 27), and unknown (N = 2)., Results: The rule-based and similarity-based approaches demonstrated almost equal performance (recall 100% vs 100%, precision 60.3% vs 57.4%, F-measure: 0.753 vs 0.729). Reasons for misclassification included the inability of the algorithms to make the same clinical judgments as human experts about the timing, severity, or presence of alternative explanations; and the identification of terms consistent with anaphylaxis but present in conditions other than anaphylaxis., Conclusions: Although precision needs to be improved before these algorithms could be used without human review, we demonstrated that applying rule-based and similarity-based algorithms to unstructured narrative information from clinical records can be used for classification of anaphylaxis in the Sentinel System. Further development and assessment of these methods in the Sentinel System are warranted., (© 2018. This article is a U.S. Government work and is in the public domain in the USA.)

Background: The combined comorbidity score, which merges the Charlson and Elixhauser comorbidity indices, uses the ninth revision of the International Classification of Diseases, Clinical Modification (ICD-9-CM). In October 2015, the United States adopted the 10th revision (ICD-10-CM)., Objective: The objective of this study is to examine different coding algorithms for the ICD-10-CM combined comorbidity score and compare their performance to the original ICD-9-CM score., Methods: Four ICD-10-CM coding algorithms were defined: 2 using General Equivalence Mappings (GEMs), one based on ICD-10-CA (Canadian modification) codes for Charlson and Elixhauser measures, and one including codes from all 3 algorithms. We used claims data from the Clinfomatics Data Mart to identify 2 cohorts. The ICD-10-CM cohort comprised patients who had a hospitalization between January 1, 2016 and March 1, 2016. The ICD-9-CM cohort comprised patients who had a hospitalization between January 1, 2015 and March 1, 2015. We used logistic regression models to predict 30-day hospital readmission for the original score in the ICD-9-CM cohort and for each ICD-10-CM algorithm in the ICD-10-CM cohort., Results: Distributions of each version of the score were similar. The algorithm based on ICD-10-CA codes [c-statistic, 0.646; 95% confidence interval (CI), 0.640-0.653] had the most similar discrimination for readmission to the ICD-9-CM version (c, 0.646; 95% CI, 0.639-0.653), but combining all identified ICD-10-CM codes had the highest c-statistic (c, 0.651; 95% CI, 0.644-0.657)., Conclusions: We propose an ICD-10-CM version of the combined comorbidity score that includes codes identified by ICD-10-CA and GEMs. Compared with the original score, it has similar performance in predicting readmission in a population of United States commercially insured individuals.

Background: High-grade cervical dysplasia or cervical intraepithelial neoplasia grade 2 or worse has been widely used as a surrogate endpoint in cervical cancer screening or prevention trials., Methods: To identify high-grade cervical dysplasia and cervical cancer, we developed claims-based algorithms that incorporated a combination of diagnosis and procedure codes using the billing data in an electronic medical records database and assessed the validity of the algorithms in an independent administrative claims database. We calculated the positive predictive value (PPV) with the 95% confidence interval (CI) of each algorithm, using new cytologic or pathologic diagnosis of cervical intraepithelial neoplasia 2 or 3, carcinoma in situ, or cervical cancer as the gold standard., Results: Having ≥1 diagnosis code for high-grade cervical dysplasia or cervical cancer had a PPV of 57.1% (95%CI, 54.7-59.5%). By requiring ≥2 diagnoses for high-grade cervical dysplasia or cervical cancer, separated by 7-30 days, the PPV increased to 60.2% (95%CI, 53.9-66.1%). At least two diagnoses and a procedure code within a month from the first diagnosis date yielded a PPV of 80.7% (95%CI, 73.6-86.2%). The algorithms had greater PPVs in identifying prevalent high-grade cervical dysplasia or cervical cancer. Overall, the PPVs of these algorithms were similar or slightly lower in the external claims data than in the sample used to derive the algorithms., Conclusions: Use of ≥2 diagnosis codes in combination with a procedure code appears to be a valid tool for studying high-grade cervical dysplasia and cervical cancer in both electronic medical record and administrative claims databases., (Copyright © 2013 John Wiley & Sons, Ltd.)

Purpose: To validate an algorithm that uses delivery date and diagnosis codes to define gestational age at birth in electronic health plan databases., Methods: Using data from 225,384 live born deliveries to women aged 15-45 years in 2001-2007 within eight of the 11 health plans participating in the Medication Exposure in Pregnancy Risk Evaluation Program, we compared (1) the algorithm-derived gestational age versus the "gold-standard" gestational age obtained from the infant birth certificate file and (2) the prenatal exposure status of two antidepressants (fluoxetine and sertraline) and two antibiotics (amoxicillin and azithromycin) as determined by the algorithm-derived versus the gold-standard gestational age., Results: The mean algorithm-derived gestational age at birth was lower than the mean obtained from the birth certificate file among singleton deliveries (267.9 vs 273.5 days) but not among multiple-gestation deliveries (253.9 vs 252.6 days). The algorithm-derived prenatal exposure to the antidepressants had a sensitivity and a positive predictive value of ≥95%, and a specificity and a negative predictive value of almost 100%. Sensitivity and positive predictive value were both ≥90%, and specificity and negative predictive value were both >99% for the antibiotics., Conclusions: A gestational age algorithm based upon electronic health plan data correctly classified medication exposure status in most live born deliveries, but trimester-specific misclassification may be higher for drugs typically used for short durations., (Copyright © 2013 John Wiley & Sons, Ltd.)

Purpose: To validate an algorithm based upon International Classification of Diseases, 9(th) revision, Clinical Modification (ICD-9-CM) codes for acute myocardial infarction (AMI) documented within the Mini-Sentinel Distributed Database (MSDD)., Methods: Using an ICD-9-CM-based algorithm (hospitalized patients with 410.x0 or 410.x1 in primary position), we identified a random sample of potential cases of AMI in 2009 from four Data Partners participating in the Mini-Sentinel Program. Cardiologist reviewers used information abstracted from hospital records to assess the likelihood of an AMI diagnosis based on criteria from the Joint European Society of Cardiology and American College of Cardiology Global Task Force. Positive predictive values (PPVs) of the ICD-9-based algorithm were calculated., Results: Of the 153 potential cases of AMI identified, hospital records for 143 (93%) were retrieved and abstracted. Overall, the PPV was 86.0% (95% confidence interval; 79.2%, 91.2%). PPVs ranged from 76.3% to 94.3% across the four Data Partners., Conclusions: The overall PPV of potential AMI cases, as identified using an ICD-9-CM-based algorithm, may be acceptable for safety surveillance; however, PPVs do vary across Data Partners. This validation effort provides a contemporary estimate of the reliability of this algorithm for use in future surveillance efforts conducted using the Food and Drug Administration's MSDD., (Copyright © 2012 John Wiley & Sons, Ltd.)

Purpose: To describe the acute myocardial infarction (AMI) validation project, a test case for health outcome validation within the US Food and Drug Administration-funded Mini-Sentinel pilot program., Methods: The project consisted of four parts: (i) case identification-developing an algorithm based on the International Classification of Diseases, Ninth Revision, to identify hospitalized AMI patients within the Mini-Sentinel Distributed Database; (ii) chart retrieval-establishing procedures that ensured patient privacy (collection and transfer of minimum necessary amount of information, and redaction of direct identifiers to validate potential cases of AMI); (iii) abstraction and adjudication-trained nurse abstractors gathered key data using a standardized form with cardiologist adjudication; and (iv) calculation of the positive predictive value of the constructed algorithm., Results: Key decision points included (i) breadth of the AMI algorithm, (ii) centralized versus distributed abstraction, and (iii) approaches to maintaining patient privacy and to obtaining charts for public health purposes. We used an algorithm limited to International Classification of Diseases, Ninth Revision, codes 410.x0-410.x1. Centralized data abstraction was performed because of the modest number of charts requested (<155). The project's public health status accelerated chart retrieval in most instances., Conclusions: We have established a process to validate AMI within Mini-Sentinel, which may be used for other health outcomes. Challenges include the following: (i) ensuring that only minimum necessary data are transmitted by Data Partners for centralized chart review, (ii) establishing procedures to maintain data privacy while still allowing for timely access to medical charts, and (iii) securing access to charts for public health uses that do not require approval from an institutional review board while maintaining patient privacy., (Copyright © 2012 John Wiley & Sons, Ltd.)

Purpose: To evaluate the validity of health plan administrative and claims data to identify pre-gestational and gestational diabetes, obesity, and ultrasounds among pregnant women., Methods: A retrospective study was conducted using the administrative and claims data of three health plans participating in the HMO Research Network. Diagnoses, drug dispensings, and procedure codes were used to identify diabetes, obesity, and ultrasounds among women who were pregnant between January 2006 and December 2008. A random sample of medical charts (n = 222) were abstracted. Positive predictive values (PPVs) were calculated. Sensitivity also was calculated for obesity among women for whom body mass index data were available in electronic medical records at two sites., Results: Overall, 190 of 222 cases of diabetes (86%) were confirmed (82% for gestational diabetes and 74% for pre-gestational diabetes). The PPV for codes to identify ultrasounds was 80%. Whereas the PPV for obesity-related diagnosis codes was high (93%), and the sensitivity was low (33%)., Conclusions: Health plan administrative and claims data can be used to accurately identify pre-gestational and gestational diabetes and ultrasounds. Obesity is not consistently coded., (Copyright © 2011 John Wiley & Sons, Ltd.)

Purpose: A semi-automated high-dimensional propensity score (hd-PS) algorithm has been proposed to adjust for confounding in claims databases. The feasibility of using this algorithm in other types of healthcare databases is unknown., Methods: We estimated the comparative safety of traditional non-steroidal anti-inflammatory drugs (NSAIDs) and selective COX-2 inhibitors regarding the risk of upper gastrointestinal bleeding (UGIB) in The Health Improvement Network, an electronic medical record (EMR) database in the UK. We compared the adjusted effect estimates when the confounders were identified using expert knowledge or the semi-automated hd-PS algorithm., Results: Compared with the 411,616 traditional NSAID initiators, the crude odds ratio (OR) of UGIB was 1.50 (95%CI: 0.98, 2.28) for the 43,569 selective COX-2 inhibitor initiators. The OR dropped to 0.81 (0.52, 1.27) upon adjustment for known risk factors for UGIB that are typically available in both claims and EMR databases. The OR remained similar when further adjusting for covariates--smoking, alcohol consumption, and body mass index-that are not typically recorded in claims databases (OR 0.81; 0.51, 1.26) or adding 500 empirically identified covariates using the hd-PS algorithm (OR 0.78; 0.49, 1.22). Adjusting for age and sex plus 500 empirically identified covariates produced an OR of 0.87 (0.56, 1.34)., Conclusions: The hd-PS algorithm can be implemented in pharmacoepidemiologic studies that use primary care EMR databases such as The Health Improvement Network. For the NSAID-UGIB association for which major confounders are well known, further adjustment for covariates selected by the algorithm had little impact on the effect estimate., (Copyright © 2011 John Wiley & Sons, Ltd.)

To evaluate the accuracy of computerized algorithms for pinpointing periods of exposure to medications during pregnancy in the absence of data on timing of conception, the authors used data from a population-based sample of nonmalformed infants in the Slone Epidemiology Center Birth Defects Study in 1998-2006 (United States and Canada; N = 3,177). The standard was defined as any antiinfective use from 2 weeks after the last menstrual period through the third gestational month, which was compared with results obtained after defining the beginning of pregnancy as either 270 days before the birth date (delivery-date algorithm) or the date of the first prenatal visit (pregnancy-indicator algorithm). The sensitivity was 92% (95% confidence interval: 88, 95) for the delivery-date algorithm and 59% (95% confidence interval: 53, 65) for the pregnancy-indicator algorithm. The specificity was higher than 98% for both algorithms. The sensitivity for the delivery-date algorithm among women with preterm births was 66% (95% confidence interval: 49, 80). For women without pregnancy complications, subtraction of 270 days from the delivery date might be accurate for timing first-trimester prescription drug use in automated databases. However, the sensitivity of this algorithm is lower for preterm deliveries, suggesting limited validity to assess drug safety for pregnancy outcomes associated with prematurity.

Letrozole is an aromatase inhibitor that has an unapproved use for ovulation induction with infertility. Because of the proximity of this use to conception, we selected letrozole to study the effect of 3 different methods for identifying the pregnancy start date and their impact on exposure misclassification. Using electronic health data from the US Sentinel database (2001–2015), we identified live-birth pregnancies conceived through in-vitro fertilization or intrauterine insemination. The pregnancy start was calculated using 1) a validated algorithm to estimate the last menstrual period (LMP), 2) LMP + 14 days (i.e. conception estimate), and 3) the fertility-procedure date. We identified 47,628 live-births after intrauterine insemination (n = 24,962) and in-vitro fertilization (n = 22,666), in which 2,458 (5.3%) mothers received letrozole. The algorithm-based conception estimate occurred within 14 days of the fertility procedure for 78.3% of pregnancies. Defining pregnancy start as LMP (45.7/1,000 pregnancies) or LMP + 14 days (12.7/1,000 pregnancies) overestimated letrozole exposure during pregnancy by 8.4-fold and 2.3-fold, respectively, compared with defining it at the date of the fertility procedure (5.5/1,000 pregnancies). While most studies of drug utilization in pregnancy use LMP as the conventional pregnancy start, this introduced substantial exposure misclassification in the example of letrozole. LMP + 14 days was less biased. Researchers should carefully consider the impact of the method for identifying the pregnancy start date on the potential for exposure misclassification. [ABSTRACT FROM AUTHOR]

Objectives Mini-Sentinel is a pilot project sponsored by the U.S. Food and Drug Administration to create an active surveillance system to monitor the safety of FDA-regulated medical products. We assessed the capability of the Mini-Sentinel pilot to provide prevalence rates of medication use among pregnant women delivering a liveborn infant. Methods An algorithm was developed to identify pregnancies for a reusable analytic tool to be executed against the Mini-Sentinel Distributed Database. Diagnosis and procedure codes were used to identify women ages 10-54 years delivering a liveborn infant between April 2001 and December 2012. A comparison group of age- and date-matched nonpregnant women was identified. The analytic code was distributed to all 18 Mini-Sentinel data partners. The use of specific medications, selected because of concerns about their safe use during pregnancy, was identified from outpatient dispensing data. We determined the frequency of pregnancy episodes and nonpregnant episodes exposed to medications of interest, any time during the pregnant/matched nonpregnant period, and during each trimester. Results The analytic tool successfully identified 1,678,410 live birth deliveries meeting the eligibility criteria. The prevalence of use at any time during pregnancy was 0.38 % for angiotensin-converting enzyme inhibitors and 0.22 % for statins. For ≤0.05 % of pregnancy episodes, the woman was dispensed warfarin, methotrexate, ribavirin, or mycophenolate. Conclusions The analytic tool developed for this study can be used to assess the use of medications during pregnancy as safety issues arise, and is adaptable to include different medications, observation periods, pre-existing conditions, and enrollment criteria. [ABSTRACT FROM AUTHOR]

Background: Multi-center studies can generate robust and generalizable evidence, but privacy considerations and legal restrictions often make it challenging or impossible to pool individual-level data across data-contributing sites. With binary outcomes, privacy-protecting distributed algorithms to conduct logistic regression analyses have been developed. However, the risk ratio often provides a more transparent interpretation of the exposure-outcome association than the odds ratio. Modified Poisson regression has been proposed to directly estimate adjusted risk ratios and produce confidence intervals with the correct nominal coverage when individual-level data are available. There are currently no distributed regression algorithms to estimate adjusted risk ratios while avoiding pooling of individual-level data in multi-center studies.Methods: By leveraging the Newton-Raphson procedure, we adapted the modified Poisson regression method to estimate multivariable-adjusted risk ratios using only summary-level information in multi-center studies. We developed and tested the proposed method using both simulated and real-world data examples. We compared its results with the results from the corresponding pooled individual-level data analysis.Results: Our proposed method produced the same adjusted risk ratio estimates and standard errors as the corresponding pooled individual-level data analysis without pooling individual-level data across data-contributing sites.Conclusions: We developed and validated a distributed modified Poisson regression algorithm for valid and privacy-protecting estimation of adjusted risk ratios and confidence intervals in multi-center studies. This method allows computation of a more interpretable measure of association for binary outcomes, along with valid construction of confidence intervals, without sharing of individual-level data. [ABSTRACT FROM AUTHOR]