Search

Search advertising, a popular method for online marketing, has been employed to improve health by eliciting positive behavioral change. However, writing effective advertisements requires expertise and experimentation, which may not be available to health authorities wishing to elicit such changes, especially when dealing with public health crises such as epidemic outbreaks. Here, we develop a framework, comprising two neural network models, that automatically generates ads. The framework first employs a generator model, which creates ads from web pages. These ads are then processed by a translation model, which transcribes ads to improve performance. We trained the networks using 114K health-related ads shown on Microsoft Advertising. We measure ad performance using the click-through rates (CTR). Our experiments show that the generated advertisements received approximately the same CTR as human-authored ads. The marginal contribution of the generator model was, on average, 28% lower than that of human-authored ads, while the translator model received, on average, 32% more clicks than human-authored ads. Our analysis shows that, when compared to human-authored ads, both the translator model and the combined generator + translator framework produce ads reflecting higher values of psychological attributes associated with a user action, including higher valence and arousal, and more calls to action. In contrast, levels of these attributes in ads produced by the generator model alone are similar to those of human-authored ads. Our results demonstrate the ability to automatically generate useful advertisements for the health domain. We believe that our work offers health authorities an improved ability to build effective public health advertising campaigns.

Encouraging people to vaccinate is a challenging endeavor, but one which has tremendous public health benefits. Doing so requires overcoming barriers of awareness, availability, and (sometimes) vaccine hesitancy. Here we focus on nudging people to vaccinate through online advertising. We conducted a pre-registered online ads campaign encouraging people to vaccinate against three diseases: influenza, human papillomavirus, and herpes zoster. Ads were shown to ~69,000 people and were compared to similar ads shown to 8.6 million people. Outcome measures were clicks on ads and future searches for relevant terms. We find that ads have two main effects: First, a congruence effect whereby ads increase the likelihood of clicks and future searches by up to 116% in people who express an interest in the disease or the vaccine. Most commercial vaccine advertising is aimed entirely at this population. Second, we observed a priming effect, where ads shown to people who were searching for terms unrelated to the vaccine could be encouraged to click on them (odds ratios of 7.5–33.0) and, more often, search for the vaccine later (hazard ratios of 6.9–157.3). We provide analysis for optimizing vaccine advertising campaign budgets to balance the two populations. These findings demonstrate that digital advertising campaigns should consider not just advertising to direct keywords or to individuals that look exactly like existing customers, but consider tangential keywords that draw a wider target population who are likely earlier in their conversion funnel, thus increasing the number of people who vaccinate and maximizing vaccines uptake.

Objective Eating disorders (EDs) compromise the health and functioning of affected individuals, but it can often take them several years to acknowledge their illness and seek treatment. Early identification of individuals with EDs is a public health priority, and innovative approaches are needed for such identification and ultimate linkage with evidence-based interventions. This study examined whether Internet activity data can predict ED risk/diagnostic status, potentially informing timely interventions. Method Participants were 936 women who completed a clinically validated online survey for EDs, and 231 of them (24.7%) contributed their Internet browsing history. A machine learning algorithm used key attributes from participants' Internet activity histories to predict their ED status: clinical/subclinical ED, high risk for an ED, or no ED. Results The algorithm reached an accuracy of 52.6% in predicting ED risk/diagnostic status, compared to random decision accuracy of 38.1%, a relative improvement of 38%. The most predictive Internet search history variables were the following: use of keywords related to ED symptoms and websites promoting ED content, participant age, median browsing events per day, and fraction of daily activity at noon. Discussion ED risk or clinical status can be predicted via machine learning with moderate accuracy using Internet activity variables. This model, if replicated in larger samples where it demonstrates stronger predictive value, could identify populations where further assessment is merited. Future iterations could also inform tailored digital interventions, timed to be provided when target online behaviors occur, thereby potentially improving the well-being of many individuals who may otherwise remain undetected.

Background : The expanding amount of information available from our use of technologies has led researchers to explore how this information can aid in the detection of mental health issues. We expand on past work in this area by exploring how browsing histories might be able to predict perceived barriers to psychological treatment. Methods : We obtained 10 days of browsing history data for 255 respondents as well as assessments of Perceived Barriers to Psychological Treatments and depression, the Patient Health Questionnaire. Results : We found that browsing histories enabled high performance classification of people with high levels of perceived barriers to psychological treatments (AUC average of 0.86). Limitations: Our high classification accuracy does not help understand why different features within the browsing histories are useful to classify people according to browsing history. We also look at people who decided to contribute their browsing history but the use of this data more generally presents additional ethical questions. Conclusions : Browsing histories might be useful to classify people's barriers to seeking psychological treatment. It is clinically relevant to find those who perceive barriers to seeking treatment to better design ways to address those concerns and help them find treatment.

Studies have shown that the traces people leave when browsing the internet may indicate the onset of diseases such as cancer. Here we show that the adaptive engines of advertising systems in conjunction with clinically verified questionnaires can be used to identify people who are suspected of having one of three types of solid tumor cancers. In the first study, 308 people were recruited through ads shown on the Bing search engine to complete a clinically verified risk questionnaire. A classifier trained to predict questionnaire response using only past queries on Bing reached an Area Under the Curve of 0.64 for all three cancer types, verifying that past searches could be used to identify people with suspected cancer. The second study was conducted using the Google ads system in the same configuration as in the first study. However, in this study the ads system was set to automatically learn to identify people with suspected cancer. A total of 70,586 people were shown the ads, and 6,484 clicked and were referred to complete the clinical questionnaires. People from countries with higher Internet access and lower life expectancy tended to click more on the ads. Over time the advertisement system learned to identify people who were likely to have symptoms consistent with suspected cancer, such that the percentage of people completing the questionnaires and found to have suspected cancer reached approximately 11\% at the end of the experiment. These results demonstrate the utility of using search engine queries to screen for possible cancer and the application of modern advertising systems to help identify people who are likely suffering from serious medical conditions. This is especially true in countries where medical services are less developed.

IntroductionAchieving optimal diabetes control requires several daily self-management behaviours, especially adherence to medication. Evidence supports the use of text messages to support adherence, but there remains much opportunity to improve their effectiveness. One key limitation is that message content has been generic. By contrast, reinforcement learning is a machine learning method that can be used to identify individuals’ patterns of responsiveness by observing their response to cues and then optimising them accordingly. Despite its demonstrated benefits outside of healthcare, its application to tailoring communication for patients has received limited attention. The objective of this trial is to test the impact of a reinforcement learning-based text messaging programme on adherence to medication for patients with type 2 diabetes.Methods and analysisIn the REinforcement learning to Improve Non-adherence For diabetes treatments by Optimising Response and Customising Engagement (REINFORCE) trial, we are randomising 60 patients with suboptimal diabetes control treated with oral diabetes medications to receive a reinforcement learning intervention or control. Subjects in both arms will receive electronic pill bottles to use, and those in the intervention arm will receive up to daily text messages. The messages will be individually adapted using a reinforcement learning prediction algorithm based on daily adherence measurements from the pill bottles. The trial’s primary outcome is average adherence to medication over the 6-month follow-up period. Secondary outcomes include diabetes control, measured by glycated haemoglobin A1c, and self-reported adherence. In sum, the REINFORCE trial will evaluate the effect of personalising the framing of text messages for patients to support medication adherence and provide insight into how this could be adapted at scale to improve other self-management interventions.Ethics and disseminationThis study was approved by the Mass General Brigham Institutional Review Board (IRB) (USA). Findings will be disseminated through peer-reviewed journals, clinicaltrials.gov reporting and conferences.Trial registration numberClinicaltrials.gov (NCT04473326).

Triggered by the COVID-19 crisis, Israel’s Ministry of Health (MoH) held a virtual datathon based on deidentified governmental data. Organized by a multidisciplinary committee, Israel’s research community was invited to offer insights to help solve COVID-19 policy challenges. The Datathon was designed to develop operationalizable data-driven models to address COVID-19 health policy challenges. Specific relevant challenges were defined and diverse, reliable, up-to-date, deidentified governmental datasets were extracted and tested. Secure remote-access research environments were established. Registration was open to all citizens. Around a third of the applicants were accepted, and they were teamed to balance areas of expertise and represent all sectors of the community. Anonymous surveys for participants and mentors were distributed to assess usefulness and points for improvement and retention for future datathons. The Datathon included 18 multidisciplinary teams, mentored by 20 data scientists, 6 epidemiologists, 5 presentation mentors, and 12 judges. The insights developed by the three winning teams are currently considered by the MoH as potential data science methods relevant for national policies. Based on participants’ feedback, the process for future data-driven regulatory responses for health crises was improved. Participants expressed increased trust in the MoH and readiness to work with the government on these or future projects.

The US Centers for Disease Control and Prevention alerted of a suspected outbreak of lung illness associated with using E-cigarette products in September 2019. At the time that the CDC published its alert little was known about the causes of the outbreak or who was at risk for it. Here we provide insights into the outbreak through analysis of passive reporting and participatory surveillance. We collected data about vaping habits and associated adverse reactions from four data sources pertaining to people in the USA: A participatory surveillance platform (YouVape), Reddit, Google Trends, and Bing. Data were analyzed to identify vaping behaviors and reported adverse events. These were correlated among sources and with prior reports. Data was obtained from 720 YouVape users, 4331 Reddit users, and over 1 million Bing users. Large geographic variation was observed across vaping products. Significant correlation was found among the data sources in reported adverse reactions. Models of participatory surveillance data found specific product and adverse reaction associations. Specifically, cannabidiol was found to be associated with fever, while tetrahydrocannabinol was found to be correlated with diarrhea. Our results demonstrate that utilization of different, complementary, online data sources provide a holistic view of vaping associated lung injury while augmenting traditional data sources.

Diabetes is often diagnosed late. This study aimed to assess the possibility for earlier detection of diabetes from search data, using predictive models trained on large-scale data. We extracted all English-language queries made by people in the USA to Bing during 1 year and identified queries containing symptoms of diabetes. We compared the ability of four different prediction models (linear regression, logistic regression, decision tree and random forest) to distinguish between users who stated that they were diagnosed with diabetes and users who did not refer to diabetes or diabetes drugs but queried about at least one of the symptoms. We identified 11,050 “new diabetes users” who stated they had been diagnosed with diabetes and approximately 11.5 million “control users” who queried about symptoms without querying for terms related to diabetes. Both the logistic regression and the random forest models were able to distinguish between the populations with an area under curve of 0.92 which translates to a positive predictive value of 56% at a false-positive rate of 1%. The model could identify patients up to 240 days before they mentioned being diagnosed. Some undiagnosed diabetes patients can be detected accurately according to their symptom queries to a search engine. Such earlier diagnosis, especially in cases of type 1 diabetes, could be clinically meaningful. The ability of search engines to serve as a population-wide screening tool could potentially be improved using additional data provided by users.

Background Men who have sex with men (MSM) increasingly use internet-based websites and geospatial apps to seek sex. Though these platforms may be useful for public health intervention, evaluations of such interventions are rare. We sought to evaluate the online behavior of young MSM of color in Philadelphia and the effectiveness of using ads to link them to DoYouPhilly.org, where users can order free condoms, lubricant, and sexually transmitted infection test kits delivered via the U.S. postal service. Method Data collection and analyses were conducted in two phases. First, we performed keyword research and analyzed web browser logs using a proprietary data set owned by Microsoft. Subsequently, we ran a Google Ads campaign using the keywords identified in the preliminary phase, and directed targeted users to the DoYouPhilly.org condom or test kit ordering pages. Results were analyzed using MATLAB 2018. Results Test kit advertisements received 5,628 impressions, 157 clicks, and 18 unique conversions. The condom advertisements received 128,007 impressions, 2,583 clicks, and 303 unique conversions. Correlation between the click-through rate and the conversion rate per keyword was ρ = −.35 ( P = .0096) and per advertisement was ρ = .40 ( P = .14). Keywords that directly related to condoms were most effective for condom ordering (42% conversion rate vs. ≤2% for other classes), while keywords emphasizing the adverse effects of unprotected sex were most effective in test kit ordering (91% conversion rate vs. 13% and 12% for other classes). Conclusions Online advertisements seemed to affect real-world sexual health behavior, as measured by orders of condoms and test kits, among a group of young MSM living in the same community.

Background Clinical reports from patients suffering from the novel coronavirus (COVID-19) reflect a high prevalence of sensory deprivation or loss pertaining to smell (dysosmia/anosmia) and/or taste (dysgeusia/ageusia). Given the importance of the senses to daily functioning and personal experience, the mental health consequences of these symptoms warrant further attention. Methods A cohort of Reddit users posting within the /r/covid19positive subforum (N = 15,821) was leveraged to analyze instantaneous risk of transition to a state of suicidal ideation or depression using Cox proportional-hazards models. Risk transition was defined by posts made in suicide- or depression-related forums, or mentions of relevant phrases with and without mention of anosmia/ageusia in /r/covid19positive. Self-diagnosis of COVID-19 was also modeled as a separate and simultaneous predictor of mental health risk. Results Mention of anosmia/ageusia was significantly associated with transition to a risk state. Users with a history of anosmia/ageusia-related posts and who self-identified as COVID-19 positive had 30% higher instantaneous risk relative to others. The highest increase in instantaneous risk of suicidal ideation or depression occurred more than 100 days after first posting in /r/covid19positive. Limitations Use of self-diagnosed disease as well as a broad array of anosmia/ageusia-related terminology may entail both information bias and overestimates of symptom incidence. Conclusions The specific effects of COVID-19 on the senses may have long-term implications for patient mental health well-being beyond the primary recovery period. Future work is needed to investigate the longitudinal mental health burden of residual COVID-19 symptom presentation.

This is the peer reviewed version of the following article: Leon, Yom-Tov, Johnson, Petticrew, Williamson, Lampos, Cox. What on-line searches tell us about public interest and potential impact on behaviour in response to minimum unit pricing of alcohol in Scotland. Addiction. 2021;116(8):2008-2015, which has been published in final form at https://doi.org/10.1111/add.15388. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions. This article may not be enhanced, enriched or otherwise transformed into a derivative work, without express permission from Wiley or by statutory rights under applicable legislation. Copyright notices must not be removed, obscured or modified. The article must be linked to Wiley’s version of record on Wiley Online Library and any embedding, framing or otherwise making available the article or pages thereof by third parties from platforms, services and websites other than Wiley Online Library must be prohibited. Aims - To investigate whether the introduction of minimum unit pricing (MUP) in Scotland on 1 May 2018 was reflected in changes in the likelihood of alcohol-related queries submitted to an internet search engine, and in particular whether there was any evidence of increased interest in purchasing of alcohol from outside Scotland. Design - Observational study in which individual queries to the internet Bing search engine for 2018 in Scotland and England were captured and analysed. Fluctuations over time in the likelihood of specific topic searches were examined. The patterns seen in Scotland were contrasted with those in England. Setting - Scotland and England. Participants - People who used the Bing search engine during 2018. Measurements - Numbers of daily queries submitted to Bing in 2018 on eight alcohol-related topics expressed as a proportion of queries on that day on any topic. These daily likelihoods were smoothed using a 14-day moving average for Scotland and England separately. Findings - There were substantial peaks in queries about MUP itself, cheap sources of alcohol and online alcohol outlets at the time of introduction of MUP in May 2018 in Scotland, but not England. These were relatively short-lived. Queries related to intoxication and alcohol problems did not show a MUP peak, but were appreciably higher in Scotland than in England throughout 2018. Conclusions - Analysis of internet search engine queries appears to show that a fraction of people in Scotland may have considered circumventing minimum unit pricing in 2018 by looking for on-line alcohol retailers. The overall higher levels of queries related to alcohol problems in Scotland compared with England mirrors the corresponding differences in alcohol consumption and harms between the countries.

Introduction Most people with psychiatric illnesses do not receive treatment for almost a decade after disorder onset. Online mental health screens reflect one mechanism designed to shorten this lag in help-seeking, yet there has been limited research on the effectiveness of screening tools in naturalistic settings. Material and methods We examined a cohort of persons directed to a mental health screening tool via the Bing search engine (n = 126,060). We evaluated the impact of tool content on later searches for mental health self-references, self-diagnosis, care seeking, psychoactive medications, suicidal ideation, and suicidal intent. Website characteristics were evaluated by pairs of independent raters to ascertain screen type and content. These included the presence/absence of a suggestive diagnosis, a message on interpretability, as well as referrals to digital treatments, in-person treatments, and crisis services. Results Using machine learning models, the results suggested that screen content predicted later searches with mental health self-references (AUC = 0·73), mental health self-diagnosis (AUC = 0·69), mental health care seeking (AUC = 0·61), psychoactive medications (AUC = 0·55), suicidal ideation (AUC = 0·58), and suicidal intent (AUC = 0·60). Cox-proportional hazards models suggested individuals utilizing tools with in-person care referral were significantly more likely to subsequently search for methods to actively end their life (HR = 1·727, p = 0·007). Discussion Online screens may influence help-seeking behavior, suicidal ideation, and suicidal intent. Websites with referrals to in-person treatments could put persons at greater risk of active suicidal intent. Further evaluation using large-scale randomized controlled trials is needed.

Cerebrovascular disease is a leading cause of mortality and disability and an immense global burden. It is partly related to aging in a metabolic syndrome-promoting environment. Prevention strategies are insufficient: they rely on intermittent screening in predominantly high-risk individuals, while most cases occur in the intermediate risk population. Screening algorithms like the Framingham Risk Score predict events in the next decade using traditional risk factors (age, diabetes, hypertension, obesity, hyperlipidemia, atrial fibrillation and smoking). No current tool can predict a near/impending stroke in the next few weeks/months. This could provide a time window for urgent preventive measures (anticoagulation, hypertension control or lipid lowering). Algorithms analyzing daily computer usage, examining motor, language and executive functions, correlate with standard cognitive tests. Covert cerebrovascular disease is linked to subtle cognitive and motor deficits and increased risk for stroke. We examined whether this could be harnessed to predict an imminent stroke. We developed an algorithm based on internet search queries to identify people at increased risk for a near stroke event. The algorithm was entirely blind to traditional cardiovascular risk factors or risks for cognitive decline. We analyzed queries submitted to the Bing search engine by 285 people for which a stroke event was identified and 1195 controls, with regards to attributes previously shown to reflect cognitive function. Controls included random people 60 years and above or similar aged individuals who queried for one of eight control conditions: myocardial infarction, atrial fibrillation, hypertension, migraine, B12 deficiency, depression, hypothyroidism and surgery. We used a random forest model with 1000 trees to distinguish the patient cohort from each of the control cohorts. Ten-fold cross-validation at the user level was used to reduce the likelihood of overfitting. All processing was conducted using Matlab 2019. Results: Our model performed well against all controls with area under the curve of the receiver operating curve (AUC) of 0.985 or higher and a true positive rate (at 1% false positive rate) above 80% for separating patients from each of the controls. The predictive power rose as the stroke date approached and if data was acquired beginning 120 days prior to the event. Good prediction accuracy was also obtained for a prospective cohort of users collected one year later. We propose that impending stroke can be identified through alterations in internet usage patterns that reflect cognitive function, with high predictive power (AUC exceeding 0.985 for a near event in comparison to an AUC of ~0.71 in the Framingham Stroke Risk Score for a 10 year event). This could potentially be applied inexpensively, continuously and on a large scale to willing individuals with the aim of reducing stroke events.

IntroductionDepression and diabetes are highly disabling diseases with a high prevalence and high rate of comorbidity, particularly in low-income ethnic minority patients. Though comorbidity increases the risk of adverse outcomes and mortality, most clinical interventions target these diseases separately. Increasing physical activity might be effective to simultaneously lower depressive symptoms and improve glycemic control. Self-management apps are a cost-effective, scalable and easy access treatment to increase physical activity. However, cutting-edge technological applications often do not reach vulnerable populations and are not tailored to an individual’s behavior and characteristics. Tailoring of interventions using machine learning methods likely increases the effectiveness of the intervention.Methods and analysisIn a three-arm randomized controlled trial we will examine the effect of a text-messaging smartphone application to encourage physical activity in low-income ethnic minority patients with comorbid diabetes and depression. The adaptive intervention group receives messages chosen from different messaging banks by a reinforcement learning algorithm. The uniform random intervention group receives the same messages, but chosen from the messaging banks with equal probabilities. The control group receives a weekly mood message. We aim to recruit 276 adults from primary care clinics aged 18 to 75 years who have been diagnosed with current diabetes and show elevated depressive symptoms (PHQ-8 >5). We will compare passively collected daily step counts, self-report PHQ-8 and most recent HbA1c from medical records at baseline and at intervention completion at 6-month follow-up.Ethics and disseminationThe Institutional Review Board at the University of California San Francisco approved this study (IRB: 17-22608). We plan to submit manuscripts describing our User Designed Methods and testing of the adaptive learning algorithm and will submit the results of the trial for publication in peer-reviewed journals and presentations at (inter)-national scientific meetings.Registrationclinicaltrials.gov: NCT03490253; pre-resultsArticle SummaryStrengths and LimitationsNovel approach of targeting diabetes and depressive symptoms using a smartphone applicationAbility to compare adaptive messaging for increasing physical activity to messages delivered with equal probabilitiesTesting of a smartphone application integrated within primary care settings in a low-income vulnerable patient populationLongitudinal design with 6-month follow-up enables assessing intervention effects over timeChallenges of this trial include supporting users in key behavior change in an automated manner with minimal in-person support

Recent research has shown the efficacy of screening for serious medical conditions from data collected while people interact with online services. In particular, queries to search engines and the interactions with them were shown to be advantageous for screening a range of conditions including diabetes, several forms of cancer, eating disorders, and depression. These screening abilities offer unique advantages in that they can serve a broad strata of the society, including people in underserved populations and in countries with poor access to medical services. However, these advantages need to be balanced against the potential harm to privacy, autonomy, and nonmaleficence, which are recognized as the cornerstones of ethical medical care. Here, we discuss these opportunities and challenges, both when collecting data to develop online screening services and when deploying them. We offer several solutions that balance the advantages of these services with the ethical challenges they pose.

Background Oral contraceptives (OCs) are a unique chronic medication with which a memory slip may result in a threat that could change a person’s life course. Subjective concerns of missed OC doses among women have been addressed infrequently. Anonymized queries to internet search engines provide unique access to concerns and information gaps faced by a large number of internet users. Objective We aimed to quantitate the frequency of queries by women seeking information in an internet search engine, after missing one or more doses of an OC; their further queries on emergency contraception, abortion, and miscarriage; and their rate of reporting a pregnancy timed to the cycle of missing an OC. Methods We extracted all English-language queries submitted to Bing in the United States during 2018, which mentioned a missed OC and subsequent queries of the same users on miscarriage, abortion, emergency contraceptives, and week of pregnancy. Results We identified 26,395 Bing users in the United States who queried about missing OC pills and the fraction that further queried about miscarriage, abortion, emergency contraceptive, and week of pregnancy. Users under the age of 30 years who asked about forgetting an OC dose were more likely to ask about abortion (1.5 times) and emergency contraception (1.7 times) (P Conclusions Our data indicate that many women missing a dose of OC might not have received sufficient information from their health care providers or chose to obtain it online. Queries about abortion and miscarriage peaking in the subsequent days indicate a common worry of possible pregnancy. These results reinforce the importance of providing comprehensive written information on missed pills when prescribing an OC.

Search advertising is one of the most commonly-used methods of advertising. Past work has shown that search advertising can be employed to improve health by eliciting positive behavioral change. However, writing effective advertisements requires expertise and (possible expensive) experimentation, both of which may not be available to public health authorities wishing to elicit such behavioral changes, especially when dealing with a public health crises such as epidemic outbreaks. Here we develop an algorithm which builds on past advertising data to train a sequence-to-sequence Deep Neural Network which “translates” advertisements into optimized ads that are more likely to be clicked. The network is trained using more than 114 thousands ads shown on Microsoft Advertising. We apply this translator to two health related domains: Medical Symptoms (MS) and Preventative Healthcare (PH) and measure the improvements in click-through rates (CTR). Our experiments show that the generated ads are predicted to have higher CTR in 81% of MS ads and 76% of PH ads. To understand the differences between the generated ads and the original ones we develop estimators for the affective attributes of the ads. We show that the generated ads contain more calls-to-action and that they reflect higher valence (36% increase) and higher arousal (87%) on a sample of 1000 ads. Finally, we run an advertising campaign where 10 random ads and their rephrased versions from each of the domains are run in parallel. We show an average improvement in CTR of 68% for the generated ads compared to the original ads. Our results demonstrate the ability to automatically optimize advertisement for the health domain. We believe that our work offers health authorities an improved ability to help nudge people towards healthier behaviors while saving the time and cost needed to optimize advertising campaigns.

Background Surveys suggest that a large proportion of people use the internet to search for information on medical symptoms they experience and that around one-third of the people in the United States self-diagnose using online information. However, surveys are known to be biased, and the true rates at which people search for information on their medical symptoms before receiving a formal medical diagnosis are unknown. Objective This study aimed to estimate the rate at which people search for information on their medical symptoms before receiving a formal medical diagnosis by a health professional. Methods We collected queries made on a general-purpose internet search engine by people in the United States who self-identified their diagnosis from 1 of 20 medical conditions. We focused on conditions that have evident symptoms and are neither screened systematically nor a part of usual medical care. Thus, they are generally diagnosed after the investigation of specific symptoms. We evaluated how many of these people queried for symptoms associated with their medical condition before their formal diagnosis. In addition, we used a survey questionnaire to assess the familiarity of laypeople with the symptoms associated with these conditions. Results On average, 15.49% (1792/12,367, SD 8.4%) of people queried about symptoms associated with their medical condition before receiving a medical diagnosis. A longer duration between the first query for a symptom and the corresponding diagnosis was correlated with an increased likelihood of people querying about those symptoms (rho=0.6; P=.005); similarly, unfamiliarity with the association between a condition and its symptom was correlated with an increased likelihood of people querying about those symptoms (rho=−0.47; P=.08). In addition, worrying symptoms were 14% more likely to be queried about. Conclusions Our results indicate that there is large variability in the percentage of people who query the internet for their symptoms before a formal medical diagnosis is made. This finding has important implications for systems that attempt to screen for medical conditions.

Background Colonoscopy as a screening and diagnostic tool is generally safe and well-tolerated, and significant complications are rare. The rate of more mild adverse effects is difficult to estimate, particularly when such effects do not result in hospital admission. We aimed to identify the rate and timing of adverse effects as reported by users querying symptoms on an internet search engine. Methods We identified queries made to Bing originating from users in the United States containing the word “colonoscopy” during a 12-month period and identified those queries in which the timing of colonoscopy could be estimated. We then identified queries from those same users for medical symptoms during the time span from 5 days before through 30 days after the colonoscopy date. Results Of 641,223 users mentioning colonoscopy, 7013 (1.1%) had a query that enabled identification of their colonoscopy date. The majority of queries about colonoscopy preceded the procedure, and concerned diet. 28% of colonoscopy-related queries were made afterwards, and included queries about diarrhea and cramps, with 2.6% of users querying respiratory symptoms after the procedure, including cough (1.2%) and pneumonia (0.6%). Respiratory symptoms rose significantly at days 7–10 after the colonoscopy. Conclusions Internet search queries for respiratory symptoms rose approximately one week after queries relating to colonoscopy, raising the possibility that such symptoms are an under-reported late adverse effect of the procedure. Given the widespread use of colonoscopy as a screening modality and the rise of anesthesia-assisted colonoscopy in the United States in recent years, this signal is of potential public health concern.

Despite widespread use of the Internet and social media platforms by the public, there has been little organized exchange of information among the academic, government, and technology sectors about how digital communication technologies can be maximized to improve public health. The second Digital Health Promotion Executive Leadership Summit convened some of the world’s leading thinkers from across these sectors to revisit how communication technology and the evolving social media platforms can be utilized to improve both individual and population health. The Summit focused on digital intelligence, the spread of misinformation, online patient communities, censorship in social media, and emerging global legal frameworks. In addition, Summit participants had an opportunity to review the original “Common Agenda” that emerged and was published after the inaugural Summit and recommend updates regarding the uses of digital technology for advancing the goals of public health. This article reports the outcomes of the Summit discussions and presents the updates that were recommended by Summit participants as the Digital Health Communication Common Agenda 2.0. Several of the assertions underlying the original Common Agenda have been modified, and several new assertions have been added to reflect the recommendations. In addition, a corresponding set of principles and related actions—including a recommendation that an interagency panel of the U.S. Department of Health and Human Services be established to focus on digital health communication, with particular attention to social media—have been modified or supplemented.

Existing frameworks have identified a range of intervention design features that may facilitate adherence to eHealth interventions; however, empirical data are lacking on whether intervention design features can predict user adherence in the real world-where the public access available tools-and whether some design aspects of behavioral eHealth interventions are more important than others in predicting adherence. This study examined whether intervention design qualities predict user adherence to behavioral eHealth interventions in real-world use and which qualities matter the most. We correlated the online activities of users of 30 web-based behavioral interventions-collected from a proprietary data set of anonymized logs from consenting users of Microsoft Internet Explorer add-on-with interventions' quality ratings obtained by trained raters prior to empirical examination. The quality ratings included: Usability, Visual Design, User Engagement, Content, Therapeutic Persuasiveness (i.e., persuasive design and incorporation of behavior change techniques), and Therapeutic Alliance. We found Therapeutic Persuasiveness (i.e., the incorporation of persuasive design/behavior change principles) to be the most robust predictor of adherence (i.e., duration of use, number of unique sessions; 40 ≤ rs ≤ .58, ps ≤ .005), explaining 42% of the variance in user adherence in our regression model. Results indicated up to six times difference in the percentage of users utilizing the interventions for more than a minimum amount of time and sessions based on Therapeutic Persuasiveness. Findings suggest the importance of persuasive design and behavior change techniques incorporation during the design and evaluation of digital behavioral interventions.

Background Although autism is often characterized in literature by the presence of repetitive behavior, in structured decision tasks, individuals with autism spectrum disorder (ASD) have been found to examine more options in a given time period than controls. Objective We aimed to examine whether this investigative tendency emerges in information searches conducted via the internet. Methods In total, 1746 search engine users stated that they had ASD in 2019. This group’s naturally occurring responses following 1491 unique general queries and 78 image queries were compared to those of all other users of the search engine. The main dependent measure was scrolled distance, which denoted the extent to which additional results were scanned beyond the initial results presented on-screen. Additionally, we examined the number of clicks on search results as an indicator of the degree of search outcome exploitation and assessed whether there was a trade-off between increased search range and the time invested in viewing initial search results. Results After issuing general queries, individuals with self-stated ASD scanned more results than controls. The scrolled distance in the results page of general queries was 45% larger for the group of individuals with ASD (P Conclusions Individuals who self-stated that they had ASD scrutinized more general search results and fewer image search results than the controls. Thus, our results at least partially support the notion that individuals with ASD exhibit investigative behaviors and suggest that textual searches are an important context for expressing such tendencies.

Background Cerebrovascular disease is a leading cause of mortality and disability. Common risk assessment tools for stroke are based on the Framingham equation, which relies on traditional cardiovascular risk factors to predict an acute event in the near decade. However, no tools are currently available to predict a near/impending stroke, which might alert patients at risk to seek immediate preventive action (eg, anticoagulants for atrial fibrillation, control of hypertension). Objective Here, we propose that an algorithm based on internet search queries can identify people at increased risk for a near stroke event. Methods We analyzed queries submitted to the Bing search engine by 285 people who self-identified as having undergone a stroke event and 1195 controls with regard to attributes previously shown to reflect cognitive function. Controls included random people 60 years and above, or those of similar age who queried for one of nine control conditions. Results The model performed well against all comparator groups with an area under the receiver operating characteristic curve of 0.985 or higher and a true positive rate (at a 1% false-positive rate) above 80% for separating patients from each of the controls. The predictive power rose as the stroke date approached and if data were acquired beginning 120 days prior to the event. Good prediction accuracy was obtained for a prospective cohort of users collected 1 year later. The most predictive attributes of the model were associated with cognitive function, including the use of common queries, repetition of queries, appearance of spelling mistakes, and number of queries per session. Conclusions The proposed algorithm offers a screening test for a near stroke event. After clinical validation, this algorithm may enable the administration of rapid preventive intervention. Moreover, it could be applied inexpensively, continuously, and on a large scale with the aim of reducing stroke events.

Objective: We investigated how cultural shifts occur with respect to the use of traditional medicine for influenza-like illness (ILI) within the Latino population near the US-Mexico border. Methods: We extracted searches of ILI treatments (modern, traditional western, and traditional Hispanic) to the Bing search engine from 5 US states near the US-Mexico border. The incidence of these searches was correlated with county-level demographic data and ILI incidence. Results: The fraction of queries related to Hispanic medicines is correlated with the distance from the US-Mexico border (Spearman rho = -0.24), which is a slower decay than that observed in the use of Spanish (Spearman rho = -0.35). Demographic and socioeconomic factors predict different searches for ILI treatments. Conclusions: People of Hispanic origin rely on traditional Hispanic medicines when treating ILI. Medical providers should guide their patients when such treatments might be clinically detrimental to patients, and know their patients are likely using such medicines.

The role of opinion leaders in the diffusion of innovation has recently come under scrutiny: On the one hand, their central role in accelerating diffusion has been recognized in industry, academia, and the popular media. On the other hand, it has been argued that opinion leaders do not create contagion processes that differ significantly from those of other types of customers. We offer here a synthesis of these opposing theses: For many applications, opinion leadership should be studied in small groups rather than in an entire network. Using data from two mobile operators, we show how our method of segmenting the market and defining local opinion leaders applies to churn. We show that opinion leaders are highly effective in small, strong-tie groups, and their effect considerably declines with larger groups and weaker ties. Opinion leaders are at the highest risk for churn compared with other group members, they are more likely to be the first to initiate churn, and when they churn, the probability of additional churners from their social group grows significantly, as compared to when non-leaders churn.

Experts are concerned about the spread and recalcitrance of vaccine misinformation and its contribution to vaccine hesitancy. Despite this risk, little research attention has been paid to understanding how individuals seek vaccine information online and evaluate its trustworthiness. Here, we hypothesized that when vaccine-hesitant parents seek information about vaccines, they prefer trustworthy sources based on their competence, integrity and benevolence. We explored this issue using 4910 questions and 2583 answers retrieved from two social question-and-answer (QA) platforms: "Yahoo! Answers" and the Facebook group "Talking about Vaccines." We examined what kinds of questions are asked about vaccines, to what extent they are explicitly directed at health professionals or parents, and what features of the answers predict perceived answer quality, based on the theory of epistemic trust. The findings indicate that on different platforms, vaccine-related questions focus on different topics; namely, questions on one platform focused on the risks and benefits of vaccination, whereas they dealt with vaccine schedules on the other. On both platforms, most questions did not specify that an answer should be based on professional expertise or parents' experience. Both pro-vaccine and anti-vaccine answers were proportionately represented among the "best answers". However, if an answer was written by a health professional, the askers and the community on "Yahoo! Answers" were twice as likely to choose it as the "best answer" to a vaccine-related question, irrespective of whether it encouraged or discouraged vaccination. By contrast, an online experiment revealed that both the identity of the respondent and the stance towards vaccination affected the perceived trustworthiness of the answers. These findings indicate that despite the proliferation of anti-vaccine messages, epistemic trust in mainstream science and medicine is robust. User responses to expert answers suggest that expert outreach in online environments may be an effective intervention to address vaccine hesitancy.

Objective To develop, apply, and evaluate, a novel web‐based classifier for screening for Parkinson disease among a large cohort of search engine users. Methods A supervised machine learning classifier learned to distinguish web users with self‐reported Parkinson's disease from controls based on their interactions with a search engine (Bing, Microsoft). It was then applied to groups of web users with low or high risk for actual Parkinson's disease. Textual content of web queries was used to sort surfers into the different risk groups, but not for classifying users as negative or positive for Parkinson's disease. Disease detection was unsolicited. Researchers did not have access to any identifying data on users. Results Applying the classifier (with an estimated positive predictive value of 25%) resulted in 17,843/1,490,987 (1.2%) web users over the age of 40 years screened positive for Parkinson's disease. This percentile was higher in at‐risk groups (Fisher exact P

Understanding processes that lead people to invest a certain amount of time in challenging tasks is important for theory and practice. In particular, researchers often assume strong linear associations between confidence, consensuality (the degree to which an answer is independently given by multiple participants), and response time. The Diminishing Criterion Model (DCM; Ackerman, 2014) is a metacognitive model which explains the stopping rules people employ under uncertainty in terms of the confidence-time association. This model is unique in predicting a curvilinear rather than a linear confidence-time association. Using consensuality as an alternative to confidence for predicting response time offers theoretical and practical opportunities. In four experiments, including replications and variations, we examined confidence (where collected) and consensuality as predictors of the time people invest in three problem-solving tasks and in real-life web searching. The results using consensuality, like those for confidence, fitted the curvilinear time pattern predicted by the DCM, with one exception: at least 30% of the population must endorse a potential answer for consensuality to predict response time based on the stopping rules in the DCM. Beyond examining consensuality as a predictor, the study brings converging evidence supporting the DCM's curvilinear confidence-time association over alternative models. The methodology used for analyzing web searching offers new directions for metacognitive research in naturally-performed tasks.

Background: Search engines display helpline notices when people query for suicide-related information. Objective: In this study, we aimed to examine if these notices and other information displayed in response to suicide-related queries are correlated with subsequent searches for suicide prevention rather than harmful information. Methods: Anonymous suicide-related searches made on Bing and Google in the United States, the United Kingdom, Hong Kong, and Taiwan in a span of 10 months were extracted. Descriptive analyses and regression models were fit to the data to assess the correlation with observed behaviors. Results: Display of helpline notices was not associated with an observed change in the likelihood of or future suicide searches (P=.42). No statistically significant differences were observed in the likelihood of people making future suicide queries (both generally and specific types of suicide queries) when comparing search engines in locations that display helpline notices versus ones that do not. Pages with higher rank, being neutral to suicide, and those shown among more antisuicide pages were more likely to be clicked on. Having more antisuicide Web pages displayed was the only factor associated with further searches for suicide prevention information (hazard=1.18, P=.002). Conclusions: Helpline notices are not associated with harm. If they cause positive change in search behavior, it is small. This is possibly because of the variability in intent of users seeking suicide-related information. Nonetheless, helpline notice should be displayed, but more efforts should be made to improve the visibility and ranking of suicide prevention Web pages.

Computer malware and biological pathogens often use similar mechanisms of infections. For this reason, it has been suggested to model malware spread using epidemiological models developed to characterize the spread of biological pathogens. However, most work examining the similarities between malware and pathogens using such methods was based on theoretical analysis and simulation. Here we extend the classical Susceptible-Infected-Recovered (SIR) epidemiological model to describe two of the most common infection methods used by malware. We fit the proposed model to malware collected over a period of one year from a major anti-malware vendor. We show that by fitting the proposed model it is possible to identify the method of transmission used by the malware, its rate of infection, and the number of machines which will be infected unless blocked by anti-virus software. The Spearman correlation between the number of actual and predicted infected machines is 0.84. Examining cases where an anti-malware "signature" was transmitted to susceptible computers by the anti-virus provider, we show that the time to remove the malware will be short and independent of the number of infected computers if fewer than approximately 60% of susceptible computers have been infected. If more computers were infected, the time to removal will be approximately 3.2 greater, and will depend on the fraction of infected computers. Our results show that the application of epidemiological models of infection to malware can provide anti-virus providers with information on malware spread and its potential damage. We further propose that similarities between computer malware and biological pathogens, the availability of data on the former and the dearth of data on the latter, make malware an extremely useful model for testing interventions which could later be applied to improve medicine., Comment: 9 pages, 2 figure, 2 tables

People increasingly turn to the Internet when they have a medical condition. The data they create during this process is a valuable source for medical research and future health services. When used for these purposes, it is imperative to balance use with user privacy. One way to understand how to harmonize these requirements is to match the perceived value that users assign to their data with the value of the services derived from them. Here we describe novel experiments where methods from Mechanism Design, Crowdsourcing and Data Science were used together to elicit truthful valuations from users for their Internet data and for services derived from these data, specifically for medical screening. In these experiments, 880 people from around the world were asked to participate in an auction to provide their data for uses differing in their contribution to the participant and to society, and in the disease they addressed. Some users were offered monetary compensation for their participation, while others were asked to pay to participate. Our findings show that 99% of people were willing to contribute their data in exchange for monetary compensation and an analysis of their data, while 53% were willing to pay to have their data analyzed. The average perceived value users assigned to their data was estimated at US$49. Their value for services which offer personal benefit to them was US$22, while the value of this service offered to the general public was US$20. Our findings show that it is possible to place a monetary value on health-related uses of highly personal data. Our methodology can be extended to other areas where sensitive data may be exchanged for services to individuals and to society.

ObjectivesRapid detection and surveillance of COVID-19 is essential to reducing spread of the virus. Inadequate screening capacity has hampered COVID-19 detection, while traditional infectious disease response has been delayed due to significant demands for healthcare resources, time and personnel. This study investigated whether an online health decision-support tool could supplement COVID-19 surveillance and detection in China and the USA.SettingDaily website traffic to Thermia was collected from China and the USA, and cross-correlation analyses were used to assess the designated lag time between the daily time series of Thermia sessions and COVID-19 case counts from 22 January to 23 April 2020.ParticipantsThermia is a validated health decision-support tool that was modified to include content aimed at educating users about Centers for Disease Control and Prevention recommendations on COVID-19 symptoms. An advertising campaign was released on Microsoft Advertising to refer searches for COVID-19 symptoms to Thermia.ResultsThe lead times observed for Thermia sessions to COVID-19 case reports was 3 days in China and 19 days in the USA. We found negative cross-correlation between the number of Thermia sessions and rates of influenza A and B, possibly due to the decreasing prevalence of influenza and the lack of specificity of the system for identification of COVID-19.ConclusionThis study suggests that early deployment of an online campaign and modified health decision-support tool may support identification of emerging infectious diseases like COVID-19. Researchers and public health officials should deploy web campaigns as early as possible in an epidemic to detect, identify and engage those potentially at risk to help prevent transmission of the disease.

The Internet continues to be an important supplemental health information resource for an increasing number of U.S. adults, especially for those with a new or existing chronic condition. Here we examine how people use the Internet to learn about Type 2 diabetes and how health literacy (HL) influences this information-seeking behavior. We analyzed the searches of approximately 2 million people who queried for diabetes-related information on Microsoft's Bing search engine. The HL of searchers was imputed through a community-based HL score. Topics searched were categorized and subsequent websites were assessed for readability. Overall, diabetes information-seeking strategies via the Internet are similar among adults with limited and adequate HL skills. However, people with limited HL take a longer time to read pages that are quickly read by people with adequate HL and vice versa. Information seeking among the former is terminated prematurely, as is evident from a Hidden Markov Model of the search process. Our findings indicate that the reading level required to understand the majority of diabetes-related information is high. Especially on government websites, more than 80% of information requires a reading level corresponding to 7th grade or higher. Our results indicate that individuals with lower HL may disproportionately struggle with Internet searches and fail to get an equivalent benefit from this information resource compared to users with greater HL. Future interventions should target the quality and ease of navigation of health care websites and find ways to leverage other relevant professionals to encourage and promote successful information access on the Web.

Background There is a debate about the effects of pro-anorexia (colloquially referred to as pro-ana) websites. Research suggests that the effect of these websites is not straightforward. Indeed, the actual function of these sites is disputed, with studies indicating both negative and positive effects. Aim This is the first study which systematically examined the differences between pro-anorexia web communities in four main aspects: web language used (posts); web interests/search behaviors (queries); users' self-reported weight status and weight goals; and associated self-reported mood/pathology. Methods We collected three primary sources of data, including messages posed on three pro-ana websites, a survey completed by over 1000 participants of a pro-ana website, and the searches made on the Bing search engine of pro-anorexia users. These data were analyzed for content, reported demographics and pathology, and behavior over time. Results Although members of the main pro-ana website investigated appear to be depressed, with high rates of self-harm and suicide attempts, users are significantly more interested in treatment, have wishes of procreation and reported the highest goal weights among the investigated sites. In contrast, users of other pro-ana websites investigated, are more interested in morbid themes including depression, self-harm and suicide. The percentage of severely malnourished website users, in general, appears to be small (20%). Conclusions Our results indicate that a new strategy is required to facilitate the communication between mental health specialists and pro-ana web users, recognizing the differences in harm associated with different websites.

IntroductionDepression and diabetes are highly disabling diseases with a high prevalence and high rate of comorbidity, particularly in low-income ethnic minority patients. Though comorbidity increases the risk of adverse outcomes and mortality, most clinical interventions target these diseases separately. Increasing physical activity might be effective to simultaneously lower depressive symptoms and improve glycaemic control. Self-management apps are a cost-effective, scalable and easy access treatment to increase physical activity. However, cutting-edge technological applications often do not reach vulnerable populations and are not tailored to an individual’s behaviour and characteristics. Tailoring of interventions using machine learning methods likely increases the effectiveness of the intervention.Methods and analysisIn a three-arm randomised controlled trial, we will examine the effect of a text-messaging smartphone application to encourage physical activity in low-income ethnic minority patients with comorbid diabetes and depression. The adaptive intervention group receives messages chosen from different messaging banks by a reinforcement learning algorithm. The uniform random intervention group receives the same messages, but chosen from the messaging banks with equal probabilities. The control group receives a weekly mood message. We aim to recruit 276 adults from primary care clinics aged 18–75 years who have been diagnosed with current diabetes and show elevated depressive symptoms (Patient Health Questionnaire depression scale-8 (PHQ-8) >5). We will compare passively collected daily step counts, self-report PHQ-8 and most recent haemoglobin A1c from medical records at baseline and at intervention completion at 6-month follow-up.Ethics and disseminationThe Institutional Review Board at the University of California San Francisco approved this study (IRB: 17-22608). We plan to submit manuscripts describing our user-designed methods and testing of the adaptive learning algorithm and will submit the results of the trial for publication in peer-reviewed journals and presentations at (inter)-national scientific meetings.Trial registration numberNCT03490253; pre-results.

Parkinson's disease (PD) is a slowly progressing neurodegenerative disease with early manifestation of motor signs. Recently, there has been a growing interest in developing automatic tools that can assess motor function in PD patients. Here we show that mouse tracking data collected during people's interaction with a search engine can be used to distinguish PD patients from similar, non-diseased users and present a methodology developed for the diagnosis of PD from these data. A main challenge we address is the extraction of informative features from raw mouse tracking data. We do so in two complementary ways: First, we manually construct expert-recommended informative features, aiming to identify abnormalities in motor behaviors. Second, we use an unsupervised representation learning technique to map these raw data to high-level features. Using all the extracted features, a Random Forest classifier is then used to distinguish PD patients from controls, achieving an AUC of 0.92, while results using only expert-generated or auto-generated features are 0.87 and 0.83, respectively. Our results indicate that mouse tracking data can help in detecting users at early stages of the disease, and that both expert-generated features and unsupervised techniques for feature generation are required to achieve the best possible performance

Background Respiratory Syncytial Virus (RSV) is the leading cause of hospitalization in children less than 1 year of age in the United States. Internet search engine queries may provide high resolution temporal and spatial data to estimate and predict disease activity. Methods After filtering an initial list of 613 symptoms using high-resolution Bing search logs, we used Google Trends data between 2004 and 2016 for a smaller list of 50 terms to build predictive models of RSV incidence for five states where long-term surveillance data was available. We then used domain adaptation to model RSV incidence for the 45 remaining US states. Results Surveillance data sources (hospitalization and laboratory reports) were highly correlated, as were laboratory reports with search engine data. The four terms which were most often statistically significantly correlated as time series with the surveillance data in the five state models were RSV, flu, pneumonia, and bronchiolitis. Using our models, we tracked the spread of RSV by observing the time of peak use of the search term in different states. In general, the RSV peak moved from south-east (Florida) to the north-west US. Conclusions Our study represents the first time that RSV has been tracked using Internet data results and highlights successful use of search filters and domain adaptation techniques, using data at multiple resolutions. Our approach may assist in identifying spread of both local and more widespread RSV transmission and may be applicable to other seasonal conditions where comprehensive epidemiological data is difficult to collect or obtain.

Introduction We conducted this study to quantify how health professionals use Twitter to communicate about the human papillomavirus (HPV) vaccine. Methods We collected 193,379 tweets from August 2014 through July 2015 that contained key words related to HPV vaccine. We classified all tweets on the basis of user, audience, sentiment, content, and vaccine characteristic to examine 3 groups of tweets: 1) those sent by health professionals, 2) those intended for parents, and 3) those sent by health professionals and intended for parents. For each group, we identified the 7-day period in our sample with the most number of tweets (spikes) to report content. Results Of the 193,379 tweets, 20,451 tweets were from health professionals; 16,867 tweets were intended for parents; and 1,233 tweets overlapped both groups. The content of each spike varied per group. The largest spike in tweets from health professionals (n = 851) focused on communicating recently published scientific evidence. Most tweets were positive and were about resources and boys. The largest spike in tweets intended for parents (n = 1,043) centered on a national awareness day and were about resources, personal experiences, boys, and girls. The largest spike in tweets from health professionals to parents (n = 89) was in January and centered on an event hosted on Twitter that focused on cervical cancer awareness month. Conclusion Understanding drivers of tweet spikes may help shape future communication and outreach. As more parents use social media to obtain health information, health professionals and organizations can leverage awareness events and personalize messages to maximize potential reach and parent engagement.

Background: Attention Deficit Hyperactivity Disorder (ADHD) is a common neurodevelopmental disorder, prevalent among 2-10% of the population. Objective: The objective of this study was to describe where, what, and when people search online for topics related to ADHD. Methods: Data were collected from Microsoft’s Bing search engine and from the community question and answer site, Yahoo Answers. The questions were analyzed based on keywords and using further statistical methods. Results: Our results revealed that the Internet indeed constitutes a source of information for people searching the topic of ADHD, and that they search for information mostly about ADHD symptoms. Furthermore, individuals personally affected by the disorder made 2.0 more questions about ADHD compared with others. Questions begin when children reach 2 years of age, with an average age of 5.1 years. Most of the websites searched were not specifically related to ADHD and the timing of searches as well as the query content were different among those prediagnosis compared with postdiagnosis. Conclusions: The study results shed light on the features of ADHD-related searches. Thus, they may help improve the Internet as a source of reliable information, and promote improved awareness and knowledge about ADHD as well as quality of life for populations dealing with the complex phenomena of ADHD. [J Med Internet Res 2017;19(4):e126]

Post-marketing drug surveillance is largely based on signals found in spontaneous reports from patients and healthcare providers. Rare adverse drug reactions and adverse events (AEs) that may develop after long-term exposure to a drug or from drug interactions may be missed. The US FDA and others have proposed that web-based data could be mined as a resource to detect latent signals associated with adverse drug reactions. Recently, a web-based search query method called a query log reaction score (QLRS) was developed to detect whether AEs associated with certain drugs could be found from search engine query data. In this study, we compare the performance of two other algorithms, the proportional query ratio (PQR) and the proportional query rate ratio (Q-PRR) against that of two reference signal-detection algorithms (SDAs) commonly used with the FDA AE Reporting System (FAERS) database. In summary, the web query methods have moderate sensitivity (80%) in detecting signals in web query data compared with reference SDAs in FAERS when the web query data are filtered, but the query metrics generate many false-positives and have low specificity compared with reference SDAs in FAERS. Future research is needed to find better refinements of query data and/or the metrics to improve the specificity of these web query log algorithms.

Background: Although hundreds of millions of dollars are spent each year on public health advertising, the advertisement content, design, and placement are usually developed by intuition rather than research. Objective: The objective of our study was to develop a methodology for testing Web-based advertisements to promote smoking cessation. Methods: We developed 10 advertisements that varied by their content (those that empower viewers to quit, help viewers to quit, or discuss the effects of smoking). We then conducted a series of Web-based randomized controlled trials that explored the effects of exposing users of Microsoft’s Bing search engine to antismoking advertisements that differed by content, placement, or other characteristics. Finally, we followed users to explore whether they conducted subsequent searches for smoking cessation products or services. Results: The advertisements were shown 710,106 times and clicked on 1167 times. In general, empowering advertisements had the greatest impact (hazard ratio [HR] 2.6, standard error [SE] 0.09 relative to nonempowering advertisements), but we observed significant variations by gender. For instance, we found that men exposed to smoking cessation advertisements were less likely than women to subsequently conduct smoking cessation searches (HR 0.2, SE 0.07), but that this likelihood increased 3.5 times in men exposed to advertisements containing empowering content. Women were more influenced by advertisements that emphasized the health effects of smoking. We also found that appearing at the top right of the page (HR 2.1, SE 0.07) or at the bottom rather than the top of a list (HR 1.1, SE 0.02) can improve smoking cessation advertisements’ effectiveness in prompting future searches related to smoking cessation. Conclusions: Advertising should be targeted to different demographic groups in ways that are not always intuitive. Our study provides a method for testing the effectiveness of Web-based antismoking advertisements and demonstrates the importance of advertisements that are tailored according to specific demographics. [J Med Internet Res 2016;18(11):e306]

BackgroundBrief intervention is a critical method for identifying patients with problematic substance use in primary care settings and for motivating them to consider treatment options. However, despite considerable evidence of delay discounting in patients with substance use disorders, most brief advice by physicians focuses on the long-term negative medical consequences, which may not be the best way to motivate patients to seek treatment information.ObjectiveIdentification of the specific symptoms that most motivate individuals to seek treatment information may offer insights for further improving brief interventions. To this end, we used anonymized internet search engine data to investigate which medical conditions and symptoms preceded searches for 12-step meeting locators and general 12-step information.MethodsWe extracted all queries made by people in the United States on the Bing search engine from November 2016 to July 2017. These queries were filtered for those who mentioned seeking Alcoholics Anonymous (AA) or Narcotics Anonymous (NA); in addition, queries that contained a medical symptom or condition or a synonym thereof were analyzed. We identified medical symptoms and conditions that predicted searches for seeking treatment at different time lags. Specifically, symptom queries were first determined to be significantly predictive of subsequent 12-step queries if the probability of querying a medical symptom by those who later sought information about the 12-step program exceeded the probability of that same query being made by a comparison group of all other Bing users in the United States. Second, we examined symptom queries preceding queries on the 12-step program at time lags of 0-7 days, 7-14 days, and 14-30 days, where the probability of asking about a medical symptom was greater in the 30-day time window preceding 12-step program information-seeking as compared to all previous times that the symptom was queried.ResultsIn our sample of 11,784 persons, we found 10 medical symptoms that predicted AA information seeking and 9 symptoms that predicted NA information seeking. Of these symptoms, a substantial number could be categorized as nonsevere in nature. Moreover, when medical symptom persistence was examined across a 1-month time period, a substantial number of nonsevere, yet persistent, symptoms were identified.ConclusionsOur results suggest that many common or nonsevere medical symptoms and conditions motivate subsequent interest in AA and NA programs. In addition to highlighting severe long-term consequences, brief interventions could be restructured to highlight how increasing substance misuse can worsen discomfort from common medical symptoms in the short term, as well as how these worsening symptoms could exacerbate social embarrassment or decrease physical attractiveness.

Background: Celiac disease, a common immune-based disease triggered by gluten, has diverse clinical manifestations, and the relative distribution of symptoms leading to diagnosis has not been well characterized in the population. Objective: This study aimed to use search engine data to identify a set of symptoms and conditions that would identify individuals at elevated likelihood of a subsequent celiac disease diagnosis. We also measured the relative prominence of these search terms before versus after a search related to celiac disease. Methods: We extracted English-language queries submitted to the Bing search engine in the United States and identified those who submitted a new celiac-related query during a 1-month period, without any celiac-related queries in the preceding 9 months. We compared the ratio between the number of times that each symptom or condition was asked in the 14 days preceding the first celiac-related query of each person and the number of searches for that same symptom or condition in the 14 days after the celiac-related query. Results: We identified 90,142 users who made a celiac-related query, of whom 6528 (7%) exhibited sustained interest, defined as making a query on more than 1 day. Though a variety of symptoms and associated conditions were also queried before a celiac-related query, the maximum area under the receiver operating characteristic curve was 0.53. The symptom most likely to be queried more before than after a celiac-related query was diarrhea (query ratio [QR] 1.28). Extraintestinal symptoms queried before a celiac disease query included headache (QR 1.26), anxiety (QR 1.10), depression (QR 1.03), and attention-deficit hyperactivity disorder (QR 1.64). Conclusions: We found an increase in antecedent searches for symptoms known to be associated with celiac disease, a rise in searches for depression and anxiety, and an increase in symptoms that are associated with celiac disease but may not be reported to health care providers. The protean clinical manifestations of celiac disease are reflected in the diffuse nature of antecedent internet queries of those interested in celiac disease, underscoring the challenge of effective case-finding strategies.

The ability of modern web services such as news aggregators and search engines to tailor their results to the tastes of individuals, together with people's preference for reading opinions which reinforce their own viewpoints, have raised concerns that people are nowadays exposed to a narrow range of view-points, a phenomenon referred to as the “filter bubble”. In this paper we focus on increasing exposure to varied political opinions with a goal of improving civil discourse. We develop a method to algorithmically encourage people to read diverse political opinions and test it when people actively seek information. First, analyzing data from a popular search engine we show that people are indeed more likely to read opinions consistent with their own. Interestingly, they are more likely to read news from opposing sites when the language model of a particular news item is close to the language model of their own political leaning. Based on this finding, we describe a method for assisting people to read divergent opinions by choosing documents of opposing viewpoints that have a language model closer to their own language model. We test our method on a number of web searchers and show that pages of the opposing side which were more similar than the average persons' own language model tended to be clicked 38% more than those below. We also describe the long-term effects of our method, showing that people who were shown more diverse results continued reading more diverse results and overall became more interested in news.

Objective Links between media portrayals of celebrities and participation in activities related to anorexia are of interest to both researchers and practitioners but are difficult to study over long time periods and in large populations. Here we aim to determine the links between media portrayals of celebrities and online practices related to anorexia. Method We examined the Internet searching activities of 9.2 million people, focusing on searches related to known celebrities, especially those perceived by the public as suffering from anorexia, and on searches indicative of anorexic practices. Additionally, we tracked media attention of individual celebrities by monitoring all messages from Twitter related to those celebrities. Results We found that a subset of users focus their browsing activities on celebrities perceived as anorexic. There was an increase of 14% in the hazard of performing anorexia-related searches after searching for information on the celebrities most perceived as anorexic. Media attention of a celebrity was a trigger for search activity, and when focused on a celebrity perceived as anorexic, resulted in a 33% increase in hazard for carrying out anorexic searches. Strikingly, when media attention included a reference to anorexia, the hazard decreased by 22%. Discussion Our findings suggest that it would be beneficial for media, when reporting on those celebrities who are known to suffer from anorexia, to include this information in their reporting. © 2013 Wiley Periodicals, Inc. (Int J Eat Disord 2014; 47:196-202)

When a new treatment is considered for use, whether a pharmaceutical drug or a search engine ranking algorithm, a typical question that arises is, will its performance exceed that of the current treatment? The conventional way to answer this counterfactual question is to estimate the effect of the new treatment in comparison to that of the conventional treatment by running a controlled, randomized experiment. While this approach theoretically ensures an unbiased estimator, it suffers from several drawbacks, including the difficulty in finding representative experimental populations as well as the cost of running randomized trials. Moreover, such trials neglect the huge quantities of available control-condition data, which in principle can be utilized for the harder task of predicting individualized effects. In this paper, we propose a discriminative framework for predicting the outcomes of a new treatment from a large dataset of the control condition and data from a small (and possibly unrepresentative) randomized trial comparing new and old treatments. Our learning objective, which requires minimal assumptions on the treatments, models the relation between the outcomes of the different conditions. This allows us to not only estimate mean effects but also to generate individual predictions for examples outside the small randomized sample. We demonstrate the utility of our approach through experiments in three areas: search engine operation, treatments to diabetes patients, and market value estimation of houses. Our results demonstrate that our approach can reduce the number and size of the currently performed randomized controlled experiments, thus saving significant time, money and effort on the part of practitioners.

Background: Human papillomavirus (HPV) is the most common sexually transmitted infection in the United States. There are several vaccines that protect against strains of HPV most associated with cervical and other cancers. Thus, HPV vaccination has become an important component of adolescent preventive health care. As media evolves, more information about HPV vaccination is shifting to social media platforms such as Twitter. Health information consumed on social media may be especially influential for segments of society such as younger populations, as well as ethnic and racial minorities. Objective: The objectives of our study were to quantify HPV vaccine communication on Twitter, and to develop a novel methodology to improve the collection and analysis of Twitter data. Methods: We collected Twitter data using 10 keywords related to HPV vaccination from August 1, 2014 to July 31, 2015. Prospective data collection used the Twitter Search API and retrospective data collection used Twitter Firehose. Using a codebook to characterize tweet sentiment and content, we coded a subsample of tweets by hand to develop classification models to code the entire sample using machine learning procedures. We also documented the words in the 140-character tweet text most associated with each keyword. We used chi-square tests, analysis of variance, and nonparametric equality of medians to test for significant differences in tweet characteristic by sentiment. Results: A total of 193,379 English-language tweets were collected, classified, and analyzed. Associated words varied with each keyword, with more positive and preventive words associated with “HPV vaccine” and more negative words associated with name-brand vaccines. Positive sentiment was the largest type of sentiment in the sample, with 75,393 positive tweets (38.99% of the sample), followed by negative sentiment with 48,940 tweets (25.31% of the sample). Positive and neutral tweets constituted the largest percentage of tweets mentioning prevention or protection (20,425/75,393, 27.09% and 6477/25,110, 25.79%, respectively), compared with only 11.5% of negative tweets (5647/48,940; P

Collaborative filtering (CF) recommendation systems are one of the most popular and successful methods for recommending products to people. CF systems work by finding similarities between different people according to their past purchases, and using these similarities to suggest possible items of interest. In this work we show that CF systems can be enhanced using Internet browsing data and search engine query logs, both of which represent a rich profile of individuals' interests.