Search

Background The ability to regulate anger is often impaired in forensic psychiatric patients, frequently resulting in aggressive behavior. Although some treatment programs are partially successful in enhancing aggression regulation and reducing recidivism among specific subgroups, generalizable conclusions on the effectiveness of these interventions cannot be drawn to date. In forensic outpatient care, low treatment adherence and a predominant focus on cognitive control in most treatment programs may entail some of the factors impeding treatment. Technology-based interventions may address some of these treatment challenges. Objective The aim of this study is to explore whether a new technology-based biocueing intervention, the Sense-IT app, can be a valuable addition to aggression regulation treatment programs in forensic outpatient care. The Sense-IT app, which provides the user with real-time physiological feedback and behavioral support, is developed to strengthen emotional awareness and facilitate real-life practice. In this study, we aim to develop and evaluate an updated version of the Sense-IT app that is suitable for forensic outpatients with aggressive behavior. Methods First, we conducted a design study to assess the attitudes of forensic professionals and patients toward biocueing and to collect requirements for a biocueing app for this specific population. On the basis of this information, we developed an updated version of the Sense-IT app. In an evaluation study, 10 forensic outpatients used the app for 2 weeks. The app’s acceptability, usability, and clinical outcomes (aggression, anger, and recognition of bodily signals related to anger) were measured before and after the intervention using both quantitative and qualitative measures. Results The design study revealed a cautiously positive attitude toward the use of biocueing as an addition to aggression regulation therapy. The evaluation study among forensic outpatients demonstrated moderate acceptability and adequate usability for the new version of the Sense-IT app. Exploratory analysis revealed a significant decrease in trait aggression postintervention; no significant changes were found in other anger-related clinical outcomes. To further increase acceptability and usability, a stable functioning app with self-adjustable settings, the use of smartwatches with a longer battery life, and the use of the patient’s own smartphone devices were recommended. Conclusions This study, which is one of the first attempts to enroll and evaluate the real-life use of a biocueing intervention among forensic outpatients, emphasized the importance of involving both patients and therapists throughout the development and implementation process. In the future, experimental studies, including single-case experimental designs using ecological momentary assessment, should be performed to evaluate the effectiveness of the Sense-IT intervention on clinical outcomes. An open attitude toward new technology, allowing exploration of the potential benefits of the Sense-IT app case-by-case, and training of therapists in using the app are expected to facilitate its integration in therapy.

Background Prenatal genetic testing is an essential part of routine prenatal care. Yet, obstetricians often lack the time to provide comprehensive prenatal genetic testing education to their patients. Pregnant women lack prenatal genetic testing knowledge, which may hinder informed decision-making during their pregnancies. Due to the rapid growth of technology, mobile apps are a potentially valuable educational tool through which pregnant women can learn about prenatal genetic testing and improve the quality of their communication with obstetricians. The characteristics, quality, and number of available apps containing prenatal genetic testing information are, however, unknown. Objective This study aims to conduct a firstreview to identify, evaluate, and summarize currently available mobile apps that contain prenatal genetic testing information using a systematic approach. Methods We searched both the Apple App Store and Google Play for mobile apps containing prenatal genetic testing information. The quality of apps was assessed based on the criteria adopted from two commonly used and validated mobile app scoring systems, including the Mobile Application Rating Scale (MARS) and the APPLICATIONS evaluation criteria. Results A total of 64 mobile apps were identified. Of these, only 2 apps were developed for a specific prenatal genetic test. All others were either pregnancy-related (61/64, 95%) or genetics-related (1/64, 2%) apps that provided prenatal genetic testing information. The majority of the apps (49/64, 77%) were developed by commercial companies. The mean quality assessment score of the included apps was 13.5 (SD 2.9), which was equal to the average of possible theoretical score. Overall, the main weaknesses of mobile apps in this review included the limited number of prenatal genetic tests mentioned; incomprehensiveness of testing information; unreliable and missing information sources; absence of developmental testing with users (not evidence based); high level of readability; and the lack of visual information, customization, and a text search field. Conclusions Our findings suggest that the quality of mobile apps with prenatal genetic testing information must be improved and that pregnant women should be cautious when using these apps for prenatal genetic testing information. Obstetricians should carefully examine mobile apps before referring any of them to their patients for use as an educational tool. Both improving the quality of existing mobile apps, and developing new, evidence-based, high-quality mobile apps targeting all prenatal genetic tests should be the focus of mobile app developers going forward.

Background Talking about experiences of sexual abuse in therapy is difficult for children and adolescents. Possible reasons for this difficulty are a lack of vocabulary to describe the situation or feelings of shame, fear, and self-blame associated with sexual abuse. The serious game Vil Du?! was developed to help children open up about their sexual abuse experiences. Vil Du?! is a nonverbal communication game that resembles a dress-up game in which children can show the therapist what happened to them. Objective This exploratory evaluation study examines which working elements of the game could be identified in therapy with victims of sexual abuse (aim 1). In addition, this study examines how therapists evaluate the acceptability of the game (aim 2). Methods The therapists completed 23 web-based surveys on the use of Vil Du?! In addition, semistructured interviews were conducted with 10 therapists. The data were analyzed in NVivo following previously reported stepwise guidelines. Results Regarding aim 1, therapists mentioned various working elements of Vil Du?!; for instance, Vil Du?! puts the child in control of the situation. In addition, Vil Du?! reduces barriers to disclosure because there is no need to talk or have eye contact with the therapist. Regarding aim 2, Vil Du?! was generally evaluated more positively than negatively by the therapists. For instance, therapists indicated that using Vil Du?! is time efficient and might make the treatment process less confronting and difficult for the client. According to therapists, most clients indeed experienced less tension and more positive (or neutral) emotions than negative emotions when using Vil Du?! Conclusions The most important working elements of Vil Du?!, according to therapists, are that it enables children to regain control over their sexual abuse experiences and reduces barriers to disclosing sexual abuse experiences. The more positive evaluation of Vil Du?! indicates the acceptability of the game for therapists as well as their clients.

BackgroundNews media coverage of antimask protests, COVID-19 conspiracies, and pandemic politicization has overemphasized extreme views but has done little to represent views of the general public. Investigating the public’s response to various pandemic restrictions can provide a more balanced assessment of current views, allowing policy makers to craft better public health messages in anticipation of poor reactions to controversial restrictions.ObjectiveUsing data from social media, this infoveillance study aims to understand the changes in public opinion associated with the implementation of COVID-19 restrictions (eg, business and school closures, regional lockdown differences, and additional public health restrictions, such as social distancing and masking).MethodsCOVID-19–related tweets in Ontario (n=1,150,362) were collected based on keywords between March 12 and October 31, 2020. Sentiment scores were calculated using the VADER (Valence Aware Dictionary and Sentiment Reasoner) algorithm for each tweet to represent its negative to positive emotion. Public health restrictions were identified using government and news media websites. Dynamic regression models with autoregressive integrated moving average errors were used to examine the association between public health restrictions and changes in public opinion over time (ie, collective attention, aggregate positive sentiment, and level of disagreement), controlling for the effects of confounders (ie, daily COVID-19 case counts, holidays, and COVID-19–related official updates).ResultsIn addition to expected direct effects (eg, business closures led to decreased positive sentiment and increased disagreements), the impact of restrictions on public opinion was contextually driven. For example, the negative sentiment associated with business closures was reduced with higher COVID-19 case counts. While school closures and other restrictions (eg, masking, social distancing, and travel restrictions) generated increased collective attention, they did not have an effect on aggregate sentiment or the level of disagreement (ie, sentiment polarization). Partial (ie, region-targeted) lockdowns were associated with better public response (ie, higher number of tweets with net positive sentiment and lower levels of disagreement) compared to province-wide lockdowns.ConclusionsOur study demonstrates the feasibility of a rapid and flexible method of evaluating the public response to pandemic restrictions using near real-time social media data. This information can help public health practitioners and policy makers anticipate public response to future pandemic restrictions and ensure adequate resources are dedicated to addressing increases in negative sentiment and levels of disagreement in the face of scientifically informed, but controversial, restrictions.

Background Many informal caregivers of older adults have limited time because of the number of responsibilities that their caregiving role entails. This population often experiences high levels of burden due to the stressful nature of their work and are vulnerable to developing negative psychological health outcomes. Easily accessible and flexible knowledge interventions are needed to alleviate the burden and stress experienced by this group. Objective This study aims to evaluate the acceptability of the web-based delivery of the Caregiving Essentials course for informal caregivers of older adults. Both the strengths and limitations of using a web-based platform to provide information and resources were explored to see whether the method of delivery enhanced or hindered the overall course experience for participants. Methods A mixed methodology of web-based pre- (n=111) and postcourse surveys (n=39) and telephone interviews (n=26) was used to collect both qualitative and quantitative data from participants. Individual interviews were also conducted with key stakeholders (n=6), and a focus group was conducted with nursing students (n=5) who were involved in the project. Results The web-based delivery of the course provided participants with greater accessibility to the course because it allowed them to work independently through the modules at their own pace wherever and whenever. The discussion boards were also identified as a major strength because of the opportunity for social interaction and the sense of community that many felt through sharing their experiences. Some barriers to participation included age-related factors, issues with navigating aspects of the course, and concerns about privacy and anonymity. Some key suggestions included more engaging methods of web-based communication and the reorganization of the module content to reduce the amount of text and streamline information. Conclusions The web-based delivery of Caregiving Essentials appeared to enhance the overall course experience by increasing accessibility and allowing participants to interact with the learning materials and other caregivers. The findings from this evaluation can be used to create and improve the web-based delivery of both the current and emerging interventions for caregivers.

Background Natural language processing has long been applied in various applications for biomedical knowledge inference and discovery. Enrichment analysis based on named entity recognition is a classic application for inferring enriched associations in terms of specific biomedical entities such as gene, chemical, and mutation. Objective The aim of this study was to investigate the effect of pathway enrichment evaluation with respect to biomedical text-mining results and to develop a novel metric to quantify the effect. Methods Four biomedical text mining methods were selected to represent natural language processing methods on drug-related gene mining. Subsequently, a pathway enrichment experiment was performed by using the mined genes, and a series of inverse pathway frequency (IPF) metrics was proposed accordingly to evaluate the effect of pathway enrichment. Thereafter, 7 IPF metrics and traditional P value metrics were compared in simulation experiments to test the robustness of the proposed metrics. Results IPF metrics were evaluated in a case study of rapamycin-related gene set. By applying the best IPF metrics in a pathway enrichment simulation test, a novel discovery of drug efficacy of rapamycin for breast cancer was replicated from the data chosen prior to the year 2000. Our findings show the effectiveness of the best IPF metric in support of knowledge discovery in new drug use. Further, the mechanism underlying the drug-disease association was visualized by Cytoscape. Conclusions The results of this study suggest the effectiveness of the proposed IPF metrics in pathway enrichment evaluation as well as its application in drug use discovery.

Background: There is extensive literature on the methodology of evaluation research and the development and evaluation of complex interventions but little guidance on the formative stages before evaluation and how to work with partner organizations that wish to have their provision evaluated. It is important to be able to identify suitable projects for evaluation from a range of provision and describe the steps required, often with academic institutions working in partnership with external organizations, in order to set up an evaluation. However, research evaluating programs or interventions rarely discusses these stages. Objective: This study aimed to extend work on evaluability assessment and pre-evaluation planning by proposing an 8-Step Scoping Framework to enable the appraisal of multiple programs in order to identify interventions suitable for evaluation. We aimed to add to the literature on evaluability assessment and more recent evaluation guidance by describing the processes involved in working with partner organizations. Methods: This paper documents the steps required to identify multiple complex interventions suitable for process and outcome evaluation. The steps were developed using an iterative approach by working alongside staff in a local government organization, to build an evidence base to demonstrate which interventions improve children’s outcomes. The process of identifying suitable programs for evaluation, thereby establishing the pre-evaluation steps, was tested using all Flying Start provision. Results: The 8-Step Scoping Framework was described using the example of the local government organization Flying Start to illustrate how each step contributes to finding projects suitable for process and outcome evaluation: (1) formulating overarching key questions that encompass all programs offered by an organization, (2) gaining an in-depth understanding of the work and provision of an organization and engaging staff, (3) completing a data template per project/program offered, (4) assessing the robustness/validity of data across all programs, (5) deciding on projects suitable for evaluation and those requiring additional data, (6) negotiating with chosen project leads, both within and outside the organization, (7) developing individual project evaluation protocols, and (8) applying for ethical approval from the university and partner organization. Conclusions: This paper describes the processes involved in identifying suitable projects for evaluation. It adds to the existing literature on the assessment of specific programs suitable for evaluation and guidance for conducting evaluations by establishing the formative steps required to identify suitable programs from a range of provision. This scoping framework particularly relates to academic partners and organizations tasked with delivering evidence-based services designed to meet local needs. The steps identified have been described in the context of early years provision but can be applied to a range of community-based evaluations, or more generally, to cases where an academic partner is working with external stakeholders to identify projects suitable for academic evaluation.

Background The Neonatal Tetanus Surveillance System (NTSS) in Yemen was established in 2009 to identify high-risk areas, determine trends, and evaluate elimination activities. Since its launch, the NTSS had never been evaluated. Objective This study aimed to assess the performance of NTSS and determine its strengths and weaknesses to recommend improvements. Methods The US Centers for Disease Control and Prevention (CDC) guidelines were used for evaluating the NTSS. Stakeholders at the central, district, and facility levels were interviewed to rate the attributes of the NTSS. The percentage scores for attributes were ranked as poor ( Results The overall usefulness score percentage was 38%, which indicates a poor performance. The performance of the NTSS was rated as average on flexibility (score percent: 68%) and acceptability (score percent: 64%) attributes and poor on stability (score percentage: 33%), simplicity (score percentage: 57%), and representativeness (score percentage: 39%) attributes. About 65% of investigation forms were completed within 48 hours of notification date. Data quality was poor, as 41% of the core variables were missing. Conclusions The overall performance of the NTSS was poor. Most of the system attributes require improvement, including stability, simplicity, quality of data, and completeness of investigation. To improve the performance of NTSS, the following are recommended: capacity building of staff (focal points), strengthening NTSS through technical support and government funding to ensure its sustainability, establishing electronic investigation forms for improving the system data quality, and expansion of NTSS coverage to include all private health care facilities.

BackgroundHuman papillomavirus (HPV) is the most common sexually transmitted infection in the United States. HPV attributes to most cancers including anal, oral, cervical, and penile. Despite infection rates in the United States, recommendations and communication campaigns have traditionally focused on females. Because of this, males lack knowledge about reasons for vaccination, the benefits of being vaccinated, and their HPV risk, overall. Gaming as a health education strategy can be beneficial as mechanism that can promote behavior change for this key demographic because of the popularity of gaming.ObjectiveWe sought to explore the relationship between gamification and HPV vaccine uptake.MethodsInterviews were conducted with experts (n=22) in the fields of cancer prevention, sexual and reproductive health, public health, game design, technology, and health communication on how a game should be developed to increase HPV vaccination rates among males.ResultsOverwhelmingly, theoretical models such as the health belief model were identified with key constructs such as self-efficacy and risk perception. Experts also suggested using intervention mapping and logic models as planning tools for health promotion interventions utilizing a digital game as a medium. In game and out of game measures were discussed as assessments for quality and impact by our expert panel.ConclusionsThis study shows that interventions should focus on whether greater utilization of serious games, and the incorporation of theory and standardized methods, can encourage young men to get vaccinated and to complete the series of HPV vaccinations.

Background The Symptom Assessment and Management (SAM) program is a structured, online, nurse-supported intervention to support symptom self-management in people receiving adjuvant chemotherapy post surgery for breast or colorectal cancer. Objective The objective of this study was to describe the development, implementation strategy, and evaluation of the SAM system. Methods The development of the SAM program involved 3 phases. In phase 1, the web app was developed through consultation with consumers and clinicians and of the literature to ensure that the system was evidence-based and reflected the realities of receiving treatment and supporting patients through treatment. In phase 2, 7 participants recorded the severity of 6 symptoms daily over the course of 1 cycle of chemotherapy. In phase 3, 17 participants recorded their symptoms daily over the course of 3 cycles of chemotherapy. Once symptoms were recorded, participants received immediate feedback on the severity of their symptoms and self-management recommendations, which could include seeking immediate medical attention. Data on quality of life, symptom burden, anxiety and depression, distress, and self-efficacy were collected during treatment; participants’ perceptions of the SAM program were evaluated following participation via interview. Results The outcomes of the SAM project include the development of a system that is reliable and easy to use and navigate. Participants reported benefits related to using the SAM program that included feeling more in control of managing their symptoms and feeling reassured. Engagement with the system on a daily basis was variable, with some participants completing the symptom tracker daily and others engaging some of the time. The feedback from all participants was that the system was easy to navigate and the information was relevant and supportive. Conclusions The SAM program has the potential to enhance the management of symptoms for people receiving chemotherapy treatment. The system creates an accurate repository of symptoms that can be accessed easily and highlight patterns in symptom experience. These can be shared with clinicians, with patient permission, to inform and support treatment plans. The potential to predict the risk of developing severe symptoms can be developed to anticipate the need for care and support. Further considerations on how to increase engagement with the system, the value of the system for people diagnosed with other tumor types and treatment regimes, and the incorporation of the system into everyday clinical practice are needed.

Background Evidence-based practice refers to building clinical decisions on credible research evidence, professional experience, and patient preferences. However, there is a growing concern that evidence in the context of electronic health (eHealth) is not sufficiently used when forming policies and practice of health care. In this context, using evaluation and research evidence in clinical or policy decisions dominates the discourse. However, the use of additional types of evidence, such as professional experience, is underexplored. Moreover, there might be other ways of using evidence than in clinical or policy decisions. Objective This study aimed to analyze how different types of evidence (such as evaluation outcomes [including patient preferences], professional experiences, and existing scientific evidence from other research) obtained within the development and evaluation of an eHealth trial are used by diverse stakeholders. An additional aim was to identify barriers to the use of evidence and ways to support its use. Methods This study was built on a case of an eHealth trial funded by the European Union. The project included 4 care centers, 2 research and development companies that provided the web-based physical exercise program and an activity monitoring device, and 2 science institutions. The qualitative data collection included 9 semistructured interviews conducted 8 months after the evaluation was concluded. The data analysis concerned (1) activities and decisions that were made based on evidence after the project ended, (2) evidence used for those activities and decisions, (3) in what way the evidence was used, and (4) barriers to the use of evidence. Results Evidence generated from eHealth trials can be used by various stakeholders for decisions regarding clinical integration of eHealth solutions, policy making, scientific publishing, research funding applications, eHealth technology, and teaching. Evaluation evidence has less value than professional experiences to local decision making regarding eHealth integration into clinical practice. Professional experiences constitute the evidence that is valuable to the highest variety of activities and decisions in relation to eHealth trials. When using existing scientific evidence related to eHealth trials, it is important to consider contextual relevance, such as location or disease. To support the use of evidence, it is suggested to create possibilities for health care professionals to gain experience, assess a few rather than a large number of variables, and design for shorter iterative cycles of evaluation. Conclusions Initiatives to support and standardize evidence-based practice in the context of eHealth should consider the complexities in how the evidence is used in order to achieve better uptake of evidence in practice. However, one should be aware that the assumption of fact-based decision making in organizations is misleading. In order to create better chances that the evidence produced would be used, this should be addressed through the design of eHealth trials.

Background User-centered design (UCD) is a powerful framework for creating useful, easy-to-use, and satisfying mobile health (mHealth) apps. However, the literature seldom reports the practical challenges of implementing UCD, particularly in the field of mHealth. Objective This study aims to characterize the practical challenges encountered and propose strategies when implementing UCD for mHealth. Methods Our multidisciplinary team implemented a UCD process to design and evaluate a mobile app for older adults with heart failure. During and after this process, we documented the challenges the team encountered and the strategies they used or considered using to address those challenges. Results We identified 12 challenges, 3 about UCD as a whole and 9 across the UCD stages of formative research, design, and evaluation. Challenges included the timing of stakeholder involvement, overcoming designers’ assumptions, adapting methods to end users, and managing heterogeneity among stakeholders. To address these challenges, practical recommendations are provided to UCD researchers and practitioners. Conclusions UCD is a gold standard approach that is increasingly adopted for mHealth projects. Although UCD methods are well-described and easily accessible, practical challenges and strategies for implementing them are underreported. To improve the implementation of UCD for mHealth, we must tell and learn from these traditionally untold stories.

Background There is currently a lack of comprehensive, intuitive, and usable formative evaluation frameworks for health information technology (HIT) implementations. We therefore sought to develop and apply such a framework. This study describes the Technology, People, Organizations, and Macroenvironmental factors (TPOM) framework we developed. Objective The aim was to develop and apply a formative evaluation framework for HIT implementations, highlighting interrelationships between identified dimensions and offering guidance for implementers. Methods We drew on an initial prototype framework developed as part of a literature review exploring factors for the effective implementation of HIT. In addition, we used qualitative data from three national formative evaluations of different HIT interventions (electronic health record, electronic prescribing, and clinical decision support functionality). The combined data set comprised 19 case studies of primarily hospital settings, and included 703 semistructured interviews, 663 hours of observations, and 864 documents gathered from a range of care settings across National Health Service (NHS) England and NHS Scotland. Data analysis took place over a period of 10 years and was guided by a framework informed by the existing evidence base. Results TPOM dimensions are intimately related and each include a number of subthemes that evaluators need to consider. Although technological functionalities are crucial in getting an initiative off the ground, system design needs to be cognizant of the accompanying social and organizational transformations required to ensure that technologies deliver the desired value for a variety of stakeholders. Wider structural changes, characterized by shifting policy landscapes and markets, influence technologies and the ways they are used by organizations and staff. Conclusions The TPOM framework supports formative evaluations of HIT implementation and digitally enabled transformation efforts. There is now a need for prospective application of the TPOM framework to determine its value.

Background Clinical governance of medical mobile apps is challenging, and there is currently no standard method for assessing the quality of such apps. In 2018, the National Institute for Health and Care Excellence (NICE) developed a framework for assessing the required level of evidence for digital health technologies (DHTs), as determined by their clinical function. The framework can potentially be used to assess mobile apps, which are a subset of DHTs. To be used reliably in this context, the framework must allow unambiguous classification of an app’s clinical function. Objective The objective of this study was to determine whether mobile health apps could be reliably classified using the NICE evidence standards framework for DHTs. Methods We manually extracted app titles, screenshots, and content descriptions for all apps listed on the National Health Service (NHS) Apps Library website on July 12, 2019; none of the apps were downloaded. Using this information, 2 mobile health (mHealth) researchers independently classified each app to one of the 4 functional tiers (ie, 1, 2, 3a, and 3b) described in the NICE digital technologies evaluation framework. Coders also answered contextual questions from the framework to identify whether apps were deemed to be higher risk. Agreement between coders was assessed using Cohen κ statistic. Results In total, we assessed 76 apps from the NHS Apps Library. There was classification agreement for 42 apps. Of these, 0 apps were unanimously classified into Tier 1; 24, into Tier 2; 15, into Tier 3a; and 3, into Tier 3b. There was disagreement between coders in 34/76 cases (45%); interrater agreement was poor (Cohen κ=0.32, 95% CI 0.16-0.47). Further investigation of disagreements highlighted 5 main explanatory themes: apps that did not correspond to any tier, apps that corresponded to multiple tiers, ambiguous tier descriptions, ambiguous app descriptions, and coder error. Conclusions The current iteration of the NICE evidence standards framework for DHTs did not allow mHealth researchers to consistently and unambiguously classify digital health mobile apps listed on the NHS app library according to their functional tier.

Background As representatives of health information communication platforms accessed through mobile phones and mobile terminals, health-related WeChat public accounts (HWPAs) have a large consumer base in the Chinese-speaking world. However, there is still a lack of general understanding of the status quo of HWPAs and the quality of the articles they release. Objective The aims of this study were to assess the conformity of HWPAs to the Health on the Net Foundation Code of Conduct (HONcode) and to evaluate the suitability of articles disseminated by HWPAs. Methods The survey was conducted from April 23 to May 5, 2019. Based on the monthly (March 1-31, 2019) WeChat Index provided by Qingbo Big Data, the top 100 HWPAs were examined to evaluate their HONcode compliance. The first four articles published by each HWPA on the survey dates were selected as samples to evaluate their suitability. All materials were assessed by three raters. The materials were assessed using the HONcode checklist and the Suitability Assessment of Materials (SAM) score sheet. Data analysis was performed with SPSS version 17.0 (SPSS Inc, Chicago, IL, USA) and Excel version 2013 (Microsoft Inc, Washington DC, USA). Results A total of 93 HWPAs and 210 of their released articles were included in this study. For six of the eight principles, the 93 HWPAs nearly consistently did not meet the requirements of the HONcode. The HWPAs certified by Tencent Corporation (66/93, 71%) were generally slightly superior to those without such certification (27/93, 29%) in terms of compliance with HONcode principles. The mean SAM score for the 210 articles was 67.72 (SD 10.930), which indicated “adequate” suitability. There was no significant difference between the SAM scores of the articles published by certified and uncertified HWPAs (P=.07), except in the literacy requirements dimension (tdf=97=–2.418, P=.02). Conclusions The HWPAs had low HONcode conformity. Although the suitability of health information released by HWPAs was at a moderate level, there were still problems identified, such as difficulty in tracing information sources, excessive implicit advertisements, and irregular usage of charts. In addition, the low approval requirements of HWPAs were not conducive to improvement of their service quality.

Background The internet is a large source of health information and has the capacity to influence its users. However, the information found on the internet often lacks scientific rigor, as anyone may upload content. This factor is a cause of great concern to scientific societies, governments, and users. Objective The objective of our study was to investigate the information about the prevention of coronavirus disease 2019 (COVID-19) on the internet. Methods On February 29, 2020, we performed a Google search with the terms “Prevention coronavirus,” “Prevention COVID-19,” “Prevención coronavirus,” and “Prevención COVID-19”. A univariate analysis was performed to study the association between the type of authorship, country of publication, and recommendations to avoid COVID-19 according to the World Health Organization (WHO). Results In total, 80 weblinks were reviewed. Most of them were produced in the United States and Spain (n=58, 73%) by digital media sources and official public health organizations (n=60, 75%). The most mentioned WHO preventive measure was “wash your hands frequently” (n=65, 81%). A less frequent recommendation was to “stay home if you feel unwell” (n=26, 33%). The analysis by type of author (official public health organizations versus digital media) revealed significant differences regarding the recommendation to wear a mask when you are healthy only if caring for a person with suspected COVID-19 (odds ratio [OR] 4.39). According to the country of publication (Spain versus the United States), significant differences were detected regarding some recommendations such as “wash your hands frequently” (OR 9.82), “cover your mouth and nose with your bent elbow or tissue when you cough or sneeze” (OR 4.59), or “stay home if you feel unwell” (OR 0.31). Conclusions It is necessary to urge and promote the use of the websites of official public health organizations when seeking information on COVID-19 preventive measures on the internet. In this way, users will be able to obtain high-quality information more frequently, and such websites may improve their accessibility and positioning, given that search engines justify the positioning of links obtained in a search based on the frequency of access to them.

Background: Facebook is increasingly being used as part of mass media campaigns in public health, including the Make Healthy Normal (MHN) campaign in New South Wales, Australia. Therefore, it is important to understand what role Facebook can play in mass media campaigns and how best to use it to augment or amplify campaign effects. However, few studies have explored this. Objective: This study aimed to investigate usage of and engagement with the MHN Facebook page and to identify influential factors in driving engagement with the page. Methods: We examined both post-level and page-level analytic data from Facebook from the campaign’s launch in June 2015 to September 2017. For post-level data, we conducted a series of negative binomial regressions with four different outcome measures (likes, shares, comments, post consumers), including some characteristics of Facebook posts as predictors. We also conducted time series analyses to examine associations between page-level outcomes (new page likes or “fans” and number of engaged users) and different measures of exposure to the page (number of unique users reached and total count of impressions) and to television advertising. Results: Of the 392 posts reviewed, 20.7% (n=81) received a paid boost and 58.9% (n=231) were photo posts. We found that posts that received a paid boost reached significantly more users and subsequently received significantly more engagement than organic (unpaid) posts (P

BackgroundThe task of monitoring and managing the entire emergency department (ED) is becoming more important due to increasing pressure on the ED. Recently, dashboards have received the spotlight as health information technology to support these tasks. ObjectiveThis study aimed to describe the development of a real-time autonomous dashboard for the ED and to evaluate perspectives of clinical staff on its usability. MethodsWe developed a dashboard based on three principles—“anytime, anywhere, at a glance;” “minimal interruption to workflow;” and “protect patient privacy”—and 3 design features—“geographical layout,” “patient-level alert,” and “real-time summary data.” Items to evaluate the dashboard were selected based on the throughput factor of the conceptual model of ED crowding. Moreover, ED physicians and nurses were surveyed using the system usability scale (SUS) and situation awareness index as well as a questionnaire we created on the basis of the construct of the Situation Awareness Rating Technique. ResultsThe first version of the ED dashboard was successfully launched in 2013, and it has undergone 3 major revisions since then because of geographical changes in ED and modifications to improve usability. A total of 52 ED staff members participated in the survey. The average SUS score of the dashboard was 67.6 points, which indicates “OK-to-Good” usability. The participants also reported that the dashboard provided efficient “concentration support” (4.15 points), “complexity representation” (4.02 points), “variability representation” (3.96 points), “information quality” (3.94 points), and “familiarity” (3.94 points). However, the “division of attention” was rated at 2.25 points. ConclusionsWe developed a real-time autonomous ED dashboard and successfully used it for 5 years with good evaluation from users.

Background Despite the increasing amount of research on Web-based mental health interventions with proven efficacy, high attrition rates decrease their effectiveness. Continued process evaluations should be performed to maximize the target population’s engagement. Google Analytics has been used to evaluate various health-related Web-based programs and may also be useful for Web-based mental health programs. Objective The objective of our study was to evaluate WalkAlong.ca, a youth-oriented mental health web-portal, using Google Analytics to inform the improvement strategy for the platform and to demonstrate the use of Google Analytics as a tool for process evaluation of Web-based mental health interventions. Methods Google Analytics was used to monitor user activity during WalkAlong’s first year of operation (Nov 13, 2013-Nov 13, 2014). Selected Google Analytic variables were overall website engagement including pages visited per session, utilization rate of specific features, and user access mode and location. Results The results included data from 3076 users viewing 29,299 pages. Users spent less average time on Mindsteps (0 minute 35 seconds) and self-exercises (1 minute 08 seconds), which are important self-help tools, compared with that on the Screener tool (3 minutes 4 seconds). Of all visitors, 82.3% (4378/5318) were desktop users, followed by 12.7 % (677/5318) mobile phone and 5.0% (263/5318) tablet users. Both direct traffic (access via URL) and referrals by email had more than 7 pages viewed per session and longer than average time of 6 minutes per session. The majority of users (67%) accessed the platform from Canada. Conclusions Engagement and feature utilization rates are higher among people who receive personal invitations to visit the site. Low utilization rates with specific features offer a starting place for further exploration of users in order to identify the root cause. The data provided by Google Analytics, although informative, can be supplemented by other evaluation methods (ie, qualitative methods) in order to better determine the modifications required to improve user engagement. Google Analytics can play a vital role in highlighting the preferences of those using Web-based mental health tools.

Background: Advancements in mobile phone technology have led to the development of smartphones with the capability to run apps. The availability of a plethora of health- and fitness-related smartphone apps has the potential, both on a clinical and public health level, to facilitate healthy behavior change and weight management. However, current top-rated apps in this area have not been extensively evaluated in terms of scientific quality and behavioral theory evidence base. Objective: The purpose of this study was to evaluate the quality of the most popular dietary weight-loss smartphone apps on the commercial market using comprehensive quality assessment criteria, and to quantify the behavior change techniques (BCTs) incorporated. Methods: The top 200-rated Health & Fitness category apps from the free and paid sections of Google Play and iTunes App Store in Australia (n=800) were screened in August 2014. To be included in further analysis, an app had to focus on weight management, include a facility to record diet intake (self-monitoring), and be in English. One researcher downloaded and used the eligible apps thoroughly for 5 days and assessed the apps against quality assessment criteria which included the following domains: accountability, scientific coverage and content accuracy of information relevant to weight management, technology-enhanced features, usability, and incorporation of BCTs. For inter-rater reliability purposes, a second assessor provided ratings on 30% of the apps. The accuracy of app energy intake calculations was further investigated by comparison with results from a 3-day weighed food record (WFR). Results: Across the eligible apps reviewed (n=28), only 1 app (4%) received full marks for accountability. Overall, apps included an average of 5.1 (SD 2.3) out of 14 technology-enhanced features, and received a mean score of 13.5 (SD 3.7) out of 20 for usability. The majority of apps provided estimated energy requirements (24/28, 86%) and used a food database to calculate energy intake (21/28, 75%). When compared against the WFR, the mean absolute energy difference of apps which featured energy intake calculations (23/28, 82%) was 127 kJ (95% CI -45 to 299). An average of 6.3 (SD 3.7) of 26 BCTs were included. Conclusions: Overall, the most popular commercial apps for weight management are suboptimal in quality, given the inadequate scientific coverage and accuracy of weight-related information, and the relative absence of BCTs across the apps reviewed. With the limited regulatory oversight around the quality of these types of apps, this evaluation provides clinicians and consumers an informed view of the highest-quality apps in the current popular app pool appropriate for recommendation and uptake. Further research is necessary to assess the effectiveness of apps for weight management. [JMIR Mhealth Uhealth 2015;3(4):e104]

BackgroundMany prescription drugs are freely available for purchase on the Internet without a legitimate prescription from a physician. ObjectiveThis study focused on the motivations for using no-prescription online pharmacies (NPOPs) to purchase prescription drugs rather than using the traditional doctor-patient-pharmacy model. We also studied whether users of NPOP-purchased drugs had poorer health outcomes than those who obtain the same drug through legitimate health care channels. MethodsWe selected tramadol as a representative drug to address our objective because it is widely prescribed as an unscheduled opioid analgesic and can easily be purchased from NPOPs. Using search engine marketing (SEM), we placed advertisements on search result pages stemming from the keyword “tramadol” and related terms and phrases. Participants, who either used the traditional doctor-patient-pharmacy model to obtain tramadol (traditional users, n=349) or purchased it on the Web without a prescription from their local doctor (ie, nontraditional users, n=96), were then asked to complete an online survey. ResultsRespondents in both groups were primarily white, female, and in their mid-forties (nontraditional users) to upper forties (traditional users). Nearly all nontraditional users indicated that their tramadol use was motivated by a need to treat pain (95%, 91/96) that they perceived was not managed appropriately through legitimate health care channels. A majority of nontraditional users (55%, 41/75) indicated they used NPOPs because they did not have access to sufficient doses of tramadol to relieve pain. In addition, 29% (22/75) of nontraditional users indicated that the NPOPs were a far cheaper alternative than seeing a physician, paying for an office visit, and filling a prescription at a local pharmacy, which is often at noninsured rates for those who lack medical insurance (37%, 35/96, of NPOP users). The remainder of participants (16%, 12/96) cited other motivations (eg, anonymity) for using NPOPs. In terms of health outcomes, nontraditional users experienced a significantly (P

Background: In eHealth research, limited insights have been obtained on process outcomes or how the use of technology has contributed to the users’ ability to have a healthier life, improved wellbeing, or activate new attitudes in their daily tasks. As a result, eHealth is often perceived as a black box. To open this Black Box of eHealth, methodologies must extend beyond the classic effect evaluations. The analyses of log data (anonymous records of real-time actions performed by each user) can provide continuous and objective insights into the actual usage of the technology. However, until now the possibilities of log data in eHealth research has not been exploited to its fullest extent. Objectives: The aim of this paper is to describe how log data can be used to improve the evaluation and understand the use of an eHealth technology with a broader approach than only descriptive statistics. This paper serves as a starting point for using log data analysis in eHealth research. Methods: First, we describe what log data is and an overview is given of research questions to evaluate the system, the context, the users of a technology as well as the underpinning theoretical constructs. Secondly, requirements for log data, the starting points for the data preparation and methods for data collection are explained. Results: In the third part, some methods for data analysis are described. Finally, a conclusion is drawn regarding the importance of the results for both scientific and practical applications. Conclusion: The analysis of log data can be of great value for opening the black box of eHealth. A deliberate log data analysis can give new insights into how the usage of the technology contributed to the found effects and can thereby help to improve the persuasiveness and effectiveness of the eHealth technology and the underpinning behavioral models. Keywords: eHealth, black box, evaluation, log data analysis

Background Website evaluation is a key issue for researchers, organizations, and others responsible for designing, maintaining, endorsing, approving, and/or assessing the use and impact of interventions designed to influence health and health services. Traditionally, these evaluations have included elements such as content credibility, interface usability, and overall design aesthetics. With the emergence of collaborative, adaptive, and interactive ("Web 2.0") technologies such as wikis and other forms of social networking applications, these metrics may no longer be sufficient to adequately assess the quality, use or impact of a health website. Collaborative, adaptive, interactive applications support different ways for people to interact with health information on the Web, including the potential for increased user participation in the design, creation, and maintenance of such sites. Objective We propose a framework that addresses how to evaluate collaborative, adaptive, and interactive applications. Methods In this paper, we conducted a comprehensive review of a variety of databases using terminology related to this area. Results We present a review of evaluation frameworks and also propose a framework that incorporates collaborative, adaptive, and interactive technologies, grounded in evaluation theory. Conclusion This framework can be applied by researchers who wish to compare Web-based interventions, non-profit organizations, and clinical groups who aim to provide health information and support about a particular health concern via the Web, and decisions about funding grants by agencies interested in the role of social networks and collaborative, adaptive, and interactive technologies technologies to improve health and the health system.