Search

There is general consensus that teaching and learning centres have a positive impact on promoting and supporting a culture that values teaching and learning, yet there is limited evidence on how centres evaluate their work. For this purpose, a survey was developed and administered to the directors of 88 Canadian centres; 46 of which completed the survey. In this paper, we share centres' approaches to evaluation that emerged from the survey findings and the artifact analysis. By raising awareness of evaluation approaches, we are better positioned to identify further opportunities to develop a more systematic and coherent approach to evaluation.

Purpose: This paper aims to examine the roles of quality managers at community colleges, their experiences balancing accountability and improvement and their insights into the future of quality assurance. Design/methodology/approach: This phenomenological, qualitative study used semi-structured interviews with eight community college quality managers to investigate their roles, experiences and perspectives. A reflexive thematic approach was used to analyze the interview data. Findings: Four themes were identified from participant responses: quality managers frame and enable program quality, quality managers drive program change, quality managers cultivate a culture of quality and quality managers seek system change. The findings illustrate the roles played by quality managers as they work to improve college education at program, institution and system-wide levels. Research limitations/implications: The decision of participants to accept the recruitment invitation might reflect particular attitudes, perspectives or experiences. Practical implications: Quality assurance has emerged as a key mechanism for ensuring postsecondary programs are current, relevant and meeting the evolving needs of students and employers. This study advances the understanding of how quality assurance processes play out at the operational level and explores the experiences of quality managers as they navigate various quality tensions. Originality/value: Quality managers play key roles in leading, evaluating and influencing quality assurance processes in postsecondary education yet they are underrepresented in the literature. The findings of this study shed new light on the aspirational and influential roles they play in advancing quality assurance.

This introduction to the special feature describes the development and application of a "full-spectrum sustainability" evaluation framework that emerged from a transdisciplinary research process. The framework and corresponding case studies described in this paper originated in the work of a Canadian Fisheries Research Network project that sought to enhance fisheries management by including diverse social-ecological considerations in fisheries management evaluation. The first section discusses the tendency of sustainability evaluation frameworks in fisheries to focus on ecological and economic considerations and introduces the Canadian Fisheries Research Network's four pillar approach, which includes ecological, economic, social and cultural, and institutional/ governance categories. To illustrate the comprehensive nature of this framework, the second section provides a comparison of the framework with elements in the United Nation's Sustainable Development Goals. The third section provides an overview of the eight papers in this special feature that explore the development and application of full-spectrum sustainability. The conclusion synthesizes some key findings, highlighting four overall critical and ongoing challenges associated with advancing full-spectrum sustainability evaluation in Canada and elsewhere: the politics of transdisciplinary research; integrating social considerations into management agencies reluctant to move beyond ecological and economic considerations; dynamic and diverse issues involved in supporting robust and inclusive governance processes; and translating technical frameworks into usable practical instruments for different societal actors. [ABSTRACT FROM AUTHOR]

Bodywork describes the work people perform on their own and other people's bodies. Often, this work is performed in an effort to meet cultural, symbolic body norms. The aims of bodywork in fitness for older exercisers are said to serve as a means of controlling and resisting the aging process. Given the popularity of group exercise for older exercisers, this paper focused on three forms of bodywork undertaken by/performed on older exercisers in group classes: assessing, monitoring and directing clients' bodies. Using an Institutional Ethnography approach, I explicate how instructor training is connected to the bodywork practices that take place in group exercise classes. Methods consisted of a textual analysis of 8 Canadian and American instructor training curricula; observations and interviews with 22 group fitness instructors; and go-alongs and interviews with 14 older exercisers. Findings revealed that some topics included in the explicit curricula taught to fitness instructors lie outside their scope of practice. The hidden curricula uphold ageist sociocultural norms, as well as conflate (older) age, (greater) weight/(larger) size and bodily (dis)function, which are then materially codified in what is considered (age) appropriate fitness apparel. Finally, the null curriculum, or what is left out of the training, has the potential to cause harm, marginalize and stigmatize older exercisers. To conclude, I argue that group exercise instructors and the organizations that develop fitness curricula, could benefit from engaging in a reflective practice that investigates: (1) how some of the sociocultural and discursive representations of older bodies in training curricula might be problematic and stigmatize older people; and (2) how the null curricula might cause harm to exercisers, especially exercisers like older persons who are already marginalized.

Two experiments were conducted to assess an online version of Keller's personalized system of instruction, called computer-aided personalized system of instruction (CAPSI), as part of a blended learning design with regard to course knowledge and critical thinking development. In Experiment 1, two lecture sections of an introduction to University course received a CAPSI assignment while two received an extra paper assignment. In Experiment 2, one lecture section of an introduction to University course received a CAPSI assignment while another lecture section was assigned a research paper. In the two experiments the CAPSI sections consistently outperformed the sections with which they were compared, indicating that CAPSI is a viable option in higher education.

This article presents a response to the paper by Li, D'ngiulli and Kendall (2007). The authors address two key aspects of this paper. The first concerns a number of errors and misconceptions in the paper that the authors think are important to clarify and correct. The second issue relates to the significant amount of research and effort that has taken place since this article was first written, particularly the increasing amount of effort, both in Canada and in Australia, to engage the Indigenous/Aboriginal populations.

Comparing group is one of the key uses of large-scale assessment results, which are used to gain insights to inform policy and practice and to examine the comparability of scores and score meaning. Such comparisons typically focus on examinees' final answers and responses to test questions, ignoring response process differences groups may engage in. This paper discusses and demonstrates the use of response process data in enhancing group comparison and fairness research methodologies. We propose two statistical approaches for identifying differential response processes which extend the differential item functioning (DIF) detection methods and demonstrate the complementary use of process data in comparing groups in two case studies. Our findings demonstrate the use of response process data in gaining insights about students' test-taking behaviors from different populations that go beyond what may be identified using response data only.

This paper investigated teachers' and educational consultants' perceptions of ICT integration in Québec English Schools, specifically with regards to the benefits and challenges of ICT integration therein. 23 teachers and educational consultants from seven different school boards participated in the focus group sessions. Results revealed higher student engagement levels, glocalization of the 21st century education and enhancement of the learning process as the main benefits of integrating ICT in English Québec Schools. In addition, participants highlighted the following challenges: lack of supporting school leadership, inconsistent investments in ICT equipment, infrastructure and resources, inflexibility of funding, the need for additional professional development and support and incorporation of technology in evaluations and curricular plans.

Purpose: The purpose of this paper is to evaluate whether unstructured graduate student research internships conducted in collaboration with community agencies build capacity and knowledge for students and community. Design/methodology/approach: The paper reports the results of four semi-structured interviews and 20 pre- and post-internship surveys of students' perceptions of their internship activities; whether participation built research capacity in students and community resulted in the creation of new knowledge and promoted ongoing partnerships and relationships. Findings: Students reported generating concrete outcomes for community partners, the acquisition of new research and professional skills, plus an increased understanding of theoretical knowledge. Many students also maintained ongoing relationships with their organizational partners beyond the terms of their internship. Research limitations/implications: Limitations to this study are the relatively small sample size and reliance on self-report measures. Practical implications: The paper describes a model for student-community engagement that benefits both community and students. Social implications: As universities explore their relationships with their local communities, graduate student internships have tremendous potential for supporting research and knowledge-based needs of local communities, while providing valuable skills and training to a cohort of students in bridging academic research to real world solutions. These students may go on to be community engaged scholars, or research trained personnel in the community. Originality/value: The results presented in this paper demonstrate the benefits to graduate students in scholarship of engagement programs that prioritize true partnership between students, universities and communities. (Contains 2 notes and 2 tables.)

Since 2004, a number of Canadian provinces have initiated comprehensive reviews of their respective public post-secondary education systems. This paper examines the ways in which these provincial post-secondary education reviews are consistent with the pervasive influence of economic globalization on higher education and a more market-driven and commercially-oriented ideological outlook on post-secondary education's raison d'etre. Taken together, these provincial reviews provide an informative and interesting repository of the current tendencies in Canadian post-secondary education policy. (Contains 1 table and 2 footnotes.)

Purpose: Variations in customer expectations pose a challenge to service quality improvement in the airline industry. Understanding airline customers' expectations and satisfaction help service providers improve their offerings. The extant literature examines airline passengers' expectations in isolation, neglecting the overall impact of online reviews on service quality improvement. This paper systematically evaluates the airline industry's passengers' expectations and satisfaction using expectation confirmation theory (ECT) and the SERVQUAL framework. The paper analyzes online reviews to examine the relationship between airline service quality attributes and passengers' satisfaction. Design/methodology/approach: The SERVQUAL framework was employed to examine the effects of customer culture, the reason for traveling, and seat type on customer's expectations and satisfaction across a large sample of airline customers. Findings: A total of 17,726 observations were gathered from the Skytrax review website. The lowest satisfaction ratings were from passengers from the USA, Canada and India. Factors that affect perceived service performance include customer service, delays and baggage management. Empathy and reliability have the biggest impact on the perceived satisfaction of passengers. Research limitations/implications: This research increases understanding of the consumer expectations through analysis of passengers' online reviews. Results are limited to a small sample of airline industries. Practical implications: This study provides airlines with valuable information to improve customer service by analyzing online reviews. Social implications: This study provides the opportunity for airline customers to gain better services when airline companies utilize the findings. Originality/value: This paper offers insights into passengers' expectations and their perceived value for money in relation to seat types. Previous studies have not investigated value for money as a construct for passengers' expectations and satisfaction relative to service quality dimensions. This paper addresses this need. [ABSTRACT FROM AUTHOR]

This paper draws on international evidence on medical spending to examine what the United States can learn about making its healthcare system more efficient. We focus primarily on understanding contemporaneous differences in the level of spending, generally from the 2000s. Medical spending differs across countries either because the price of services differs (for example, a coronary bypass surgery operation may cost more in the United States than in other countries) or because people receive more services in some countries than in others (for example, more bypass surgery operations). Within the price category, there are two further issues: whether factors earn different returns across countries and whether more clinical or administrative personnel are required to deliver the same care in different countries. We first present the results of a decomposition of healthcare spending along these lines in the United States and in Canada. We then delve into each component in more detail--administrative costs, factor prices, and the provision of care received--bringing in a broader range of international evidence when possible. Finally, we touch upon the organization of primary and chronic disease care and discuss possible gains in that area. [ABSTRACT FROM AUTHOR]

Copyright of Canadian Journal of Program Evaluation is the property of University of Toronto Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Summative and benchmarking surveys to measure the postgraduate student research experience are well reported in the literature. While useful, we argue that local instruments that provide formative resources with an academic development focus are also required. If higher education institutions are to move beyond the identification of issues and benchmarking practices, the scope of survey results and their reporting need to enable and foster appropriate changes in disciplinary practices. Robust, locally developed instruments can provide detailed, programme-specific information and foster timely changes in practice with direct benefits for postgraduate respondents. Unlike benchmarked surveys, local tools can adapt to explore and examine specific concerns of students, supervisors and academic developers. Coupling high-response rates and follow-on engagement with participant feedback, well-designed local instruments provide clear and irrefutable indicators to programme and university administrators of specific disciplinary strengths and weaknesses in postgraduate pathways. In this paper, we discuss the development of a research student survey specifically designed to support academic development purposes in strengthening and enhancing the postgraduate experience.

Despite the emerging body of literature on the benefits of youth peer support, there is also evidence that peer support can have unintended negative impacts on peers themselves. It is important to explore what aspects of the peer role contribute to these difficulties in order to mitigate risks. This paper uses a participatory approach to examine the unique attributes of youth peer practice and the related challenges. We conducted semi-structured interviews and focus groups with both peer and non-peer staff from a community-based youth mental health program that provides peer support services (N = 29). Thematic analyses were completed using QSR NVivo. Analyses capture the defining features and related challenges of the peer support role (self-disclosure, boundaries, role confusion and dynamic recovery), and risk factors that affect peers (stigma, exposure to harm and burnout). This paper contributes to the literature on peer support as well as youth participatory evaluation. The findings will be useful to support the development of improved organizational contexts for peer practice and more effective peer support programming. [ABSTRACT FROM AUTHOR]

Evaluation is an essential organizational practice in sport, but many organizations do not have adequate capacity to engage in evaluative work. To address this gap, academic researchers partnered with Canada's Sport Information Resource Centre, a nationally serving nonprofit dedicated to knowledge translation in sport, to develop, deliver and evaluate a series of webinars and knowledge products (e.g. blog posts, videos) that aimed to build evaluation capacity among sport organizations. The initiative produced four webinars and 16 knowledge products that reached 753 sport stakeholders, with 86% of survey respondents reporting an increase in evaluation knowledge. Using the Knowledge to Action and RE-AIM frameworks, this paper provides a blueprint for higher education professionals seeking to co-develop, co-deliver and co-evaluate knowledge translation initiatives in partnership with nonprofit sport organizations in Canada. [ABSTRACT FROM AUTHOR]

Issues: Drug consumptions rooms (DCR) and supervised injecting facilities (SIF) are expanding internationally. Previous reviews have not systematically addressed evaluation methodologies.Approach: Results from systematic searches of scientific databases in English until June 2017 were coded for paper type, country and year of publication. For evaluation papers, study outcome, methodology/study design and main indicators of DCR/SIF 'exposure' were recorded.Key Findings: Two hundred and nineteen eligible peer-reviewed papers were published since 1999: the majority from Canada (n = 117 papers), Europe (n = 36) and Australia (n = 32). Fifty-six papers reported evaluation outcomes. Ecological study designs (n = 10) were used to assess the impact on overdose, public nuisance and crime; modelling techniques (n = 6) estimated impact on blood-borne diseases, overdose deaths and costs. Papers using individual-level data included four prospective cohorts (n = 28), cross-sectional surveys (n = 7) and service records (n = 5). Individual-level data were used to assess safer injecting practice, uptake into health and social services and all the other above outcomes except for impact on crime and costs. Four different indicators of DCR/SIF attendance were used to measure service 'exposure'.Implications: Research around DCRs/SIFs has used ecological, modelling, cross-sectional and cohort study designs. Further research could involve systematic inclusion of a control group of people who are eligible but do not access SIFs, validation of self-reported proportion of injections at SIFs or a stepped-wedge or a cluster trial comparing localities.Conclusions: Methodologies appropriate for DCR/SIF evaluation have been established and can be readily replicated from the existing literature. Research on operational aspects, implementation and transferability is also warranted. [ABSTRACT FROM AUTHOR]

Assessing the comparability of different groups is an issue facing many researchers and evaluators in a variety of settings. Commonly, null hypothesis significance testing (NHST) is incorrectly used to demonstrate comparability when a non-significant result is found. This is problematic because a failure to find a difference between groups is not equivalent to showing that the groups are comparable. This paper provides a comparison of the confidence interval approach to equivalency testing and the more traditional analysis of variance (ANOVA) method using both continuous and rating scale data from three geographically separate medical education teaching sites. Equivalency testing is recommended as a better alternative to demonstrating comparability through its examination of whether mean differences between two groups are small enough that these differences can be considered practically unimportant and thus, the groups can be treated as equivalent. (Contains 5 tables and 2 footnotes.)

Many assessment and evaluation studies use statistical hypothesis tests, such as the independent samples t test or analysis of variance, to test the equality of two or more means for gender, age groups, cultures or language group comparisons. In addition, some, but far fewer, studies compare variability across these same groups or research conditions. Tests of the equality of variances can therefore be used on their own for this purpose but they are most often used alongside other methods to support assumptions made about variances. This is often done so that variances can be pooled across groups to yield an estimate of variance that is used in the standard error of the statistic in question. The purposes of this paper are twofold. The first purpose is to describe a new nonparametric Levene test for equal variances that can be used with widely available statistical software such as SPSS or SAS, and the second purpose is to investigate this test's operating characteristics, Type I error and statistical power, with real assessment and evaluation data. To date, the operating characteristics of the nonparametric Levene test have been studied with mathematical distributions in computer experiments and, although that information is valuable, this study will be an important next step in documenting both the level of non-normality (skewness and kurtosis) of real assessment and evaluation data, and how this new statistical test operates in these conditions. (Contains 1 figure, 4 tables, and 1 footnote.)

Purpose: The dynamics of organizational change related to environmental sustainability on university campuses are examined in this article. Whereas case studies of campus sustainability efforts tend to classify leadership as either "top-down" or "bottom-up", this classification neglects consideration of the leadership roles of the institutional "middle"--namely the faculty and staff. Design/methodology/approach: The authors draw from research conducted on sustainability initiatives at the University of Guelph combined with a review of faculty and staff-led initiatives at universities across Canada and the USA, as well as literature on best practices involving campus sustainability. Using concepts developed in business and leadership literature, faculty and staff are shown to be universities' equivalent to social "intrapreneurs", i.e. those who work for social and environmental good from within large organizations. Findings: Faculty and staff members are found to be critical leaders in efforts to achieve lasting progress towards campus sustainability, and conventional portrayals of campus sustainability initiatives often obscure this. Greater attention to the potential of faculty and staff leadership and how to effectively support their efforts is needed. Originality/value: In the paper, a case is made for emphasizing faculty and staff leadership in campus sustainability efforts and several successful strategies for overcoming barriers are presented. (Contains 4 tables and 1 figure.)

This paper demonstrates and discusses the use of think aloud protocols (TAPs) as an approach for examining and confirming sources of differential item functioning (DIF). The TAPs are used to investigate to what extent surface characteristics of the items that are identified by expert reviews as sources of DIF are supported by empirical evidence from examinee thinking processes in the English and French versions of a Canadian national assessment. In this research, the TAPs confirmed sources of DIF identified by expert reviews for 10 out of 20 DIF items. The moderate agreement between TAPs and expert reviews indicates that evidence from expert reviews cannot be considered sufficient in deciding whether DIF items are biased and such judgments need to include evidence from examinee thinking processes.

The new institutionalism predicts that professionalism is a key element of organizations' ability to be seen as legitimate. Emphasizing the professionalism and formal credentials of its members lends legitimacy to the organization, protecting it from scrutiny. What happens when this norm of professionalism is absent? How do schools legitimate themselves, if not through professionalism? This paper examines a population of small, secular non-elite private schools that overwhelmingly hire uncertified teachers. Using data from 60 private school principals in Toronto, Canada, I examine the ways in which private schools tap into alternate means of legitimacy. This study finds that small, secular "rogue" private schools fail to invoke norms of professionalism as a means to garner constituent support and legitimacy. I argue that these schools substitute an innovative, unconventional "caring consumer ethos" in place of teacher professionalism. (Contains 4 notes.)

The article focuses on article, published by Nousiainen and colleagues, which evaluates surgical fellowship training in Canada. It evaluates the kind of guidance which is available from the Royal College of Physicians and Surgeons of Canada. It puts emphasis on several concepts and questions which papers by Nousiainen and colleagues have brought forward for detailed examinations.

Background: The purpose of this paper is to provide a preliminary evaluation of treatment outcomes, retention and client satisfaction following a 12-week combined cognitive behavioural therapy (CBT) and motivational enhancement therapy (MET) group treatment for cannabis use disorder (CUD) delivered in an outpatient setting. Implementation of the program is also described. Methods: A retrospective observational cohort study was conducted using data collected from medical records and self-report assessments. Participants were treatment-seeking cannabis users at the Centre for Addiction and Mental Health, Toronto. Cannabis use, cannabis-related problems, craving, withdrawal symptoms, self-efficacy for remaining abstinent, depression and anxiety were assessed pre- and post-treatment. Treatment retention was calculated by inspecting clinic attendance records, and client satisfaction was evaluated using an anonymous feedback survey. Potential predictors of treatment outcomes and retention were investigated in exploratory analyses. Results: Cannabis use was lower and days of abstinence higher post-treatment (vs pre-treatment). Post-treatment improvements in cannabis-related problems, craving, withdrawal symptoms, self-efficacy and mood were also observed. Completion of group treatment (⩾75% of sessions attended) was 57% and moderate levels of treatment satisfaction were reported. Conclusions: This study provides preliminary evidence that a 12-week combined CBT and MET treatment for cannabis use disorder delivered in a novel group setting improves cannabis use outcomes. Potential predictors of reduced cannabis use and retention were identified. Future controlled studies are warranted, and strategies for increasing retention should be explored. [ABSTRACT FROM AUTHOR]

Background: Social Frailty is a significant public health concern affecting the elderly, particularly with the global population aging rapidly. Older adults with social frailty are at significantly higher risk of adverse outcomes such as disability, cognitive impairment, depression, and even death. In recent years, there have been more and more studies on social frailty, but no bibliometrics has been used to analyze and understand the general situation in this field. Therefore, by using CiteSpace, VOSviewer, and Bilioshiny software programs, this study aims to analyze the general situation of the research on social frailties of the older adults and determine the research trends and hot spots. Methods: A bibliometric analysis was conducted by searching relevant literature on the social frailty of the older adults from 2003 to 2022 in the Web of Science core database, using visualization software to map publication volume, country and author cooperation networks, keyword co-occurrences, and word emergence. Results: We analyzed 415 articles from 2003 to 2022. Brazil has the highest number of articles in the field of social frailty of the older adults, and the United States has the highest number of cooperative publications. Andrew MK, from Canada, is the most published and co-cited author, with primary research interests in geriatric assessment, epidemiology, and public health. "Social Vulnerability," "Health," "Frailty," "Mortality," and "Older Adult" are among the research hotspots in this field. "Dementia," "Alzheimer's disease," "Population," and "Covid-19" are emerging research trends in social frailty among the older adults. Conclusion: This scientometric study maps the research hotspots and trends for the past 20 years in social frailty among the older adults. Our findings will enable researchers to better understand trends in this field and find suitable directions and partners for future research. [ABSTRACT FROM AUTHOR]

Purpose: This paper aims to explore users' perceptions of whether the Leadership Development Impact Assessment (LDI) Toolkit is valid, reliable, simple to use and cost-effective as a guide to its quality improvement. Design/methodology/approach: The Canadian Health Leadership Network codesigned and codeveloped the LDI Toolkit as a theory-driven and evidence-informed resource that aims to assist health-care organizational development practitioners to evaluate various programs at five levels of impact: reaction, learning, application, impact and return on investment (ROI) and intangible benefits. A comparative evaluative case study was conducted using online questionnaires and semistructured telephone interviews with three health organizations where robust leadership development programs were in place. A total of seven leadership consultants and specialists participated from three Canadian provinces. Data were analyzed sequentially in two stages involving descriptive statistical analysis augmented with a qualitative content analysis of key themes. Findings: Users perceived the toolkit as cost-effective in terms of direct costs, indirect costs and intangibles; they found it easy-to-use in terms of clarity, logic and structure, ease of navigation with a coherent layout; and they assessed the sources of the evidence-informed tools and guides as appropriate. Users rated the toolkit highly on their perceptions of its validity and reliability. The analysis also informed the refinement of the toolkit. Originality/value: The refined LDI Toolkit is a comprehensive online collection of various tools to support health organizations to evaluate the leadership development investments effectively and efficiently at five impact levels including ROI. [ABSTRACT FROM AUTHOR]

Background: Adequate housing is a basic human right. The many millions of people experiencing homelessness (PEH) have a lower life expectancy and more physical and mental health problems. Practical and effective interventions to provide appropriate housing are a public health priority. Objectives: To summarise the best available evidence relating to the components of case‐management interventions for PEH via a mixed methods review that explored both the effectiveness of interventions and factors that may influence its impact. Search Methods: We searched 10 bibliographic databases from 1990 to March 2021. We also included studies from Campbell Collaboration Evidence and Gap Maps and searched 28 web sites. Reference lists of included papers and systematic reviews were examined and experts contacted for additional studies. Selection Criteria: We included all randomised and non‐randomised study designs exploring case management interventions where a comparison group was used. The primary outcome of interest was homelessness. Secondary outcomes included health, wellbeing, employment and costs. We also included all studies where data were collected on views and experiences that may impact on implementation. Data Collection and Analysis: We assessed risk of bias using tools developed by the Campbell Collaboration. We conducted meta‐analyses of the intervention studies where possible and carried out a framework synthesis of a set of implementation studies identified by purposive sampling to represent the most 'rich' and 'thick' data. Main Results: We included 64 intervention studies and 41 implementation studies. The evidence base was dominated by studies from the USA and Canada. Participants were largely (though not exclusively) people who were literally homeless, that is, living on the streets or in shelters, and who had additional support needs. Many studies were assessed as having a medium or high risk of bias. However, there was some consistency in outcomes across studies that improved confidence in the main findings. Case Management and Housing Outcomes: Case management of any description was superior to usual care for homelessness outcomes (standardised mean difference [SMD] = −0.51 [95% confidence interval [CI]: −0.71, −0.30]; p < 0.01). For studies included in the meta‐analyses, Housing First had the largest observed impact, followed by Assertive Community Treatment, Critical Time Intervention and Intensive Case Management. The only statistically significant difference was between Housing First and Intensive Case Management (SMD = −0.6 [–1.1, −0.1]; p = 0.03) at ≥12 months. There was not enough evidence to compare the above approaches with standard case management within the meta‐analyses. A narrative comparison across all studies was inconclusive, though suggestive of a trend in favour of more intensive approaches. Case Management and Mental Health Outcomes: The overall evidence suggested that case management of any description was not more or less effective compared to usual care for an individual's mental health (SMD = 0.02 [−0.15, 0.18]; p = 0.817). Case Management and Other Outcomes: Based on meta‐analyses, case management was superior to usual care for capability and wellbeing outcomes up to 1 year (an improvement of around one‐third of an SMD; p < 0.01) but was not statistically significantly different for substance use outcomes, physical health, and employment. Case Management Components: For homelessness outcomes, there was a non‐significant trend for benefits to be greater in the medium term (≤3 years) compared to long term (>3 years) (SMD = −0.64 [−1.04, −0.24] vs. −0.27 [−0.53, 0]; p = 0.16) and for in‐person meetings in comparison to mixed (in‐person and remote) approaches (SMD = −0.73 [−1.25,−0.21]) versus −0.26 [−0.5,−0.02]; p = 0.13). There was no evidence from meta‐analyses to suggest that an individual case manager led to better outcomes then a team, and interventions with no dedicated case manager may have better outcomes than those with a named case manager (SMD = −0.36 [−0.55, −0.18] vs. −1.00 [−2.00, 0.00]; p = 0.02). There was not enough evidence from meta‐analysis to assess whether the case manager should have a professional qualification, or if frequency of contact, case manager availability or conditionality (barriers due to conditions attached to service provision) influenced outcomes. However, the main theme from implementation studies concerned barriers where conditions were attached to services. Characteristics of Persons Experiencing Homelessness: No conclusions could be drawn from meta‐analysis other than a trend for greater reductions in homelessness for persons with high complexity of need (two or more support needs in addition to homelessness) as compared to those with medium complexity of need (one additional support need); effect sizes were SMD = −0.61 [−0.91, −0.31] versus −0.36 [−0.68, −0.05]; p = 0.3. The Broader Context of Delivery of Case Management Programmes: Other major themes from the implementation studies included the importance of interagency partnership; provision for non‐housing support and training needs of PEH (such as independent living skills), intensive community support following the move to new housing; emotional support and training needs of case managers; and an emphasis on housing safety, security and choice. Cost Effectiveness: The 12 studies with cost data provided contrasting results and no clear conclusions. Some case management costs may be largely off‐set by reductions in the use of other services. Cost estimates from three North American studies were $45–52 for each additional day housed. Authors' Conclusions: Case management interventions improve housing outcomes for PEH with one or more additional support needs, with more intense interventions leading to greater benefits. Those with greater support needs may gain greater benefit. There is also evidence for improvements to capabilities and wellbeing. Current approaches do not appear to lead to mental health benefits. In terms of case management components, there is evidence in support of a team approach and in‐person meetings and, from the implementation evidence, that conditions associated with service provision should be minimised. The approach within Housing First could explain the finding that overall benefits may be greater than for other types of case management. Four of its principles were identified as key themes within the implementation studies: No conditionality, offer choice, provide an individualised approach and support community building. Recommendations for further research include an expansion of the research base outside North America and further exploration of case management components and intervention cost‐effectiveness. [ABSTRACT FROM AUTHOR]

Background: In order to meet the challenges presented by increasing demand and scarcity of resources, healthcare organizations are faced with difficult decisions related to resource allocation. Tools to facilitate evaluation and improvement of these processes could enable greater transparency and more optimal distribution of resources. Methods: The Resource Allocation Performance Assessment Tool (RAPAT) was implemented in a healthcare organization in British Columbia, Canada. Recommendations for improvement were delivered, and a follow up evaluation exercise was conducted to assess the trajectory of the organization's priority setting and resource allocation (PSRA) process 2 years post the original evaluation. Results: Implementation of RAPAT in the pilot organization identified strengths and weaknesses of the organization's PSRA process at the time of the original evaluation. Strengths included the use of criteria and evidence, an ability to reallocate resources, and the involvement of frontline staff in the process. Weaknesses included training, communication, and lack of program budgeting. Although the follow up revealed a regression from a more formal PSRA process, a legacy of explicit resource allocation was reported to be providing ongoing benefit for the organization. Conclusion: While past studies have taken a cross-sectional approach, this paper introduces the first longitudinal evaluation of PSRA in a healthcare organization. By including the strengths, weaknesses, and evolution of one organization's journey, the authors' intend that this paper will assist other healthcare leaders in meeting the challenges of allocating scarce resources. [ABSTRACT FROM AUTHOR]

This paper describes how a study using institutional ethnography (IE) was used as a decolonizing method of inquiry in a rural Indigenous community in Canada. IE honors lived experience, reveals institutional and colonial practices, provides clear empirical evidence, and can offer clear recommendations that can benefit Indigenous communities. At the heart of decolonizing research is the task of shifting whose knowledge is privileged – from those with power (often researchers) to those who are being researched (those subject to the effects of colonization). To highlight how IE can be used as a decolonizing method of inquiry, the authors of this paper (a) point out common pitfalls of academic research and knowledge translation (KT) practices in Indigenous health; (b) highlight decolonizing research principles and how IE can be a decolonizing method of inquiry; and (c) share an example to illustrate how IE was used in a decolonizing health study in a First Nations community context. This paper also outlines critiques of mainstream research and KT practices, highlights principles for conducting research with Indigenous people in Canada, and further discusses how IE is well positioned to facilitate both decolonizing research and strategic KT. [ABSTRACT FROM AUTHOR]

This paper describes the findings of a quantitative study of 1,698 public library websites in Australia, Canada, and the United States over a period of three years using a spreadsheet protocol. The purpose of the research was to evaluate public library websites, available online sources, and whether library staff were available to respond to users' questions and concerns regarding the website. Descriptive statistics are used to report the results. The study provides public library website information regarding which protocol criteria each country's libraries attained. [ABSTRACT FROM AUTHOR]

Discusses functions of the Alberta (Canada) Computer Technology Project's courseware clearinghouse, reviews findings on instructional software quality, identifies software development trends, and discusses need for support systems to facilitate the incorporation of computer assisted instruction in Canadian schools. (MBR)

An example is provided of an evaluation in which the formulation of explicit decision rules was a central activity. The case for and against the use of decision rules in program evaluation is considered, and the appropriate context identified. Implications for the rights of decision makers are addressed. (Author/CTM)

Reviews research results and arguments supporting Keller's Personalized System of Instruction (PSI) as a paradigm for distance learning, describes a project at Athabasca University which compared various PSI configurations for distance learning, and discusses the advantages, disadvantages, and flexibility of PSI for such use. Nineteen references are listed. (EAO)

Background: There are several well-known risk factor monitoring systems, but few examples of comprehensive surveillance systems designed specifically to inform physical activity (PA) policy. This paper examines the utility of Canada`s Physical Activity and Sport Monitoring System in guiding policy and practice. Methods: Indicators were determined in conjunction with government, nongovernmental associations and academics. Serial measures were collected from representative population (telephone interviews, n = 4000 to 11,000) and setting-based (postal surveys, n = 1425 to 4304) surveys. Results: Adult PA was higher in 2014 (47%) than 1998 (37%). The prevalence of knowledge about sufficient PA to meet national guidelines increased (31% to 57%). Most adults (66%) reported having many safe places to walk locally. Having policies to encourage walking and cycling when redeveloping communities increased by community size (5% to 37%). PA promotion was available in 10% to 15% of workplaces. Most parents (64%) provided transportation to support their child's PA. The prevalence of policies mandating daily PE increased 2001 to 2011 (36% to 55%), as did having no policy to hire qualified PE teachers (25% to 34%). Conclusions: Canada's surveillance system has provided information for guiding policy planning, resource allocation, setting and tracking national goals, assessing changes in PA determinants, and evaluating national campaigns, naturally occurring experiments, and innovative policies. [ABSTRACT FROM AUTHOR]

In the second paper of this two part series on Key Elements of Interprofessional Education (IPE), we highlight factors for success in IPE based on a systematic literature review conducted for Health Canada in its “Interprofessional Education for Patient Centred Practice” (IECPCP) initiative in Canada (Oandasan et al., 2004). The paper initially discusses micro (individual level) meso (institutional/organizational level) and macro (socio-cultural and political level) factors that can influence the success of an IPE initiative. The discussion provides the infrastructure for the introduction of a proposed framework for educators to utilize in the planning and implementation of an IPE program to enhance a learner's opportunity to become a collaborative practitioner. The paper also discusses key issues related to the evaluation of IPE and its varied outcomes. Lastly, it gives the reader suggestions of outcome measurements that can be used within the proposed IPE framework. [ABSTRACT FROM AUTHOR]

For 20 years Active Healthy Kids Canada (AHKC) has worked to inspire the country to engage all children and youth in physical activity (PA). The primary vehicle to achieve this is the AHKC Report Card on Physical Activity for Children and Youth, which has been released annually since 2005. Using 10 years of experience with this knowledge translation and synthesis mechanism, this paper aggregates and consolidates diverse evidence demonstrating the impact of the Report Card and related knowledge translation activities. Over the years many evaluations, consultations, assessments, and surveys have helped inform changes in the Report Card to improve its impact. Guided by a logic model, the various assessments have traversed areas related to distribution and reach, meeting stakeholder needs, use of the Report Card, its influence on policy, and advancing the mission of AHKC. In the past 10 years, the Report Card has achieved > 1 billion media impressions, distributed > 120,000 printed copies and > 200,000 electronic copies, and benefited from a collective ad value > $10 million. The Report Card has been replicated in 14 countries, 2 provinces, 1 state and 1 city. AHKC has received consistent positive feedback from stakeholders and end-users, who reported that the Report Card has been used for public awareness/education campaigns and advocacy strategies, to strengthen partnerships, to inform research and program design, and to advance and adjust policies and strategies. Collectively, the evidence suggests that the Report Card has been successful at powering the movement to get kids moving, and in achieving demonstrable success on immediate and intermediate outcomes, although the long-term goal of improving the PA of Canadian children and youth remains to be realized. [ABSTRACT FROM AUTHOR]

Best practice in mass reach physical activity campaigns includes a role for communities to support the initiative with sustained programs and shorter-term events. The purpose of this paper was to report on an outcome evaluation of ParticipACTION's 150 Play List, a population-level, year-long, national mass reach campaign that included community events. Participants (N = 1,185) were recruited in the last month of the program to complete a questionnaire measuring demographic information, leisure-time physical activity, campaign awareness, attitudes, intentions, and behavioral trialing. Data were also collected, measured using cell phone proximity, on the number of people who attended 150 Play List events, which ranged from large-scale (e.g. national sports events) to smaller community events. Approximately 43% of respondents were aware of the 150 Play List and 19.5% reported participating in some way (e.g. visiting the website). Almost half of the participants who were aware of the campaign reported increased sport or physical activity-related intentions. Among those who participated, 90.6% reported trying at least one physical activity or sport related behavior as a result of the 150 Play List, whereas only 27.5% (n = 75) of those who were aware but did not participate in the 150 Play List tried a behavior. Event attendance goals were mostly met or exceeded. The 150 Play List was valued by those aware of it and the campaign was related to interest in sport and physical activity in Canada. The community events had potential to augment campaign effects but adequate evaluation requires sufficient resources. [ABSTRACT FROM AUTHOR]

Purpose: This paper aims at the implementation and early evaluation of a comprehensive, formative annual physician performance feedback process in a large academic health-care organization. Design/methodology/approach: A mixed methods approach was used to introduce a formative feedback process to provide physicians with comprehensive feedback on performance and to support professional development. This initiative responded to organization-wide engagement surveys through which physicians identified effective performance feedback as a priority. In 2013, physicians primarily affiliated with the organization participated in a performance feedback process, and physician satisfaction and participant perceptions were explored through participant survey responses and physician leader focus groups. Training was required for physician leaders prior to conducting performance feedback discussions. Findings: This process was completed by 98 per cent of eligible physicians, and 30 per cent completed an evaluation survey. While physicians endorsed the concept of a formative feedback process, process improvement opportunities were identified. Qualitative analysis revealed the following process improvement themes: simplify the tool, ensure leaders follow process, eliminate redundancies in data collection (through academic or licensing requirements) and provide objective quality metrics. Following physician leader training on performance feedback, 98 per cent of leaders who completed an evaluation questionnaire agreed or strongly agreed that the performance feedback process was useful and that training objectives were met. Originality/value: This paper introduces a physician performance feedback model, leadership training approach and first-year implementation outcomes. The results of this study will be useful to health administrators and physician leaders interested in implementing physician performance feedback or improving physician engagement. [ABSTRACT FROM AUTHOR]

Background: Efforts to engage patients as partners in health research have grown and thereby the need for feedback and evaluation. In this pilot evaluation study, we aimed to 1) evaluate patient engagement in health research projects in Newfoundland and Labrador, Canada, and 2) learn more about how to best monitor and evaluate patient engagement. This paper presents the results of our participatory evaluation study and the lessons learned. The evaluation of the projects was driven by questions patients wanted answered. Methods: We conducted a formative evaluation of patient engagement in health research projects. Projects spanned a variety of topics, target groups, research designs and methods of patient engagement. Participants included principal investigators (n = 6) and their patient partners (n = 14). Furthermore, graduate students (n = 13) working on their own research projects participated. Participants completed an online survey with closed and open-ended questions about their patient engagement efforts, experiences and preliminary outcomes. Patients were involved as co-investigators in the entire evaluation study. We used qualitative methods to evaluate our participatory process. Results: The evaluation study results show that most patients and researchers felt prepared and worked together in various phases of the research process. Both groups felt that the insights and comments of patients influenced research decisions. They believed that patient engagement improved the quality and uptake of research. Students felt less prepared and were less satisfied with their patient engagement experience compared to researchers and their patient partners. Involvement of patient co-investigators in this evaluation resulted in learnings, transparency, validation of findings and increased applicability. Challenges were to select evaluation questions relevant to all stakeholders and to adapt evaluation tools to local needs. Conclusions: Our findings show that researchers, patient partners and students value patient engagement in health research. Capacity building at the supervisor level in academic institutions is needed to better support students. Sufficient time is also needed to permit observable outcomes. Participatory evaluation may increase the relevance and usefulness of information, but it also raises issues such as who defines and designs the content of evaluation tools. A co-creation process is required to develop appropriate monitoring and evaluation strategies. [ABSTRACT FROM AUTHOR]

Copyright of Canadian Journal of Program Evaluation is the property of University of Toronto Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The study aimed to evaluate the feasibility of implementing lifestyle interventions in primary care settings with hypertensive patients and their effect on blood pressure, body composition, cardiometabolic markers, and antihypertensive drug use. Sixty participants diagnosed with stage 1 hypertension were randomly assigned to 4 groups: (1) Standard medical care (control), (2) Physical activity protocol, (3) Dietary Approach to Stop Hypertension (DASH) diet, and (4) Combination of physical activity protocol and DASH diet. Participants received counseling from family physicians, nurses, kinesiologists, and registered dietitians. Various assessments were conducted before (T0) and after (T6) the interventions, including 24-h ambulatory blood pressure monitoring, blood and urine tests, anthropometric measurements, computed tomography to measure adipose tissue, submaximal exercise test to estimate maximal oxygen consumption and health questionnaires. Fifty-one (51) participants (51/57, 89%) completed the program. All interventions reduced blood pressure indices between T0 and T6, except the combined interventions group. Body composition and cardiometabolic parameters were improved in all groups, except for the control group. In total, 28% of participants (7/23) reduced or stopped their antihypertensive medications at T6. The results suggest that structured lifestyle interventions are feasible in primary care and improve blood pressure and cardiometabolic parameters in patients with stage 1 hypertension. [ABSTRACT FROM AUTHOR]

Purpose: Remote patient monitoring (RPM) and virtual visits have the potential to transform care delivery and outcomes but require intentional planning around how these technologies contribute to integrated care. Since maturity models are useful frameworks for understanding current performance and motivating progress, the authors developed a model describing the features of RPM that can advance integrated care. Design/methodology/approach: This work was led by St. Joseph's Health System Centre for Integrated Care in collaboration with clinical and programme leads and frontline staff offering RPM services as part of Connected Health Hamilton in Ontario, Canada. Development of the maturity model was informed by a review of existing telehealth maturity models, online stakeholder meetings, and online interviews with clinical leads, programme leads, and staff. Findings: The maturity model comprises 4 maturity levels and 17 sub-domains organised into 5 domains: Technology, Team Organisation, Programme Support, Integrated Information Systems, and Performance and Quality. An implementation pillars checklist identifies additional considerations for sustaining programmes at any maturity level. Finally, the authors apply one of Connected Health Hamilton's RPM programmes to the Team Organisation domain as an example of the maturity model in action. Originality/value: This work extends previous telehealth maturity models by focussing on the arrangement of resources, teams, and processes needed to support the delivery of integrated care. Although the model is inspired by local programmes, the model is highly transferable to other RPM programmes. [ABSTRACT FROM AUTHOR]

Summary: A variety of hip protectors are available, but it is not clear which is the most effective and there is no standard test to evaluate their performance. This is the first study that uses a standard mechanical test on hip protectors. Some protectors perform well but others are almost ineffective, providing little to no protection to the wearer during a fall. Introduction: Each year, over 70,000 patients are admitted to hospital in the UK with hip fractures. There are a variety of commercial hip protectors currently available. However, it is not explicitly clear which is the most effective with regard to maximum force attenuation, whilst still being both comfortable for the user and providing reasonable force reduction if misplaced from the intended position. The numerous test methods reported in the literature have given conflicting results, making objective comparison difficult for users, researchers, and manufacturers alike. The Canadian Standards Association (CSA) has therefore published an express document (EXP-08-17) with a draft standard test method. This paper presents initial results for a range of hip protectors. Methods: Eighteen commercially available hip protectors were tested according to EXP-08-17. Each hip protector was impacted five times in correct anatomical alignment over the greater trochanter and once at 50 mm displacements in the anterior, posterior, and lateral directions. Results: Considerable differences were identified between individual hip protectors in their ability to reduce impact forces on the femur (between 3% and 36% reduction in peak force). The performance was reduced when misplaced in many cases (maximum reduction only 20%). Conclusions: This is the first study that uses a standard mechanical test on hip protectors. Previous studies have used a variety of methods, making it difficult to interpret results. We hope that these results using a standard test method will facilitate the effective comparison of results, as well as providing useful data for clinicians, users, and purchasers. [ABSTRACT FROM AUTHOR]

Comprehensive or multisector community change initiatives (CCIs) represent a promising approach to address the overrepresentation of Indigenous children and families in Canada's child welfare systems. However, such initiatives are difficult to establish and sustain - in part due to the difficulty of evaluating their impacts and outcomes using standardized Western evaluation methodologies. Consequently, over the past 20 years there have been extensive efforts to develop evaluation principles, methodologies, methods, and tools that are more able to illustrate the benefits of these kinds of initiatives. A systematic review of the CCI evaluation literature found that while many of these principles, methodologies, methods and tools show considerable promise, there has been limited attention to or incorporation of Indigenous ways of knowing or approaches to research with Indigenous peoples. This paper presents two examples of Indigenous led multisector community change initiatives to enhance Indigenous well-being and notes the importance of evaluating their impacts. It argues that collaborative research is needed with participants in Indigenous led multisector collaborations to advance knowledge of culturally relevant approaches to their evaluation. [ABSTRACT FROM AUTHOR]