Search

Abstract Background The Health of the Nation Outcome Scales was developed to routinely measure outcomes for adults with mental illness. Comparable instruments were also developed for children and adolescents (the Health of the Nation Outcome Scales for Children and Adolescents) and older people (the Health of the Nation Outcome Scales 65+). All three are being widely used as outcome measures in the United Kingdom, Australia and New Zealand. There is, however, no comprehensive review of these instruments. This paper fills this gap by reviewing the psychometric properties of each. Method Articles and reports relating to the instruments were retrieved, and their findings synthesised to assess the instruments' validity (content, construct, concurrent, predictive), reliability (test-retest, inter-rater), sensitivity to change, and feasibility/utility. Results Mostly, the instruments perform adequately or better on most dimensions, although some of their psychometric properties warrant closer examination. Conclusion Collectively, the Health of the Nation Outcome Scales family of measures can assess outcomes for different groups on a range of mental health-related constructs, and can be regarded as appropriate for routinely monitoring outcomes.

The article explores how routine outcome measurement (ROM) was used to support benchmarking within mental health services in Australia. It describes the issues that has to be considered when using ROM to support benchmarking and offers advice on how to facilitate the use of ROM within benchmarking. It refers to benchmarking as a systematic process of searching for and implementing a standard of best practices within an individual service or the same groups of services.

The article discusses the results of a study that examined social inclusion as a measure of mental health and well-being. Some candidate measures of social inclusion are the Activity and Participation Questionnaire (APQ-6), the Composite Measure of Social Inclusion (CMSI) and the Social Inclusion Measure (SIM). It concludes with a suggestion that social inclusion measures should be utilized in choosing an appropriate measure for use in public sector mental health services.

The article presents the findings of a research conducted on examining whether feedback on outcome measures which enhance consumer outcomes, can represent clinically meaningful information regarding individual consumers to clinicians. It discusses the methodology of the research including Health of the Nation Outcomes Scales (HoNOS). It concludes that it is difficult to relate to the total scores on outcome measures and individual items to make them clinically meaningful for clinicians.

Objective: To present a framework for exploring the potential applications of routine outcome measurement (ROM) to improve clinical practice.Conclusions: ROM is now commonplace in Australian mental health services, but it is fair to say that it has not yet achieved its potential in guiding service quality improvements. The framework described in this paper articulates the key components of ROM, noting that data from standardized outcome measures must be augmented by contextual information from other datasets. It considers the key users of ROM, and suggests the kind of questions that ROM might answer for each user group. It presents the use of ROM as a cycle of understanding the context, prioritization, brainstorming and planning, action and review. It is hoped that this framework might help to maximize the clinical utility of ROM at a number of levels. [ABSTRACT FROM AUTHOR]

Background: Australia's National Mental Health Strategy has emphasised the quality, effectiveness and efficiency of services, and has promoted the collection of outcomes and casemix data as a means of monitoring these. All public sector mental health services across Australia now routinely report outcomes and casemix data. Since late-2003, the Australian Mental Health Outcomes and Classification Network (AMHOCN) has received, processed, analysed and reported on outcome data at a national level, and played a training and service development role.T his paper documents the history of AMHOCN's activities and achievements, with a view to providing lessons for others embarking on similar exercises. Method: We conducted a desktop review of relevant documents to summarise the history of AMHOCN. Results: AMHOCN has operated within a framework that has provided an overarching structure to guide its activities but has been flexible enough to allow it to respond to changing priorities. With no precedents to draw upon, it has undertaken activities in an iterative fashion with an element of 'trial and error'. It has taken a multi prongedapproach to ensuring that data are of high quality: developing innovative technical solutions; fostering 'information literacy'; maximising the clinical utility of data at a local level; and producing reports that are meaningful to a range of audiences. Conclusion: AMHOCN's efforts have contributed to routine outcome measurement gaining a firm foothold in Australia's public sector mental health services. [ABSTRACT FROM AUTHOR]

Objective: The aim of this paper is to describe the way in which particular Australian and New Zealand mental health services are making use of routine outcome data to foster clinical improvements for consumers. Method: We invited individuals who are responsible for implementing outcome data collection across the two countries to describe exemplary practices occurring within their own services, and present the resultant information in the form of case studies. Results: Outcome measurement is being used to guide clinical decision-making, engage consumers in treatment, foster a collaborative approach to care planning and goal setting, review consumers'' progress with treatment, inform questions about consumers'' eligibility for given programs, assist with discharge planning, improve the evidence-base on which services are founded, and evaluate particular models of service delivery. Conclusions: A number of mental health services are deriving useful information from routine outcome measurement, and using this to guide clinical practice. The examples provided here may offer some lessons for other services wishing to make better use of outcome data, and there may be some opportunities for the sharing of resources or infrastructure. Services that are already using outcome data to inform their clinical activities, and services that are keen to do so, will need ongoing support. [ABSTRACT FROM AUTHOR]

Objective: This paper describes the adult mental health forums that were conducted as part of the National Mental Health Benchmarking Project (NMHBP). Method: Eight adult mental health forums were attended by staff from eight adult mental health services from around the country. The forums provided an avenue for these participants to document their organizations'' performances against previously agreed key performance indicators (KPIs), and to compare this performance with that of their peers. The forums also encouraged discussion about appropriate targets. Results: Forum participants found that the inter-organizational range for many of the KPIs was substantial, and they used this to inform practice change within their own organizations. They also found that they could set ''alert targets'' and ''good practice targets'' for some KPIs but not others. The discussion that ensued informed participants'' understanding of factors that were within the control of their organizations that could be modified to improve service quality. Conclusions: Benchmarking in adult mental health services is not only possible but also likely to be extremely worthwhile as an exercise in improving service quality. For benchmarking to realize its potential, it requires strong national and local leadership, and a spirit of openness on the part of participating organizations. [ABSTRACT FROM AUTHOR]

Objective: The aim of this paper was to identify risk factors associated with readmission within 28 days of discharge from eight Australian adult acute mental health inpatient services. Method: A detailed file audit was conducted comparing 222 patients readmitted within 28 days of discharge with 253 patients not readmitted during the same period. Results: There was an association between early readmission and having had contact with the service in the previous 12 months (51% vs 21%), having been admitted in the previous 12 months (65% vs 36%), and having been diagnosed with an emotionally unstable personality disorder (14% vs 4%). Those who were not readmitted had a significantly higher score in the Health of the Nation Outcome Scales on index admission than those who were readmitted. Those who had community team contact on the day of discharge and those who received follow up by the mental health team within 7 days of discharge were more likely to be readmitted (55% vs 45% and 29% vs 19% respectively). Conclusions: Significant determinants of early readmission identified were a history of recent previous admissions and a diagnosis of unstable personality disorder. The usefulness of this performance measure to Area Mental Health Services and clinicians is discussed. [ABSTRACT FROM AUTHOR]

Objective: This paper describes the forensic mental health forums that were conducted as part of the National Mental Health Benchmarking Project (NMHBP). These forums encouraged participating organizations to compare their performance on a range of key performance indicators (KPIs) with that of their peers. Method: Four forensic mental health organizations took part in the NMHBP. Representatives from these organizations attended eight benchmarking forums at which they documented their performance against previously agreed KPIs. They also undertook three special projects which explored some of the factors that might explain inter-organizational variation in performance. Results: The inter-organizational range for many of the indicators was substantial. Observing this led participants to conduct the special projects to explore three factors which might help explain the variability -- seclusion practices, delivery of community mental health services, and provision of court liaison services. The process of conducting the special projects gave participants insights into the practices and structures employed by their counterparts, and provided them with some important lessons for quality improvement. Conclusions: The forensic mental health benchmarking forums have demonstrated that benchmarking is feasible and likely to be useful in improving service performance and quality. [ABSTRACT FROM AUTHOR]

Objective: This paper describes aspects of the child and adolescent benchmarking forums that were part of the National Mental Health Benchmarking Project (NMHBP). These forums enabled participating child and adolescent mental health organizations to benchmark themselves against each other, with a view to understanding variability in performance against a range of key performance indicators (KPIs). Method: Six child and adolescent mental health organizations took part in the NMHBP. Representatives from these organizations attended eight benchmarking forums at which they documented their performance against relevant KPIs. They also undertook two special projects designed to help them understand the variation in performance on given KPIs. Results: There was considerable inter-organization variability on many of the KPIs. Even within organizations, there was often substantial variability over time. The variability in indicator data raised many questions for participants. This challenged participants to better understand and describe their local processes, prompted them to collect additional data, and stimulated them to make organizational comparisons. These activities fed into a process of reflection about their performance. Conclusions: Benchmarking has the potential to illuminate intra- and inter-organizational performance in the child and adolescent context. [ABSTRACT FROM AUTHOR]

Objective: This paper provides an overview of the National Mental Health Benchmarking Project (NMHBP) which explored the value of benchmarking within the four main program areas of public sector mental health (general adult, child and adolescent, older persons and forensic). The NMHBP involved a series of forums which enabled participating organizations to benchmark themselves against each other, with a view to understanding variability in performance against a range of key performance indicators (KPIs). Method: Twenty-three mental health organizations took part in the NMHBP. The forums culminated in each mental health organization documenting its performance against relevant KPIs. The processes, impacts and outcomes of the NMHBP were evaluated via a multi-methods evaluation. Results: There was considerable variability across program streams for many of the KPIs, much of which could be explained by contextual factors. Within program streams there was considerable intra-organization variability. Participants found the examination of intra-organization variability on particular indicators to be useful. Conclusions: The NMHBP has shown that benchmarking is possible. Managers and clinical leaders will need to champion benchmarking and highlight its utility in relation to quality improvement and service development if the accountability goals of the Fourth National Mental Health Plan are to be realized. [ABSTRACT FROM AUTHOR]

Objective: This paper offers a reflection about the outcomes of the older persons benchmarking forums that formed part of the National Mental Health Benchmarking Project (NMHBP). Method: Seven older persons mental health organizations took part in the NMHBP. Representatives from these organizations attended eight benchmarking forums at which they documented their performance against relevant key performance indicators (KPIs). In order to better understand the differential performance of organizations on particular KPIs, participants gathered additional contextual data. This included organization-level data, data on the local catchment area, and data with which to profile the consumers accessing services through the given organization. Results: Participants'' average performance on some indicators was stable over time, while the average performance on others demonstrated fluctuations. Perhaps more importantly, the inter-organization range for almost all of the indicators was substantial. Conclusions: The older persons benchmarking forums provided an opportunity for participants to gauge the performance of their own organizations on a range of KPIs, come to understand some of the reasons for their own organization's performance and that of their counterparts, consider which of these reasons may be within their control, and reflect upon opportunities for quality improvement within their own organizations. [ABSTRACT FROM AUTHOR]

Objective: Although Australia and the UK have both made efforts to systematize outcome measurement in mental health services, surprisingly little attention has been paid to how best to analyse routine outcome data in order to determine how services are performing. Methods: Outcome data collected in acute inpatient and ambulatory mental health services across Australia during the 2006-2007 financial year were used. three approaches to measuring effectiveness were explored: effect size (ES); the reliable change index (RCI); and standard error of measurement (SEM). Results: The most conservative results were produced by the RCI and the least conservative by the medium ES statistic and the SEM. By way of example, only 38.0% of inpatient admission-discharge periods of care showed significant improvement for adults when the RCI was used, whereas 67.4% and 72.9% did so when the medium ES and the SEM statistics were used, respectively. Conclusions: In any routine outcome measurement exercise, the degree of effectiveness demonstrated by services will depend on the specific statistical indicator used to judge effectiveness. Routine outcome measurement has the potential to answer a range of crucial performance-related questions, but only if the same metric is used. Discussion of the appropriate statistical approach to take to facilitate cross-service, cross-area and even cross-national comparisons warrants attention. [ABSTRACT FROM AUTHOR]

Objective: The aim of this paper was to improve understanding of what 'clinical significance' means in relation to the Health of the Nation Outcome Scales (HoNOS) and its older persons and child/adolescent equivalents (the HoNOS65+ and HoNOSCA). Method: An anonymous, web-based survey was completed by 94 outcome measurement experts, most of whom had clinical responsibilities. Respondents were asked to indicate for acute inpatient and ambulatory settings: the rating on each item which represented a clinically significant problem; the relative importance of each item in determining overall clinical severity; and the items which would not be expected to improve between admission and review, admission and discharge, review and review, and review and discharge. Results: A score of 2 ('mild problem but definitely present') on each HoNOS/HoNOS65 + /HoNOSCA item resonates with experts as being evidence of a clinically significant problem that requires active monitoring or intervention. In the main, all items on these instruments are viewed as equally important in making an overall judgement of clinical severity. The items making up the impairment and, to a lesser extent, social subscales are least likely to demonstrate change during the course of an episode of care, according to expert opinion. Generally, these findings apply across instruments and service settings. Conclusions: Overall, the findings provide support for the content validity and clinical utility of the HoNOS/HoNOS65 + /HoNOSCA. Further exploration of the question of clinical significance as reflected in these instruments could take a number of forms. [ABSTRACT FROM AUTHOR]

Relationships within acute psychiatric units between patient-level experiences and events and fluctuations in mental state have rarely been examined. Data from a multi-centre service evaluation (11 units, 5,546 admissions) were used to examine mental state patterns and associations with clinical characteristics, events and adverse incidents. During the 12-month evaluation period, nursing staff completed shift-level ratings using a new rating scale, the observed mental state (OMS) scale, which assessed active psychopathology (emotional distress, disinhibition, psychosis, cognitive impairment) and withdrawal (45,885 sets of day/afternoon shift ratings). The OMS scale performed satisfactorily and is worth considering elsewhere (e.g., active psychopathology: internal consistency, α = 0.72; short-term stability, r = 0.72; sensitivity to change, adjusted standardised difference, ASD = 0.71). Levels of active psychopathology were much higher on shifts in which reportable (ASD = 1.47) and less serious aggression occurred (ASD = 1.44), compared with other shifts in which pro re nata medications were also administered (ASD = 0.76), suggesting that medication usage often followed these events, and possibly that agitation and distress levels either rose rapidly or went initially unnoticed on these shifts. Although mental state improved steadily across the admission, one-fifth of the patients with schizophrenia received OMS psychosis ratings in the moderate to severe range during the days prior to discharge. Observed mental state ratings were strongly linked with diagnosis and reflected key events and incidents. Routine recording using the OMS scale may assist clinical decision-making and evaluation in acute psychiatric units. [ABSTRACT FROM AUTHOR]

In this study conducted by consumer consultants, 50 consumers who have a Barwon Health case manager (the majority of whom were nurses) were interviewed using a structured questionnaire to ascertain their attitudes towards the routine use of outcome measures. Forty participants (80% of those interviewed) reported they had been offered the Behaviour and Symptom Identification Scale (BASIS-32) to complete in routine care by their case managers and of those, 95% (n = 38) completed it. On those who completed the BASIS-32, 42% said their case manager had explained what the BASIS-32 would be used for, 45% said that the case manager had discussed their responses with them, 76% stated that completing the BASIS-32 had helped the case manager to understand them better and 66% believed that completing the BASIS-32 had led to them receiving better care. Only 30% of the group interviewed were aware that their case manager regularly completed a Health of the Nation Outcome Scales and Life Skills Profile. Feedback about the process of completing the BASIS-32 was obtained as well as suggestions on how the process may be improved. The results indicate that consumers see the benefit of routine outcome measurement and believe it leads to improved care. More information about outcome measures, including the clinician-rated outcome measures, needs to be provided to consumers if they are to be engaged constructively in this exercise. [ABSTRACT FROM AUTHOR]

Background: The Health of the Nation Outcome Scales was developed to routinely measure outcomes for adults with mental illness. Comparable instruments were also developed for children and adolescents (the Health of the Nation Outcome Scales for Children and Adolescents) and older people (the Health of the Nation Outcome Scales 65+). All three are being widely used as outcome measures in the United Kingdom, Australia and New Zealand. There is, however, no comprehensive review of these instruments. This paper fills this gap by reviewing the psychometric properties of each. Method: Articles and reports relating to the instruments were retrieved, and their findings synthesised to assess the instruments' validity (content, construct, concurrent, predictive), reliability (test-retest, inter-rater), sensitivity to change, and feasibility/utility. Results: Mostly, the instruments perform adequately or better on most dimensions, although some of their psychometric properties warrant closer examination. Conclusion: Collectively, the Health of the Nation Outcome Scales family of measures can assess outcomes for different groups on a range of mental health-related constructs, and can be regarded as appropriate for routinely monitoring outcomes. [ABSTRACT FROM AUTHOR]