Search

This paper addresses non-academic training in evaluation with a twofold perspective. The first one will focus on the reconstruction of a map of the existing training offer in Italy. This map, updated at September 2022, will propose an overview of the existing courses in terms of both main features (e.g.: duration, structure, providers) and macro-contents. The second perspective will match the results of this analysis with the profile, overall and by competences, of the Expert in Evaluation of Policies, Programmes, Projects and Institutions as defined by the Italian Evaluation Society (A.I.V.). This match intends to explore the correspondence between the courses and the A.I.V. proposal. In such a way, an attempt will be made, on the one hand, to contextualize the courses with respect to A.I.V. evaluator's profile and competences and, on the other hand, to do the same with A.I.V. profile in relation to evaluation courses. Finally, some recommendations will be proposed on the basis of the evidence produced by the analysis. [ABSTRACT FROM AUTHOR]

The main objective of the paper is to address and explore a much-debated issue within the evaluation community: the idea of "democratic evaluation". What exactly is meant by this term? When can an evaluation be said to be democratic? Through the adoption of what behaviours should this ideal translate into concrete working practice? The essay aims to contribute to broadening the community's debate on these questions, which are central at a historical-political moment such as the post-emergence from Covid-19 and in the midst of an international crisis, that between Russia and Ukraine, which calls democratic evaluation into question in our country. On a general level, in fact, the situation of uncertainty due to the unpredictable effects of the pandemic, the social and economic crisis in the country and the consequent growing distrust of the citizens towards the administration seem to have increased the demand for accountability, leading to talk, as in the rest of Europe as well, of a "democratic deficit". On the other hand, recent administrative re-centralization, even assisted by current evaluative practices, only aggravates the inefficiency of public administration. Accountability, freedom and democracy are never given, they need to be defended, always regained: evaluation assumes a strategic role to this end; it can help bring citizens closer to public administration. The crisis between Russia and Ukraine is changing geopolitical balances; the effects of the war economy, inflation, the energy crisis, and the possible escalation of the conflict add to the uncertain situation due to the pandemic. New inequalities and an increase in poverty threaten to generate deep rifts in the country's socioeconomic fabric. Systemic shocks add to chronic problems: a demographic collapse, stagnation, job insecurity, and a lag in digitization. At the same time, the political system fails to represent and stabilize consensus over time. Governments also suffer from this instability, with the inability to design and implement structural reforms for the country, including that of the bureaucracy. Social complexity, chronic problems and systemic shocks, require the ability to apply effective public policies. State intervention and the ability of the bureaucracy to be efficient and responsive to social change are central issues for the health of democracy. Public policies move within a framework of very high complexity, in which the role of evaluation emerges, which should tend to the development of democratic accountability, illuminating decision-making choices through its methodological rigor, analytical capacity and scientific knowledge. The essay thus aims to address the potential of a democratic evaluation that can broaden the number of stakeholders, the field of values that are taken into account, the repertoire of research methods, the field of interest of policies, and the very definition of the objects of evaluations. [ABSTRACT FROM AUTHOR]

The paper wants to explore, using the case of the PRIN, what criteria are most influential in the evaluation of proposals submitted for funding, in particular what is the relevance of the criteria related to the scientific quality of the proposal in itself (originality, methodological rigor etc.) than the quality of the proposing team, or alternative criteria of evaluation such as the impact expected of the research or the geographical location of the proposers. It is assumed that there is a difference between the quality criteria that scientists belonging to a scientific community say they use when they work as evaluators in the process of peer review, and the perception that scientists themselves have of the importance of scientific quality when they are objects of a process of evaluation. This difference is based on the existence of mechanisms distorting the peer review that: a) derive from the internal organizational characteristics of the scientific field; b) tend to become deeper when rules circumscribe the open debate of the reviewers and selection committees, and introduce the use of technical constraints (eg. methods and rating scales and decisions based on average scores) favouring the emergence of factors other than the scientific quality. [ABSTRACT FROM AUTHOR]

The first relevant attempt to describe mixed methods (MM) in the field of evaluative research is attributable to Greene and Caracelli (1997). The authors distinguish various forms of MM starting from the purposes for which they are used. Since 1997 important developments occurred in academic and professional debates related to MM in evaluative research. Starting from the reconstruction of the debate on the usefulness and use of MM in evaluative research, this paper reflects on the ontological, epistemological, methodological, and axiological issues related to the evaluative research processes using MM. This consideration needs to involve an extended vision of the issues underlying the choices of specific methods in the research process. This is even more necessary in evaluative research that constitutionally engages with complex dilemmas, especially on the axiological level. [ABSTRACT FROM AUTHOR]

The paper, in light of the growing interest that the Third Mission of the University knows at international and national level, delves into the ways in which it has been detected and evaluated by ANVUR in recent years. It shows that this assessment is based on two logics, one of a linearincremental type (more is better) and one of adequacy of the activities carried out face to the needs of the territory and the internal resources of the University. The second appears to be the most useful for the learning function that the evaluation should carry out in order to continuously improve the universities in their positioning into their settlement area. Finally, it suggests some lines of work to use TM assessment for learning purposes. [ABSTRACT FROM AUTHOR]

The guiding hypothesis of the paper assumes that big data (BD) started affecting evaluation practices (formal and informal) in / of planning. The delay in developing 'public internet' for the benefit of the platforms prevents knowledge from being recognized as a common good. But, we must not underestimate the fact that the individual data in a social practice such as planning, have little value if not aggregated and aimed at offering innovative, and not only marketable, services. BDs open up new possibilities, but they also raise new issues of a cognitive, regulatory and decision-making type, as well as ethical and deontological. Four case studies provide evidence in this regard, particularly in the domains of mapping, mobility management and participation in public policies. These evidences help to identify relevant criteria for the evaluation of BD contributions in planning processes. [ABSTRACT FROM AUTHOR]

The paper describes a research work about evaluation of the acted and desired quality of professional activity within the school for foreigners of the multiethnic cultural center "La Tenda" in Milan. Competences useful for the professional development are the fulcrum of this evaluative research that has tried, not only to describe what implements the studied quality, but also to highlight the premises and consequences. In addition to the skills necessary for the professional mission, we therefore refer to the pre-requisites for carrying out the task: personal (eg: education), material (eg structural endowment) and intangible (eg: the ethical principles of the school). We refer to the facilitating elements and the consequent result that can be achieved by the work that the school teachers carry out (learning Italian and integrating with the new culture). [ABSTRACT FROM AUTHOR]

In line with a constructivist approach and a pratical participatory evaluation, the paper used brainstorming technique to define effectiveness' criteria of children adoption. 26 judges of the Juvenile Court of Bari took part in the evaluational brainstorming, which have been organized in a creative phase, a classification step and a final relevance analysis and group discussion. Applying the Lawshe formula, it has been possible identify the most shared success standards (children well-being: self-esteem, adjustment and secure attachment; adoptees' and parents' skill to revise their life story; adoptees' scholastic and social integration; a good parentsadoptees relationship) and failure ones (parents reject adoptees or adoptees abandon adoptive family). The study highlights suggestions regarding the flexible use of brainstorming technique and underlines the potentials to use a shared evaluation procedure in order to improve both adoption research and practices. [ABSTRACT FROM AUTHOR]

The aim of this paper is to propose a reflection on specific learnings, arising from a cyclic evaluation of the same intervention. The proposed topic is the study of Placement of Vocational Training in Piedmont Region (IT), held from 2011 to 2014. Through a diachronic reading of the four annual reports of the Placement, are presented three dimensions of analysis: financial dynamics, employment outcomes and net impacts. This contribute aims to discuss learnings in terms of method and policy, starting by the main results emerging from the analysis of the three dimensions listed above. [ABSTRACT FROM AUTHOR]

In the last decades, a lot of resources for active labour market policies have been devoted to training programs. Even if the opinion is diffused that training provides both an increase in skills and (thus) employability, following the international literature the impact of training on working conditions is very low. Evidence from Italy is quite scarce. In this paper an evaluation from Piedmont (in the north-west of Italy) is presented. Using administrative data from the job training and the labour market information systems, the employment rates of two groups of unemployed enrolled at Public Employment Service offices are compared. The impact estimates, obtained via propensity score matching, are (except for some quite peculiar courses) close to zero. Results are slightly better in the case of long term courses. While presenting the research, main advantages and disadvantages of different evaluation designs are discussed. [ABSTRACT FROM AUTHOR]