Search

The paper discusses the idea that the evaluation of public policies is witnessing a remarkable enlargement of its domain, in four relevant dimensions- its object, effect, methods, and data.Regarding the objects of evaluation, policy makers are increasingly interested not only in the evaluation of the content of policies (output) or the result (outcome), but also on a broader notion of impact, which includes direct and indirect effects, on the sub-system under analysis and on society at large. Regarding the effects of policies, the paper calls the attention on the rising of the notion of conditionality, requiring more flexible and sophisticated approaches for in itinere evaluation. With respect to new methods, the paper discusses new developments in counterfactual methods, robust nonparametric approaches (robust and conditional), as well as methods to deal with non commensurable indicators. Finally, the implications for evaluation of policies of the availability of microdata from public sources are examined. In all these devekopments new challenges are identified, calling for advanced methodological and quantitative work. [ABSTRACT FROM AUTHOR]

In order to evaluate the scientific products of the Italian legal scholarship and make them comparable with the outputs in other scientific areas, some institutional actors have proposed to classify different scientific works (monographs, articles, notes etc.), publishers and journals. Part of learned societies in legal area have published a joint "Document on evaluation" (October 1, 2010) about methods and criteria that should be applied by various institutional organizations for several purposes. The aim of this paper is to offer a critique of that document, as well as to suggest that digital technologies (in particular, Open Access to science) might play a prominent role in the evaluation of products in the legal science field. [ABSTRACT FROM AUTHOR]

In the current contexts in which the accountability purpose of the evaluation is emphasized, communicating the results of the evaluation research is a priority, so that all stakeholders can understand its processes and results. Also with regard to social accountability, the digitalization process opens up new scenarios for methods, tools and communicative languages able to optimize the communication opportunities provided by technologies. The paper aims to foster a first reflection on a issue that has not yet been adequately addressed, combining models of communicative languages based on Infographic and Data Visualization, already experimented in Information Design, and problematic aspects of evaluation. The paper, therefore, aims to be an opportunity to discuss about innovative ways of communicating the evaluation, investigating how they can contribute to spread a culture of evaluation and to legitimize it as a cognitive tool to improve practices and processes. [ABSTRACT FROM AUTHOR]

The main objective of the paper is to illustrate the theoretical and methodological construction of an evaluation research design "tailored maded" on a complex program by the "Mediamonitor Minor" of the Sapienza University of Rome, "Media Usage in Pre-school. Analysis and Evaluation of the Influence of Digital Media on the Socialization of Children between 0-6". The paper, in particular, illustrates the stages of the research "mixed" strategy designed to analyze and evaluate the effects of the introduction of digital teaching in the kindergarten, with a focus on the observation of changes in attitudes and behavior of children at school and on the development of fields of expertise (the self and the other, images, sounds and colors, etc.), according to Ministerial Guidelines (2012) for the implementation of future skills of children between 3 and 6 years. [ABSTRACT FROM AUTHOR]

This paper addresses non-academic training in evaluation with a twofold perspective. The first one will focus on the reconstruction of a map of the existing training offer in Italy. This map, updated at September 2022, will propose an overview of the existing courses in terms of both main features (e.g.: duration, structure, providers) and macro-contents. The second perspective will match the results of this analysis with the profile, overall and by competences, of the Expert in Evaluation of Policies, Programmes, Projects and Institutions as defined by the Italian Evaluation Society (A.I.V.). This match intends to explore the correspondence between the courses and the A.I.V. proposal. In such a way, an attempt will be made, on the one hand, to contextualize the courses with respect to A.I.V. evaluator's profile and competences and, on the other hand, to do the same with A.I.V. profile in relation to evaluation courses. Finally, some recommendations will be proposed on the basis of the evidence produced by the analysis. [ABSTRACT FROM AUTHOR]

The main objective of the paper is to address and explore a much-debated issue within the evaluation community: the idea of "democratic evaluation". What exactly is meant by this term? When can an evaluation be said to be democratic? Through the adoption of what behaviours should this ideal translate into concrete working practice? The essay aims to contribute to broadening the community's debate on these questions, which are central at a historical-political moment such as the post-emergence from Covid-19 and in the midst of an international crisis, that between Russia and Ukraine, which calls democratic evaluation into question in our country. On a general level, in fact, the situation of uncertainty due to the unpredictable effects of the pandemic, the social and economic crisis in the country and the consequent growing distrust of the citizens towards the administration seem to have increased the demand for accountability, leading to talk, as in the rest of Europe as well, of a "democratic deficit". On the other hand, recent administrative re-centralization, even assisted by current evaluative practices, only aggravates the inefficiency of public administration. Accountability, freedom and democracy are never given, they need to be defended, always regained: evaluation assumes a strategic role to this end; it can help bring citizens closer to public administration. The crisis between Russia and Ukraine is changing geopolitical balances; the effects of the war economy, inflation, the energy crisis, and the possible escalation of the conflict add to the uncertain situation due to the pandemic. New inequalities and an increase in poverty threaten to generate deep rifts in the country's socioeconomic fabric. Systemic shocks add to chronic problems: a demographic collapse, stagnation, job insecurity, and a lag in digitization. At the same time, the political system fails to represent and stabilize consensus over time. Governments also suffer from this instability, with the inability to design and implement structural reforms for the country, including that of the bureaucracy. Social complexity, chronic problems and systemic shocks, require the ability to apply effective public policies. State intervention and the ability of the bureaucracy to be efficient and responsive to social change are central issues for the health of democracy. Public policies move within a framework of very high complexity, in which the role of evaluation emerges, which should tend to the development of democratic accountability, illuminating decision-making choices through its methodological rigor, analytical capacity and scientific knowledge. The essay thus aims to address the potential of a democratic evaluation that can broaden the number of stakeholders, the field of values that are taken into account, the repertoire of research methods, the field of interest of policies, and the very definition of the objects of evaluations. [ABSTRACT FROM AUTHOR]

The Principi Attivi programme (PA) aimed at promoting meaningful lifelong learning experiences to improve young people's job chances and personal development. Based on a survey that involved young leaders of 141 funded projects (75,40% of 187), this paper reports and discusses the effects generated on the individual life paths of the young participants. Specifically, the paper focuses on the role played by a number of personal factors (cultural capital, background experiences, innovative cognitive style) and contextual factors (social support, collective self-efficacy) on professional empowerment as well as on the development of planning and networking skills. By adopting a methodological triangulation strategy, the results of the survey have been further discussed drawing on the narrative data of in-depth interviews within 16 case studies. [ABSTRACT FROM AUTHOR]

The paper proposes a case study about integration of focus group and survey to listen teenagers and preteens in the participatory evaluation of a local educational project. The contribution discusses strengths and weaknesses of the focus group as a participatory instrument in the evaluation process and the integration with the survey. Furthermore, the authors reflect about possible ways to enhance the approach, starting from the Youth Participatory Evaluation model and focusing on the role of teenagers in different evaluation stages. [ABSTRACT FROM AUTHOR]

The Covid-19 pandemic has required new reflections on the evaluation of multi-dimensional programs aimed at multiple stakeholders. The traditional approach, based on the blueprint approach, resulted unsuitable for grasping the complexity of the new scenario, due to its focus on objectives achieved. This paper aims to describe the evaluation of two projects for the inclusion of marginalized young people, through the Positive Thinking lenses. In fact, it turned out to be the theoretical approach most capable of responding to the complexities that emerged in this peculiar historical period. The evaluation becomes useful for triggering improvement actions and for capturing the efforts of the organizations. The contribution intends to open the theoretical reflection on the complexity of the programs, and on the role of the evaluator in accompanying the organizations by supporting them, in order to develop effective actions for the beneficiaries. [ABSTRACT FROM AUTHOR]

This paper proposes a reflection on the connection between social innovation-driven social design and social impact evaluation (SIE). The author argues for a cognitive and learning function of SIE that can be conducted through the choice of methods, approaches and tools that can capture and enhance the innovations introduced by the programs. To this end, the construction path of the research design of the "Ri-belli" project, funded by the "Educational Poverty Alleviation Fund", is presented. The intervention is representative of the renewed system of second welfare, marked by the involvement of the private social sector in the policy making process and the desire to disseminate evaluative practice in the social sphere. [ABSTRACT FROM AUTHOR]

This paper aims to contribute to the understanding of the relationship between the mixed methods approach and the evaluation field, with an overall look at the international social science area. Analysing a set of papers that make use of mixed methods, a reflection on the prevalence of the evaluation field is presented. Text analysis through the Structural Topic Model (STM) technique allows to extract topics from a corpus of documents. Findings show how the evaluation filed has a relevant role within the area of studies that use a mixed methods approach. Evaluation emerges indeed as a topic itself. Moreover, relationships among evaluation and other disciplinary fields wherein mixed methods are used can be highlighted. The overview presented in this study allows to look at trends within the mixed methods approach in social science and within evaluation, as well as intersections between the two areas. [ABSTRACT FROM AUTHOR]

The paper aims to introduce the reflection upon the Mixed Methods (MM) approach and evaluation research that will be attained in the following contributions. It is by now largely recognised how rarely a single evaluation strategy can fully capture the entire complexity of social phenomena happening in the "real" world. Nowadays, many recognise that combining different strategies and techniques within one evaluative process is more efficient than using conventional evaluation approaches with a single research strategy. This article recomposes the development of a need for a MM approach in Italy. Moreover, other papers in this special issue are briefly presented by identifying a common thread, represented by the reflection suggested in the paper by Jennifer Greene. [ABSTRACT FROM AUTHOR]

This paper aims to analyze the reflexivity issue in scientific research practice. Starting from a theoretical reflection on the epistemological and methodological reasons listing reflexivity among the main practical issues of social and evaluative research, the contribution seeks to highlight how the sus-tainability of reflective research practices actually swings between adequacy and challenges. With particular reference to pragmatic approaches to knowledge, it is believed that the main elements that can undermine reflex-ivity are the contexts in which the investigation takes place and the language's indexicality. Finally, particular attention is paid to the transactional nature of the evaluative research in which contexts and languages constitute a mix of indisputable relevance, for its effectiveness and quality. [ABSTRACT FROM AUTHOR]

Percorsi is a matched savings program aimed at helping students from low-income households attend college. Previous experimental research has demonstrated the positive effects of the program on college enrollment and persistence. This paper investigates the extent to which these experimental results are attributable to the applied targeting strategy: students were admitted to the study through a targeting mechanism based on household income and on a predictive model of university enrollment probability. Applicants with a very high predicted university enrollment probability and those who revealed no intention of going to university were excluded from the program. The authors assess the performance of this approach by comparing cost-effectiveness estimates against simulated scenarios, showing that both the implemented and a simplified targeting yield a cost-effectiveness gain over a no targeting scenario. The study makes the case for an increased use of prior knowledge and collaboration between researchers and program managers to enhance programs' cost-effectiveness. [ABSTRACT FROM AUTHOR]

The paper proposes a contribution to the reflection on the effectiveness of the SEA instrument as a process of environmental sustainability integration in planning choices in general, and, in particular, for some policy areas (adaptation to climate change) characterized by lack of methodological and operational tools. To this end, the experience of the SEA of the Calabria Region Transport Plan is analyzed in the light of the open questions in the international scientific debate. The SEA conducted is described by illustrating the used methodologies in relation to the type of approach adopted in the process. The reflection highlights how, even in the presence of constructive approaches, as defined in the reference literature, the integration of environmental sustainability is not immediately verifiable, since it is postponed to the subsequent implementation phases. This difficulty is even more evident in relation to the application to the policies on adaptation to climate change, an area characterized by the lack, at national and regional level, of well-structured and operationally defined reference instruments. In this case, even if adopting a constructive approach, the achieved result is limited, necessarily, to general indications, typical of a compliance approach with respect to the policy itself. [ABSTRACT FROM AUTHOR]

The transformations characterizing the Italian university and research system reproduce a constant oscillation between autonomy and responsibility. A new pathway has been paved with the introduction of evaluation and the creation of an agency with a specific mandate for managing it, the National Agency for the Evaluation of Universities and Research Institutes (ANVUR). The process of institutional creation has given rise to a new postbureaucratic model, inspired by independence and professionalism principles. Specific organizational devices have been implemented to this aim, such as the appointment procedures of the agency's Governing Board as well as the introduction of a new officer profile, the technical officer for evaluation, working side by side administrative staff, to serve the evaluation scope. In this paper, the agency's organizational model will be analysed, together with the main characteristics and the principles of the agency in the framework of the Italian university and research system. [ABSTRACT FROM AUTHOR]

The paper wants to explore, using the case of the PRIN, what criteria are most influential in the evaluation of proposals submitted for funding, in particular what is the relevance of the criteria related to the scientific quality of the proposal in itself (originality, methodological rigor etc.) than the quality of the proposing team, or alternative criteria of evaluation such as the impact expected of the research or the geographical location of the proposers. It is assumed that there is a difference between the quality criteria that scientists belonging to a scientific community say they use when they work as evaluators in the process of peer review, and the perception that scientists themselves have of the importance of scientific quality when they are objects of a process of evaluation. This difference is based on the existence of mechanisms distorting the peer review that: a) derive from the internal organizational characteristics of the scientific field; b) tend to become deeper when rules circumscribe the open debate of the reviewers and selection committees, and introduce the use of technical constraints (eg. methods and rating scales and decisions based on average scores) favouring the emergence of factors other than the scientific quality. [ABSTRACT FROM AUTHOR]

This paper concerns the evaluation of a pilot health project aimed ameliorate the quality of life and the management of the illness of a group of patients and their families. The evaluation was requested following the results of the monitoring carried out by the project managers in order to understand why the program was not achieving the desired effects. In this paper we show how evaluation can give to decision-makers several elements for re-thinking and adjusting the programme through the use of an approach to democratic and pluralist evaluation. This approach has allowed to review the program theory and to give greater importance to economic social variables. [ABSTRACT FROM AUTHOR]

Digital tools and ICTs have long since begun to change the field of education assessment and have all the potential to change it further. One positive aspect is that to administrate a computer-based test is more efficient and economical than a traditional paper&pencil one (Bridgeman 2009). Other positive features speak in their favour: digital and ICT devices in assessment can better reflect the domains of skills to be evaluated. Specific cognitive constructs are difficult to evaluate with traditional tools and have largely emerged as an integral part of the digital era (Kelley, Haber 2006). Moreover, the use of digital assessment allows to better investigate the dynamic interactions between students and the items of assessment (response times, methods of constructing the answer or solving the problem). However, in addition to strictly technical issues about efficacy, assessment via digital tools has opened a series of very delicate debates for the immediate future of evaluation in educational. Is the use of ICT tools and digital practices in large-scale assessments a progressive development of traditional methodologies or does it involve a profound pedagogical shift of the way in which education and learning will be managed in the immediate future? In developing this paper, authors address a number of crucial limitations and problems that are emerging with the rapid dissemination of digital assessment tools and practices. [ABSTRACT FROM AUTHOR]

Il contributo propone un'analisi comparata di due valutazioni condotte su due progetti finanziati dall'impresa sociale Con i Bambini sul Bando "Adolescenza", che intende contrastare la dispersione scolastica degli adolescenti (11-17 anni). L'obiettivo è mettere in luce punti in comune e differenze, per identificare sia gli aspetti che caratterizzano la complessità di tali interventi e i loro esiti in diversi contesti sia i meccanismi causali potenzialmente generalizzabili ad altre esperienze progettuali. Il paper combina la Teoria del Cambiamento prevista dal modello DOME (Tomei, 2020) con una comparazione "processuale" (Bartlett e Vavrus, 2017) comprensiva delle dimensioni organizzativa, territoriale-contestuale, strategico-causale e di outcome. Nonostante le differenze organizzative, i due progetti presentano diversi punti di convergenza sulla strategia e i risultati, stimolando la riflessione (sia valutativa che di policy) su una serie di questioni quali, ad esempio, a) l'introduzione di metriche che consentano di misurare outcome ulteriori rispetto al rendimento scolastico (motivazione, autostima, capitale sociale), b) il supporto alle "comunità educanti" e c) il coinvolgimento di insegnanti e famiglie. [ABSTRACT FROM AUTHOR]

The first relevant attempt to describe mixed methods (MM) in the field of evaluative research is attributable to Greene and Caracelli (1997). The authors distinguish various forms of MM starting from the purposes for which they are used. Since 1997 important developments occurred in academic and professional debates related to MM in evaluative research. Starting from the reconstruction of the debate on the usefulness and use of MM in evaluative research, this paper reflects on the ontological, epistemological, methodological, and axiological issues related to the evaluative research processes using MM. This consideration needs to involve an extended vision of the issues underlying the choices of specific methods in the research process. This is even more necessary in evaluative research that constitutionally engages with complex dilemmas, especially on the axiological level. [ABSTRACT FROM AUTHOR]

The paper explores the potential role of Institutional Repositories for processes of evaluation of research results. It presents a methodology for the definition and classification of the content of Institutional Repositories that will be useful to outline the standards of comprehensive, comparable and defined outputs, to select those corresponding to the different mission and institutional activities carried out by research organizations (Universities and Public Research Institutions). The research question is: what is the value of the Institutional Repositories for the evaluation of public research organizations even beyond research activities? What kinds of output should an Institutional Repository contain to respond to both scientific requirements and comparability assumptions in the evaluation process, besides the sharing capacity and the flexibility inherent to the notion of Open Science? Through a case study on a specific Institutional Repository (CNR National Council), the paper proposes a conceptual framework to address the research questions, which is supposed to contribute outreach purposes, evaluation and sharing of science. [ABSTRACT FROM AUTHOR]

This paper proposes a methodological and sociological reflection on the relationship between digital skills and social capital of families, through the description of the activated procedures to assess the digital competency and the analysis of the first results of a pilot survey (already in progress) on a sample of pre-teens in Rome. Specifically, this paper presents the analysis of two of three main objectives of the survey: namely, 1. Planning a methodological path of mapping and systematization of all dimensions and all levels of digital competencies, through the planning and the testing of the first sociological research tool, according to the international and national policies for the digital agenda. 2. Giving back the first picture on the digital competencies of a preteens sample in the informal environment. 3. Deepening the relationship between clusters of digital competencies and the types of social capital connected to them. [ABSTRACT FROM AUTHOR]

The paper, in light of the growing interest that the Third Mission of the University knows at international and national level, delves into the ways in which it has been detected and evaluated by ANVUR in recent years. It shows that this assessment is based on two logics, one of a linearincremental type (more is better) and one of adequacy of the activities carried out face to the needs of the territory and the internal resources of the University. The second appears to be the most useful for the learning function that the evaluation should carry out in order to continuously improve the universities in their positioning into their settlement area. Finally, it suggests some lines of work to use TM assessment for learning purposes. [ABSTRACT FROM AUTHOR]

The essay presents some results of a national research about the perception of the Department's deans of the changes in the governance of universities after L.N. 240/2010 and the introduction of management and evaluation control tools developed by Anvur. This paper focuses on the changes in resources for universities and departments and on criteria used in universities for the distribution of resources between departments, as perceived by Department Directors. The perception and the evaluation of these changes, by central players in the universities, appears to be very informative about the outcome of the reforms and the problems associated with them and offer elements of reflection about the uses of the tools and results of the evaluation, because of the relevance that this issue is gaining in the scaling back of resources for the University. Indeed, while 'macro' data are enabling the detection of the impact that VQR has had on different universities, empiric findings on its impact in universities, among different departments, are still modest. [ABSTRACT FROM AUTHOR]

The research question in this paper is what the role of evaluation could be to improve the ability of universities to affect territorial development. Universities are supposed to be the driving force of territorial development in a global perspective, which therefore includes both local interests and the wider international and European context. What elements of the current institutional assessment can be used or valorized, and what other forms of assessment could be proposed for this purpose?. The aim is to discuss the concept of diversity of the national university system, trying to identify the potential inherent in the assessment, to avoid that diversification processes based on principles of competition and marketization can become marginalization factors of the universities located in less economically developed regions such as those located in southern Italy. [ABSTRACT FROM AUTHOR]

This paper illustrates how the selection and training of experts for the external evaluation of schools takes places. According to the typology proposed by Bezzi (2007), our choice was on the one hand towards a competent evaluation, conducted by professionals with high experience in the school sector, on the other hand towards an expert evaluation, carried out by social sciences experts with evaluation skills. These professionals work in teams in order to maximize the skills they possess. The National Institute for the Educational Evaluation of Instruction and Training (INVALSI) has invested in their professional development and has sought to identify guidelines by which to offer training courses aimed at enhancing strategic skills, such as those related to the field data collection and those for feedback. Furthermore, INVALSI will set up a process of further training in order to support the attainment of a professional expertise. [ABSTRACT FROM AUTHOR]

The guiding hypothesis of the paper assumes that big data (BD) started affecting evaluation practices (formal and informal) in / of planning. The delay in developing 'public internet' for the benefit of the platforms prevents knowledge from being recognized as a common good. But, we must not underestimate the fact that the individual data in a social practice such as planning, have little value if not aggregated and aimed at offering innovative, and not only marketable, services. BDs open up new possibilities, but they also raise new issues of a cognitive, regulatory and decision-making type, as well as ethical and deontological. Four case studies provide evidence in this regard, particularly in the domains of mapping, mobility management and participation in public policies. These evidences help to identify relevant criteria for the evaluation of BD contributions in planning processes. [ABSTRACT FROM AUTHOR]

In this paper we present a self-evaluation model of the research production of a scientific community. This model is based on a case study: an analysis of the publications of the academic staff at the Department of Communication and Social Research (La Sapienza, University of Rome) from 2011 to 2014. In the first part of the paper, we describe the publications production from a quantitative perspective; then, we introduce more qualitative aspects considering an index of editorial ranking of the publications. In both sections, the synchronic analysis is combined with a diachronic analysis, in order to identify trends. We believe that this model will be useful both for the research institutes to evaluate outputs and outcomes of their policies, and for the researchers to benchmark their publication style against the scientific production of their communities. [ABSTRACT FROM AUTHOR]

The paper is a passionate "reaction" to the evaluation system designed and implemented of the ANVUR in recent years. The main objective of this paper is to argue that the culture of evaluation in Italian universities producing irrational economic, cultural and social effects on the universities. The central idea of this paper is that the quality of the university is closely linked to the quality of its evaluation process, in the essay critically discusses the reasons for the decline of the university also to stimulate the attention of sociologists on the processes of change occurring after the 'ANVUR' and encourage them to study with a "sociological eye" some effects of the evaluation system on the quality of the university, on the scientific research and on the categories of subjects that animate. [ABSTRACT FROM AUTHOR]

The paper describes a research performed by the Department of Anthropological Sciences for the Regione Liguria, finalized to the definition of a set of indicators for the distribution of the Regional Social Fund, feeding a consistent part of Liguria's social policies, governed by the Regional Social Plan and by Local Plans. The research is inserted in a setting of quite rigid normative and programming indications, even if, as it will be described, have taken places interesting retroactions between scientific results and the re-thinking of adopted criteria (or of their practical translation) by the decision maker, considering the results obtained though the operationalization of pre-defined criteria. These aspects have been recently observed by the Italian theoretical framework (Parra Saiani, 2009; Bezzi, Cannavò e Palumbo, a cura di, 2010), being always characterized by a proficient mix between descriptive-interpretative use and practical function. In these cases it is mainly used the Lazarsfeld sociological approach (Lazarsfeld, 1965), such as the selection of principal dimensions of the concept and the successive realization of variable (indicators) permitting to observe such dimensions. The research has maintained a certain degree of continuity with the Lazarsfeld's paradigm, even if opening to some critical reflections about it. More specifically, it emerges a continuous retroaction between indicators, dimensions and indicated concepts, that in their evaluative use produces also a wider consciousness of practical implications of adopted choices, both on the methodological plan, both on the political one. In this way, it is obtained a result that produces innovations and empowerment both on classic working models in social research, both on interested decisional processes. [ABSTRACT FROM AUTHOR]

The impact assessment of social and labour policies is mainly based on direct inquiry on their recipients. In the case of net impact assessments, one has to observe a group of untreated individuals that is very homogeneous with respect to the treated. However, the very high cost of direct surveys limits their scope, as it prevents large sample size, hindering the reliability of estimates. The access to databases created for different purposes (fiscal, administrative, monitoring) could overcome the above limitations, with much convenience in terms of cost and data quality. This paper presents a validation exercise based on a very broad and complex employment database, the Comunicazioni Obbligaorie (COB), that collects employers' compulsory notifications in the case of changes in a job contract. This exercise was possible thanks to an extraordinary condition, giving contemporaneous access to two different data sources, the COB database and a direct survey on recipients of vocational training policies in the Piedmont Region. The paper discusses the main differences between employment indicators evaluated on direct survey data and on COB data. Inconsistencies involve the 20% sample. In addition, such inconsistencies do distort the net impact assessment, since the key variables for placement outcomes vary significantly. Hence, whenever administrative databases are used as a source for socioeconomic analyses, extreme caution and critical thinking must be used, verifying the reliability of estimates deriving from their use and comparing results with other sources. In the lack of such preliminary validation, which should be carried out in close collaboration with the regional or national authorities managing the information systems, we run the risk not only to uncritically accept information producing systematic bias, but also to prevent the development of procedures that are essential for the system development. [ABSTRACT FROM AUTHOR]

This paper proposes an evaluation study related to one of the most used instruments in the field of active labour policies: public incentives for the creation and start-up of new youth enterprises. The case study analysed refers to a specific law of the Umbria Region in support of youth entrepreneurship (Regional Law No 12/1995), with the aim of providing an overall picture of the main results achieved, both from an implementation point of view and in terms of effectiveness on its recipients. Through a reporting approach, regional policy is examined from the point of view of the operation of the instruments implemented. In this case, the analysis is aimed at verifying the extent to which potential recipients have recourse to regional funding and what are the main critical issues that emerged during the implementation of the intervention. With the counterfactual evaluation, we try to judge the ability of the policy to influence the survival of the companies financed, identifying the type of companies in which the major effects of the law are displayed. [ABSTRACT FROM AUTHOR]

The paper describes a research work about evaluation of the acted and desired quality of professional activity within the school for foreigners of the multiethnic cultural center "La Tenda" in Milan. Competences useful for the professional development are the fulcrum of this evaluative research that has tried, not only to describe what implements the studied quality, but also to highlight the premises and consequences. In addition to the skills necessary for the professional mission, we therefore refer to the pre-requisites for carrying out the task: personal (eg: education), material (eg structural endowment) and intangible (eg: the ethical principles of the school). We refer to the facilitating elements and the consequent result that can be achieved by the work that the school teachers carry out (learning Italian and integrating with the new culture). [ABSTRACT FROM AUTHOR]

Supporting youth-led projects is part of the youth policy implemented by the Council of Europe and the European Commission since the early 90s. Taking an active role in a project is both an empowerment experience and a contribution to the development of the own community. The paper presents an ex-post evaluation of a programme for youth-led projects in the South of Italy. The evaluation study focused on the sustainability of the new youth organizations established thanks to the programme, namely their continuity after 4 years from the first annual project funded by the programme. Research methods included a survey of young project leaders and in-depth interviews aimed at exploring forms, barriers and facilitators of the sustainability paths taken up by the youth groups. [ABSTRACT FROM AUTHOR]

This paper presents the results of a study on the implementation of the Youth Guarantee plan for Neets in Lazio. It illustrates the social profile of users by means of indicators provided by Anpal and aims to compare it with the characteristics of Neet population of Lazio, through data analysis of the RCFL Istat. Paradoxically, our results indicate that the program has showed greater difficulty in involving the most disadvantaged youths. On the basis of semistructured interviews with public service workers and directors of 12 Employment Centers, we formulated hypotheses on the self-selection and selection mechanisms that would operate in the programme. Measures of the Plan would work as a catalyst, that is able to increase opportunities and to reduce the duration of unemployment of users who display more resources and skills beforehand. On the other hand, the contribution to reducing the risk of social exclusion of socially disadvantaged Neets seems limited. [ABSTRACT FROM AUTHOR]

The paper critically reviews the methods used for the evaluation of antypoverty programs in Italy to reflect on the relationship between evaluation and learning. The article analyzes the main methods found in practice, and examines the main research questions these methods answered, what results they produce and what problems they face once they were applied. [ABSTRACT FROM AUTHOR]

Moving from some critical issues that characterized the management of the product-reviewer matching in the VQR 2011-2014 this article proposes an operational solution that allows improving the consistency of the attribution process, by bringing the gap between the reviewer's skills and the evaluand. This is achieved through the use of text analysis techniques applied to big corpora, made by the publications/abstracts of the actors involved. The paper is structured as follows: an introduction about the use of Big Data in the context of the Italian university system evaluation; then, the main problems faced in the last VQR exercise will be presented; finally, a textual analysis procedure, designed to provide univocal labels, both for the products under review and for those belonging to referees, will be presented in the last paragraph. [ABSTRACT FROM AUTHOR]

The use of the Big Data in social sciences has raised a number of very delicate questions both in terms of access and in the management strategies of these sources of information. In particular, Big Data has posed a challenge mainly to data analysis from a research, decision and evaluation perspective. The aim of the paper is to illustrate the role of data visualization in assisting the process of data analysis and communication. After a brief introduction to the reasons behind the growing interest in this area and the importance of seeking new forms of data representation, attention will focus on two of the key components: the analytical component, oriented in the analysis of the data, and the communicative component, focuses on the forms of dissemination of the information. Analyzing and communicating data are the elements of the understanding process on which data visualization can make an important contribution. [ABSTRACT FROM AUTHOR]

The "Standardized Score" indicator for University Departments based on the research assessment exercise VQR 2004-2010. In this paper the authors describe an indicator (called Standardized Score) for the evaluation of university departments based solely on the quality of the research-related products presented in the framework of the Italian research assessment exercise VQR 2004-2010. This indicator is built by normalizing with the Italian disciplinary sectors (SSD), thus taking full advantage of the information collected by ANVUR. It has the advantage of comparing and sorting, according to research quality, departments very different form each other both in terms of number of members and/or disciplinary sectors (SSD) thereof. This indicator is complementary to those provided by ANVUR according to the Ministry rules, for it provides support to different needs and it is not normalized over research areas (Aree CUN). The paper describes the concepts behind the Standardized Score indicator, its characteristics, the questions it aims to answer and the possible limitations of its application. [ABSTRACT FROM AUTHOR]

The intense reform process that invests the Italian higher education and scientific research system is hereby considered in the light of the concept of accountability. Within that framework, the paper addresses the definition of accountability along with the associated principles of transparency, responsibility and responsiveness. The different logics and forms of accountability are thereafter discussed by highlighting the difficulties which arise with respect to the necessary link between accountability rhetoric and organizational models. The author argues that the present situation of widespread discomfort within Italian universities is related to the disconnection between prevailing types of accountability and sense of the reform process. The paper ends by proposing the concept of social accountability as a possible way to be explored in order to overcome the difficulties highlighted in the previous paragraphs. The principle is briefly presented. Some considerations are introduced as to its potentials but also as to some possible limitations [ABSTRACT FROM AUTHOR]

The paper presents an impact assessment of vocational training courses, which the Piedmont Region co-financed by the ESF, both discussing the methodological feasibility and proposing a quasi-experimental evaluation strategy based on the medium-term job placement of vocational training students. The authors illustrate the operational design and the implementation of the assessment, with particular attention to the identification of a proper control sample, finally identified on the basis of the course noshows and drop-outs. The paper proposes an accurate description of gross and net impact evaluation strategies. The gross impact evaluation is carried out through an analysis of the employment outcomes in the mid-term, while the net impact is estimated through net employment differentials between the main and the counterfactual sample, as well as through a multivariate probit analysis, which investigates the effects of individual characteristics on the probability of being employed a year later the end of the course. The selection bias problem is discussed, illustrating the strategy adopted to identify and counteract its effects. In conclusion, the authors distill this research experience through a series of lessons learned about both the methods and the evaluation assessment of training effectiveness. [ABSTRACT FROM AUTHOR]

In recent years, the question about the role of evaluation research concerning the local social policies planning has gained more and more centrality into the national debate. This paper intends to give an account of the results obtained from a survey on social-health service users satisfaction (realized within a participatory evaluation research) provided in an Area Plan in northern Sardinia. The author argues that the detection of the user satisfaction of socialhealth services can be understood as the first step of a participatory evaluation procedure that helps and supports the analysis of the planning process of the Area Plan. The main aim of the user satisfaction survey within the participatory evaluation research is to provide useful information to the drafting of the next three-year planning document of the Area Plan. [ABSTRACT FROM AUTHOR]

This paper focuses on the potentiality of the sequence analysis as a tool to study and better understand the university students' career paths. Compared to the cross-sectional studies, the sequence analysis allows not only a more correct approach to the phenomena of the dispersion within tertiary education, but also an immediate return of the results. Also, through its features, sequence analysis helps the marketability of these results for comparative and/or evaluation purposes. The holistic view lets administrative information into diachronic sociological data. This work shows the suitability of the sequence analysis tools for studying the complexity of the Italian higher education students' careers, also analysing a case study: the Sapienza University of Rome first time enrolled in AY 2001/2002. [ABSTRACT FROM AUTHOR]

In the context of programming of the European Structural and Investment funds 2014-2020, this paper aims to highlight the positive aspects of the application of financial instruments to be use in substitution of classical subsides. The authors present the case study of ex ante evaluation of the Revolving Fund of the Autonomous Region Friuli Venezia Giulia in the Rural Development Programme 2014-2020. The evaluation was carried out in the last half of 2015. In the article are listed, starting with the reference context and the specific evaluation questions, methods and analytical techniques used to get to demonstrate the validity and use of the opportunities of Financial Instrument. [ABSTRACT FROM AUTHOR]

The aim of the paper is to explain what are the determinants of Tuscan municipalities' absorption capacity of European Structural Funds. For this purpose we use a hurdle model which consists of two parts. Firstly, a binary logistic model is used to estimate the probability that a municipality gets at least one project financed with EU structural funds. Secondly, a regression with negative binomial distribution is estimated only for municipalities with at least one project financed, in order to identify the determinants of the number of projects. The results show that the probability of access to at least one European project is mainly influenced by three factors: the demographic size of the municipality, the experience in previous programming cycles and by the geographic area. When it comes to the determinants of the number of projects financed (for those municipalities which got at least one project financed), other factors play an important role. One of these is the adequacy of civil servants both in terms of size and quality, the latter measured by the incidence of graduated civil servants in the municipality. Also economic and financial factors impact on the number of projects financed at the municipality level: the amount of revenues has a positive effect while the presence of deficit and Internal Stability Pact constraints seem to have a negative impact. Even political stability finally appears to have a positive role in the municipality absorption capacity of Structural Funds. [ABSTRACT FROM AUTHOR]

The paper analyzes, in a diachronic perspective, the sociological scientific production of Sapienza University. The type and languages of publication, and the co-authorship phenomenon are examined from 2004 to 2014, assuming that the implementation of the policies of evaluation of research ("VTR", "VQR") and regulatory changes on careers (comparative evaluations known as "ASN") had an effect on the characteristics of the scientific production. The explorative analysis here presented, with all the limitations of a case study, shows (through the data extracted from the scientific production repository used by this University) different trends of changes in the characteristics of the contributions and in the different ways of publishing, in the decade 2004-2014. The results provide an interesting framework on sociological scientific production and - in relation to the implementation of the evaluation policies - evidence on the effects that the evaluation processes are producing on scientific communication. [ABSTRACT FROM AUTHOR]

This paper aims to describe the experience carried out in the Rural Development Programme (RDP) of Tuscany Region for the period 2007-2013 in the assessment of the activities implemented by the Local Action Groups (LAG) using a participatory approach. This assessment method has been adopted to stimulate awareness of the LAGs as local development agents and enhancer of the Social Capital of the territories in which they operated. Through an evaluation process well-structured step by step, the self-assessment tried to maximize the acquisition of information from the individual LAG for the RDP evaluation at regional level and to strengthen their empowerment and reliability as agents of rural and local development. [ABSTRACT FROM AUTHOR]

The paper presents a reflection on the policy of cost-cutting and spending review in the Italian public sectors. The thesis is twofold. First of all, the paradigm used by the spending reviews' policies is based on the macro-economic theory that doesn't take into account the complexity of the implementation processes. Secondly, the weakness of the administrative technical bureaucracies in coordinating modernisation processes produces an incoherent diffusion of innovations. The paper discusses, as empirical examples, two relevant national programmes in the field of the judicial sector; these programmes are mobilising an interesting governance network at the territorial (local) level and a vast number of projects in specific territorial areas. The implementation of these interventions shows, however, that many judicial offices of the Nation lack the necessary resources and capabilities to sustain modernisation projects and, at the same time, the difficulty of central bureaucracies to oversee and safeguard a coherent fulfilment of the diffusion objectives, due to the fragility of technical bodies. [ABSTRACT FROM AUTHOR]

This paper argues that, despite a very large volume of management reform in the EU over the past quarter century or more, our knowledge of the impacts of these efforts is patchy and fragile. It calls on a number of key studies which each, in a different way, carries this same basic message. In the second part of the paper the reasons for this state of affairs are analyzed. One set of reasons is concerned with the considerable methodological difficulties of evaluating major reforms, which are commonly complex, evolving and spread out over time. A second set of reasons concerns the limited incentives for politicians and top officials to take a sustained interest in evaluation. A concluding reflection is that, while this basic situation is not likely to change, there are nevertheless ways in which the academic contribution to reform evaluation can be strengthened. [ABSTRACT FROM AUTHOR]