Meta-evaluation and evaluation standards in social policy

Meta-evaluation is an “evaluation of evaluations” to improve future evaluation work. As a highly relevant topic for professionals working in social policy, the third module of the virtual Bridge-Building Summer School of Evaluation in Social Policies (August 25-27, 2021) was about meta-evaluation and evaluation standards in social policy.

15/9/2021: By Rahel Kahlert, European Centre

Meta-evaluation is an “evaluation of evaluations” to improve future evaluation work. As a highly relevant topic for professionals working in social policy, the third module of the virtual Bridge-Building Summer School of Evaluation in Social Policies (25-27 August 2021) was about meta-evaluation and evaluation standards in social policy. This module was one of eight modules covered by the Summer School, which combined capacity building by enhancing knowledge and information sharing on the state-of the-art research and policy practices, exchanging shared experiences among participants, and developing ideas and proposals for “real-world” projects. “Real world” refers to the idea that evaluations take place under budget, time, data, and political constraints (see Bamberger & Mabry, 2019). One central focus was the interactive, hands-on small-group work with practical exercises to foster peer learning. Eye level interactions were core building blocks. The thirteen participants shared their expertise in either conducting or commissioning evaluations. The participants were social policy professionals from both the public and private sector in the Western Balkan region and Eastern Partnership region (in particular, Azerbaijan, Albania, Bosnia and Herzegovina, Kosovo, Moldova, North Macedonia, and Ukraine).

The module on meta-evaluation and standards started with a short introduction about meta-evaluation by guest speaker Thomas Vogel, head of programmes at Horizon 3000, the largest Austrian non-governmental development cooperation organisation, where evaluation plays a key role for learning and accountability. Mr. Vogel introduced the difference between evaluation synthesis and meta-evaluation. Evaluation synthesis analyses a series of evaluation reports and aggregates findings on a higher level such as programme, country, or sector level. The aim is to find out how programmes or projects can be improved. Meta-evaluation also analyses a series of evaluation reports, but then analyses the quality of each evaluation. The aim is to find out how evaluations can be improved in the future. Meta-evaluation can enhance the quality in evaluation by using and applying evaluation standards for all levels of quality including scope, methodology, independence of evaluators, utility etc.

Rahel Kahlert then provided a brief input on widely used standards in evaluation such as the UNEG Norms and Standards (updated in 2016) to be upheld in the conduct of any evaluation. Professional evaluation associations such as the German Evaluation Society or the Swiss Evaluation Society developed their own standards for evaluators to apply in their daily professional work. Common features include accuracy (methodological and technical standards), propriety (legal and ethical standards), utility (serving information needs of intended users), and feasibility (realistic, cost-effective evaluation). As evaluators, we have to remind ourselves that we are dealing with persons and intervene in a setting. Therefore, it is important to follow ethical guidelines and protect human participants similar to any research setting.

The small-group breakout sessions dealt with the question: “How can standards enhance the quality of an evaluation?” The teams of 4 to 5 persons selected two to three evaluation standards and then applied them to a “real-world” setting. The teams used the online pad tool Google JamBoard (see Figure 1) to make their case by adding post-its. For all three teams, utility stood out as important evaluation standard. Involving stakeholders was regarded as a means of enhancing utility, i.e. improving the uptake of findings. This may sometimes conflict with the standard of independence, when donors or programme staff would like to influence evaluation findings (“interested only in the good stories (positive findings)”). Accuracy can be enhanced through triangulation and cross-referencing data, which ultimately also can improve utility and uptake.

Summarily, the module on meta-evaluation and standards was a lively, interactive experience with all participants engaged, and set the scene for further productive interactions. At the end of the Summer School, participants expressed their intention that they will apply evaluation standards in their evaluation work more strongly and engage in meta-evaluation in the future to ensure evaluation quality.

Closed (27/08/21): Get Balkanized Internship Programme

Since 2019, the University of Applied Sciences Bonn-Rhein-Sieg has been offering the Get Balkanized Internship Programme. They are optimistic to fully resume the programme in 2022 and fund up to 20 vacancies in the partner countries Albania, Bosnia-Hercegovina, Kosovo and Serbia.

Closed (27/08/21): Get Balkanized Internship Programme

Since 2019, the University of Applied Sciences Bonn-Rhein-Sieg has been offering the Get Balkanized Internship Programme. They are optimistic to fully resume the programme in 2022 and fund up to 20 vacancies in the partner countries Albania, Bosnia-Hercegovina, Kosovo and Serbia

In the following find the most important information on the programme’s framework conditions. Have as well a look at the cartoon that is summarising the programme, available here.

Please send an email expressing your interest and annex the corresponding vacancy to michael.sauer@h-brs.de by Friday, 27 August 2021

New project: Updated review of Western Balkan economies regarding the EPSR

The European Centre is currently updating the six reviews of the Western Balkan countries regarding the European Pillar of Social Rights (EPSR).

New project: Updated review of Western Balkan economies regarding the EPSR

The European Centre is currently updating the six reviews of the Western Balkan countries: Albania, Bosnia & Herzegovina, Kosovo, Montenegro, Republic of North Macedonia, & Serbia regarding the European Pillar of Social Rights (EPSR). The European Centre team is working with country experts to create high-quality reports to monitor the countries’ achievements and reform efforts of social rights as candidate and potential candidate countries and preparing a comparative overview report for 2021. Within the framework of the Employment and Social Affairs Platform 2 (ESAP 2) project that is implemented by the Regional Cooperation Council (RCC) and funded by the EU. The European Centre was also engaged as the international expert organization in the reviews prepared in 2018/19 and coordinated the 2020 reviews. Read more

15/10/21: CfP – Support to ‘Western Balkans 6’ for strengthening media freedom & fighting domestic & gender-based violence

The purpose of this Call for Proposals is to contribute to reforms processes underpinned by EU integration by supporting civil society actors in the ‘Western Balkans 6’ countries – Albania, Bosnia and Herzegovina, Kosovo, Montenegro, North Macedonia, and Serbia.

15/10/21: CfP – Strengthening civil society capacities & resilience in Armenia – The EU Delegation to Armenia 

The Balkan Trust for Democracy’s financial and technical assistance for this sub-granting process is part of the larger Norwegian efforts to support Albania, Bosnia and Herzegovina, Kosovo, Montenegro, North Macedonia, and Serbia.

THE PURPOSE OF THE CALL

The purpose of this Call for Proposals is to contribute to reforms processes underpinned by EU integration by supporting civil society actors in the ‘Western Balkans 6’ countries – Albania, Bosnia and Herzegovina, Kosovo, Montenegro, North Macedonia, and Serbia.

This sub-granting scheme of the Balkan Trust for Democracy (BTD) is being realized with the support of the Norwegian Ministry of Foreign Affairs and the Royal Norwegian Embassy in Belgrade.

Support is given to two priority areas:

I. Media Freedom
II. Fight against Domestic and Gender-based Violence 

This is an open call, and project proposals will be assessed continuously from 5 August 2020 up until 15 October 2021. Read more

Foster care in Albania

Referring to the latest annual report of State Social Services, during 2020 15 children from the public residential institutions of children in Albania have been reunited with their biological families, while 27 children have been adopted. The national program for the establishment of the new foster care service is institutionalized since 2008 in the Strategy of social protection and the action plan for its implementation.

By Merita Xhumari and Megi Xhumari, University of Tirana

There are approximately 700,000 children in Albania, representing a dependency ratio of 25.0 children per 100 working age population. Close to 0.12 per cent (1.2 per 1,000) of the child population lives in residential institutional care, as reported by the State Inspectorate for Social Services in June 2016. Referring to the latest annual report of State Social Services, during 2020 15 children from the public residential institutions of children in Albania have been reunited with their biological families, while 27 children have been adopted. The national program for the establishment of the new foster care service is institutionalized since 2008 in the Strategy of social protection and the action plan for its implementation. Law 18/2017, “On the children’s rights and protection” defines the institutional mechanisms for the protection of children’s rights at central and local level. These developments reflect the commitments of the Albanian government towards the United Nations Sustainable Development Goals 2030, Objective 1.4 Article 20: “Children deprived of their family environment are entitled to special protection and assistance from the State which provides… an alternative care” in the National Agenda for Children Rights 2017-2020. Albania’s EU integration process also requires the development of policies and services according to Principle 11 “Care and support for children” of the European Pillar of Social Rights.

In order to support alternative care for children, SOS Children’s Villages Albania in partnership with Tirana Legal Aid Society (TLAS) have been implementing the project “The Development of CSOs for foster care in Albania”, financed by European Union. The objective of the project was to: “Support good practices and models of protection of children from abuse and violence, alternative care, family strengthening, inclusive education and early childhood development with a special focus on vulnerable & minority groups”. The project started in April 2018 and ended in March 2021, implemented in 3 main regions of Albania, Tirana, Durrësi and Korça.

Despite the fact that in the last years, de-institutionalization and the development of family-based services and foster care have become a priority for governmental agendas, the accessibility to a range of care options, including foster care, has not really been prioritized and residential cares prevails and continues to account for the highest number of children in alternative care. The main challenges faced that were identified in the course of the project implementation on foster care have been the lack of information and public awareness about this new family-based service. In Albania, the placement of children in kinship families has traditionally been applied without any court process. On the one hand, it has been the mentality of biological parents to leave their children in residential care rather than in foster care families, as they fear they might lose their child forever. On the other hand, there is the mentality of Albanian families, who want to have a child permanently as part of the family, to better decide for adoption rather than have a foster family that provides a temporary care for children without parental care. Furthermore, there are also macro level challenges related to the capacities of local government, courts and other local actors that need to be developed to be able to manage the new foster care service.

The key recommendations drawn from the project implementation include the actions to increase public awareness on foster care, the creation of a juvenile court, free legal service for foster families; the establishment of better synergies across institutions; and for foster family to be recognized as a profession and to have an adequate payment.

In support of these recommendations, one of the main outputs of the project was the development of a Practical Training Manual for Foster Care in Albania by Bethany Social Services, the first local NGO piloting foster care service in Albania, to guide local professionals and key stakeholders in foster care service implementation. The Manual published in the Albanian language can assist professionals from Kosovo, North Macedonia, as well as Montenegro where a considerable Albanian population is living.

The central axis for addressing the main topics of the Manual is the United Nations Convention on the Rights of the Child ratified by the Albanian government, which stipulates that “… the child, for the full and harmonious development of his / her personality, must grow up in a family environment, in a happy atmosphere, with love and understanding”, as well as the UN Guidelines for Alternative Care for Children.

Content-wise, the Training Manual aims to provide knowledge, develop skills and competencies related to:

  • The value and importance of the family for children;
  • Laws, regulations and policies governing the foster care service;
  • The process of implementing the foster care service;
  • Categories of competencies for the foster care service;
  • Commitment to the role as a member of a professional Cross-cutting Technical Group.

Closed (31/05/21): Call for partnerships & scholarships – APPEAR

APPEAR contributes to achieving the Sustainable Development Goals (SDGs). The overall objective is to strengthen the scientific foundation and institutional capacities in higher education, research andmanagement in the partner countries through academic partnerships with Austrian higher education institutions and master’s and PhD scholarships.

Closed (31/05/21): Call for partnerships & scholarships in higher education – Austrian Development Cooperation (ADC) APPEAR

 APPEAR contributes to achieving the Sustainable Development Goals (SDGs). The overall objective is to strengthen the scientific foundation and institutional capacities in higher education, research andmanagement in the partner countries through academic partnerships with Austrian higher education institutions and master’s and PhD scholarships. Furthermore, APPEAR aims at increasing the commitment to international cooperation and development at Austrian higher education institutions.

APPEAR is a programme of the Austrian Development Cooperation (ADC) with the aim to realise its strategy for support of higher education and research for development on an academic institutional level.  Read more

Guidelines for Applicants

Online article: Every Child Matters? Ambivalences and Convergences in Migration Management and Child Protection in Albania

Online article: Every Child Matters? Ambivalences and Convergences in Migration Management and Child Protection in Albania by Vathi et al.

Research on the intersection between migration management and child protection is limited and, yet, contradictions between these two domains and ambivalence in the area of child migrants’ rights across the developed world have already been highlighted. Based on fieldwork with policymakers and service providers in Albania, a middle-to-high income country with a significant history of emigration, this paper aims to shed light on the interaction of these institutional and policy domains and the impact they have on professional practice. The State’s impact on migrant children’s rights at the domestic level is affected by the resourcefulness of its system of social and child protection. Read more

Closed (30/12/20) CfP: Welfare state and social policies in the CEE region

The peer-reviewed, open access scientific journal Problemy Polityki Społecznej/Social Policy Issues invites original paper submissions for a special issue to bring together papers exploring a broad array of research questions related to welfare state and social policies in the CEE region.

Journal of Social Policy Issues: Cfp on Special Issue “Welfare state and social policies in the CEE region: past developments, present challenges and prospects for the future”

Deadline: 30 December 2020

University of Warsaw invites to submit original papers to the special issue of Problemy Polityki Społecznej. Social Policy Issues – a peer-reviewed, OPEN ACCESS journal, published by University of Warsaw.

The planned Special Issue “Welfare state and social policies in the CEE region: past developments, present challenges and prospects for the future” aims to bring together papers exploring a broad array of research questions related to welfare state and social policies in the CEE region, that may touch upon past developments, present challenges as well as future prospects. By CEE region we mean a large group of countries including Albania, Belarus, Bulgaria, Bosnia and Herzegovina, Croatia, Czechia, Estonia, Hungary, Kosovo, Latvia, Lithuania, Montenegro, North Macedonia, Poland, Romania, Serbia, Slovakia, Slovenia, and Ukraine as well as sub-country regions, e.g. east of Germany.

The best paper submitted to this issue will receive an award of 400 euro. Read more

07/12/20: Online project presentation: Intra Yugoslav Albanian migration

07/12/20; 17-18: To the Northwest! Intra Yugoslav Albanian migration (1953-1989)

In this talk, CSEES researchers Rory Archer and Mladen Zobec present the
research project “To the Northwest! Intra Yugoslav Albanian migration
(1953-1989)” funded by the Austrian Science Fund (FWF) which began earlier this year. The research examines the phenomenon of Yugoslav Albanian migration from the Yugoslav south (Kosovo, Macedonia) to the Northern republics of Slovenia and Croatia. Read more

2020 Updated Reviews: on the European Pillar of Social Rights in the Western Balkans

2020 Updated Reviews: on the European Pillar of Social Rights in the Western Balkans

The European Centre updated the existing reviews on the European Pillar of Social Rights in the Western Balkans: Albania, Bosnia & Herzegovina, Kosovo, Montenegro, Republic of North Macedonia, and Serbia. The European Centre had already been engaged as international expert organization in the original reviews prepared in 2018. On behalf of the Regional Cooperation Council, the European Centre team was working with country experts and provided quality assurance to ensure that the reports were of high quality, consistent, clear, accurate, sourced, appropriate and timely. Read or download the six reports for 2020 here.