R4D Pakistan

Do Value-Added Estimates Add Value? Accounting for Learning Dynamics

Fri, 14 Jun 2013 08:02 GMT

Production Year: 2011

Citation: Andrabi, T.; Das, J.; Khwaja, A.J.; Zajonc, T. Do Value-Added Estimates Add Value? Accounting for Learning Dynamics. American Economic Journal: Applied Economics (2011) 3 (3) 29-54. [DOI: 10.1257/app.3.3.29]

Summary: This paper illustrates the central role of persistence in estimating and interpreting value-added models of learning. Using data from Pakistani public and private schools, we apply dynamic panel methods that address three key empirical challenges: imperfect persistence, unobserved heterogeneity, and measurement error. Our estimates suggest that only one-fifth to one-half of learning persists between grades and that private schools increase average achievement by 0.25 standard deviations each year. In contrast, value-added models that assume perfect persistence yield severely downward estimates of the private school effect. Models that ignore unobserved heterogeneity or measurement error produce biased estimates of persistence.

Providers of policy research (except for education) database.

Fri, 26 Apr 2013 13:47 GMT

Production Year: 2013

Citation: Sustainable Policy Development Institute (SDPI). Providers of policy research (except for education) database. Sustainable Policy Development Institute (SDPI), Islamabad, Pakistan (2013)

Summary:

This database comprises 4 Excel spreadsheets:

Database of organizations of policy research (all themes excluding education) by province and city
List of organizations consulted for the study
List of providers of policy research
List of acronyms

Educational policy research database.

Fri, 26 Apr 2013 13:30 GMT

Production Year: 2013

Citation: Sustainable Policy Development Institute (SDPI). Educational policy research database. Sustainable Policy Development Institute (SDPI), Islamabad, Pakistan (2013)

Summary:

This database comprises 4 Excel spreadsheets:

List of Providers of Educational Policy Research
Educational Policy Research Centres at the Institutes of Higher Education
Key Donors for Education Policy Research
Organizational Database of Educational Policy Research by Province

Paper 3. Landscaping the Institutions of Educational Policy Research in Pakistan: Who is doing what?

Fri, 26 Apr 2013 13:11 GMT

Production Year: 2013

Citation: Arif Naveed. Paper 3. Landscaping the Institutions of Educational Policy Research in Pakistan: Who is doing what? Sustainable Policy Development Institute (SDPI), Islamabad, Pakistan (2013) 79 pp.

Summary: This paper presents an overview of the institutions of educational research in Pakistan. It is structured into six sections. After the introduction, the second section describes the methodology adopted for this landscaping. The third section identifies the major actors in educational policy research briefly describing their research interests, and wherever possible, the methodological and disciplinary orientation and geographic focus of their research. Given the absence of the institutes of higher education in the research landscape, the fourth section discusses the tremendous yet untapped research potential of the educational faculties of universities across the country. The fifth section discusses the major sources of funding available for conducting policy research on the issues of education. The final section provides suggestions to strengthen the policy research on educational issues in Pakistan. The information about the key research organizations, their institutional capacity and some examples of their projects/publications and contact details are provided in the annexes along with the list of organizations interviewed

Paper 2. Landscaping Policy Relevant Research in Pakistan: Identifying the Key Actors

Fri, 26 Apr 2013 09:40 GMT

Production Year: 2013

Citation: Arif Naveed. Paper 2. Landscaping Policy Relevant Research in Pakistan: Identifying the Key Actors. Sustainable Policy Development Institute (SDPI), Islamabad, Pakistan (2013) 107 pp.

Summary:

This paper is the first attempt to landscape the policy relevant research institutions in Pakistan. In addition to the paper on political economy analysis, it provides a broad overview of the institutions engaged in policy research in the country, their thematic priorities, geographic concentration and sources of funding available to them. It attempts to identify the major challenges and opportunities for strengthening policy relevant research in Pakistan.

This paper is organized thematically and the significant actors in policy research under each theme are listed along with a brief description of their priorities. It also provides an overview of the sources of funding available to the institutions of policy research in Pakistan.

This paper is structured in three sections. After the introduction, the second section presents the landscape of key policy research organizations and the focus of their research under each of the seven selected themes. The third section presents an overview of the sources of funding available for policy research and the fourth section summarises the analysis and provides some key recommendations.

Paper 1. Architects and Contractors: Political Economy Analysis of Policy Research in Pakistan

Fri, 26 Apr 2013 08:59 GMT

Production Year: 2013

Citation: Wood, G. Paper 1. Architects and Contractors: Political Economy Analysis of Policy Research in Pakistan. Sustainable Policy Development Institute (SDPI), Islamabad, Pakistan (2013) 59 pp.

Summary:

This study was undertaken as an assignment commissioned by the Research and Evidence Division, DFID London in conjunction with DFID-Pakistan. Its immediate purpose is to provide DFID with a database of policy relevant research activity in the country through mapping the overall landscape, as well as a political economy analysis (PEA) of the research to policy interface. This paper represents the PEA part of the assignment.

This study draws upon large qualitative data gathered from three types of actors: (a) research organizations/groups, including academic institutions and think-tanks in the public sector, non-government or private sector; (b) public sector organizations such as Planning Commission, ministries and departments, engaged in policy making; and (c) key international donor agencies often engaged in commissioning research and supporting policy processes. Data from over 100 institutions, supported by interviews in most cases, was collected during September-December 2012

Key messages:
1. The decentralisation of many government functions and services (i.e. the 18th Amendment to the Constitution) moves the Planning Commission’s role away from detailed command planning towards more indicative planning, with a focus upon inclusive growth and developing human capital and infrastructure. This shift will have a profound effect upon the research/policy interface, since so much of it is presently Islamabad focussed.
2. Participants in the study suggest the links between research and policy in Pakistan are particularly weak due to high levels of political insecurity and volatility which prompts short-term, highly politicised decision making rather than evidence-based choices and policy.
3. Our study findings indicate that this impact of research on policy is also weak because policy makers do not have the capacity or incentives to absorb complex analysis whether quantitative or qualitative. Policy makers do not, therefore, develop functional networks with researchers.
4. Respondents suggest that although the social sciences are a crucial underpinning of much policy analysis, especially across the sectors identified in this study, the social sciences are seriously neglected in Pakistan, and, outside economics, are especially weak. Qualitative forms of research (e.g. from anthropology) are not valued. Underlying social science capacity is weakened by parental preferences for students to follow more obviously lucrative subjects in engineering, medicine, management and other applied sciences. The resulting quality of teaching and research in the social sciences thereby suffers.
5. Participants in this study said that donors dominate the public policy research space through funding and commissioning, but they tend to have short term, projectised priorities across a range of thematic narratives and thus do not build long term capacity and relationships with the longer term, core development narrative of the country. Thus the work that they sponsor is often marginalised by central planners.
6. Overall, research outlets are relatively few and highly concentrated for a country of this size and complexity. While some of the economics-focussed institutions are closely allied with counterparts in government (especially Ministry of Finance and the Planning Commission), other disciplines and institutes, e.g. in agriculture, nutrition and social policy, tend to be allied with particular programmes like the Benazir Income Support Programme (BISP) rather than a core policy area.
7. There are numerous barriers to undertaking research and its uptake which are summarised in the main text and set out in more detail in the Appendices. Some of these barriers refer to the intrinsic weaknesses of the social sciences, others to socio-cultural sensibilities, and others to the security issues.

A 13-page summary version of the full background paper is also appended.

Helpdesk Report: Evidence Base for Local Block Grants in Health

Tue, 16 Apr 2013 15:27 GMT

Production Year: 2012

Citation: Bolton, L. Helpdesk Report: Evidence Base for Local Block Grants in Health. Human Development Resource Centre, UK (2012) 20 pp.

Summary: Query: Provide a literature / evidence review of the effectiveness or otherwise of sub-national flexible block grants in health in delivering public health or health system objectives in developing countries.

Summary: It was difficult to find examples that met the criteria of sub-national flexible block grants in health. Sections 2 and 3 cover the main two that were identified. These are the Tanzania Essential Health Interventions Project (TEHIP) and Generasi, Indonesia.

TEHIP aims to bring health spending more in line with cost-effective approaches to the local disease burden. Information from Demographic Surveillance Systems allows planners to determine spending priorities. A series of simple management tools enable those district planners to allot funds to interventions that will have a greater impact on local causes of mortality.

PNPM Generasi is an incentivised community block grant program that builds on the architecture of the Government of Indonesia’s (GOI’s) community driven development programme. It uses a facilitated community decision-making process to allocate block grant funds to target 12 health and education indicators. Communities work with facilitators and health and education service providers to improve access to and use of health and education services. To give communities incentives to focus on the most effective polities to target program indicators, the GOI determines the size of the village’s PNPM Generasi block grant for the subsequent year partly on the village’s performance on each of the 12 targeted health and education indicators.

Section 4 covers other examples in Nigeria, Pakistan, Brazil and Kenya.

Effective methods to improve police performance in cities and regions in developing countries

Mon, 25 Mar 2013 16:24 GMT

Production Year: 2012

Citation: Sumedh Rao. Effective methods to improve police performance in cities and regions in developing countries. Governance and Social Development Resource Centre, (2012) 8 pp.

Summary: Query: Identify examples of effective methods to improve police performance in cities and regions in developing (but not post-conflict) countries.

Key findings:

There are few detailed case studies of cities or regions in developing countries that outline effective methods for police reform. This is mainly due to the lack of evaluation that objectively identifies certain types of police reform as having improved performance. It is also true that changes in performance may not be universally judged as an improvement, reform that is effective in one context may not be so in another, and police reform often takes place on a national, rather than regional or city level.

The case studies identified in this study – from India, Mexico, Colombia, Peru and Pakistan – suggest that the reason for improvements in performance has not necessarily been the use of particularly innovative reform policies. Rather there has been a political and social context which has allowed improved practices to take hold. The literature suggests the following to ensure effective police reform:

sustained political and social support for reform;
police leadership;
adequate resources and/or low-cost measures;
good systems of evaluation.

A Model for ICT based Services for Agriculture Extension-Phase-II. Second Progress Report.

Tue, 12 Feb 2013 12:28 GMT

Production Year: 2012

Citation: Mahrukh Siraj. A Model for ICT based Services for Agriculture Extension-Phase-II. Second Progress Report. CABI South Asia, Rawalpindi, Pakistan (2012) 269 pp.

Summary:

This project aims to improve livelihoods for farmers in target states in Pakistan through improved reach and monitoring of extension services to the farming community at a lower cost. This report covers progress in achievement of the following output indicators:

Output 1: A functional MIS for agriculture extension, helpdesk and mobile applications for use by the farmers and extension providers.
The MIS application, consisting of a web interface and mobile survey application was completed and the help desk was established.

Output 2: Content developed for the system.
More than 500 FAQs (Frequently Asked Questions) on the four target crops; cotton, rice, wheat and citrus were developed. This content has been informed by workshops with extension workers, baseline survey and review of extension advisory materials from the Department of Agriculture – Punjab.

Output 3: Training and capacity building of extension services.
Training in the use of the MIS application was provided to local champions from Vehari district. The application will be launched for all staff in Vehari by the end of 2012.

Output 4: Scale-up and sustainability planning leads to development of a robust model for adoption of the service in other states/districts.
A series of communication activities were completed and the project was presented at a seminar ‘What can telecoms do for Agriculture’ arranged by Pakistan Telecommunications Authority. The project was also presented at 12th IT and Telecom Exhibition and Conferences, a premier IT and Telecom event in Pakistan. At both events the project was much appreciated. The events provided an opportunity to network with other possible stakeholders and collaborators.

The report also comprises 4 annexes:

Annex-A Business Analysis V2
Annex-B User Guide E-Zaraat
Annex-C FAQs
Annex-D Baseline Survey Report

A systematic review of the evidence of the impact of school voucher programmes in developing countries

Fri, 01 Feb 2013 10:48 GMT

Production Year: 2013

Citation: Morgan, C.; Petrosino, A.; Fronius, T. A systematic review of the evidence of the impact of school voucher programmes in developing countries. EPPICentre, Social Science Research Unit, Institute of Education, University of London, London, UK (2013) iii + 92 pp. ISBN 978-1-907345-45-6

Summary:

Background
Demand-side financing is based on the principle of governments either channelling education resources through students and their parents, or basing school funding on enrolments or attendance. Regardless of the direct path of funding, demandside financing is viewed as a way of addressing inequities that prevent poor children from continuing their education, as well as a means of introducing school choice. Education vouchers, a demand-side financing intervention involving the public subsidy of private schooling based on the number of eligible voucher students per school, generally aim to expand parental school choice, which is often promoted to increase competition in the school system. Opponents of vouchers argue that private schools do not necessarily provide a higher-quality education; affluent families with more social capital and access to voucher programme information are more likely to find the best schools; and it is very difficult to set up effective systems of accountability to guard against 'cream skimming' and sorting. In this paper, we report on a systematic review of evaluations of education voucher programmes in developing countries.

Methods
Through extensive searching, including electronic keyword searches of bibliographic databases, handsearches of relevant journals, examinations of online holdings of international development organisations and research firms, citation chasing, examining grey literature, and contacting experts in the field, we identified studies that responded to the following question:

What is the evidence of the impact of school vouchers in developing countries?

Eligible studies had to meet the following criteria: The evaluation took place in a low-income developing nation as defined by the World Bank at the time of the intervention; and the evaluation directly assessed the impact of a school voucher programme on participants' educational outcomes. With the intent to conduct meta-analysis, we focused on identifying randomised controlled trials (RCTs) or quasi-experimental evaluations (QEDs) with some evidence that the groups being compared were equivalent. In addition, we sought for contextual information - but not for inclusion in effect size estimates - i.e., quasi-experiments without pre-test group equivalency, and other quantitative and qualitative studies that shed light on implementation and context issues.

Each RCT or QED located in the search that appeared to be a possibility for inclusion was carefully reviewed by two authors and a structured abstract was prepared for each study, detailing the context, methodology and findings. For each study deemed eligible for inclusion following this screening process, a coding instrument was completed that included items in the following areas: researcher and study characteristics, study methods and methodological quality, intervention and control conditions data, participants in the study and outcome data. To evaluate study quality, we recorded details on three key implementation issues: how the groups were equated and whether any problems with equating were reported, information on attrition, and whether the programme experienced significant implementation or fidelity problems.

Results
We identified two studies that met our inclusion criteria - one examining the Colombia PACES programme and the other evaluating the Quetta, Pakistan Urban Fellowship programme. We also identified four quantitative studies on the Chile voucher system that did not meet our criteria for inclusion in effect size estimates but were examined to shed light on possible theory, implementation and context issues. Given the very small number of studies that met our inclusion criteria, we provide the results in a narrative fashion, rather than through meta-analysis. Both the Colombia and Pakistan programmes increased private school enrolment amongst the countries' poorest income groups, thus probably improving equity. The Pakistan programme resulted in girls being educated for less than it would have cost for the government to create public school spaces, while the Colombia programme cost rather more, but will most likely prove cost-effective in terms of long-term economic gains.

Conclusions
Clearly, more rigorous research in developing country contexts is necessary to determine whether the gains from these two programmes can be replicated and enhanced and to elucidate the many issues surrounding vouchers. Pilot programmes employing random assignment or lotteries should be accompanied by rigorous impact evaluation. This approach would enable governments to design innovative initiatives and target resources most efficiently and equitably.