Peru (Independent Evaluation Group Studies)
Contents:
And their drivers don't care about road safety regulations. But we don't have good regulations for them. They are fragile and dangerous as well. We also need to educate drivers of moto-taxis. With regard to suggested actions to improve the implementation of current RTI interventions, the most important ones were the need for a stronger multisectoral coordination of scaling-up activities; and the need to develop strategies to overcome the cultural, social and geographical diversity of the country, and to address the specificities and modalities for enforcing regulations in the different regions.
As a local policymaker explained:. Then we can have more efficient actions, better results. But we should remember that each place has its own problems, moto-taxis for instance, are a problem here, but not so much in Lima. Also, you know that use of seat belts are going to be very difficult in Ucayali, because it's so hot here. And can you imagine seat belts in moto-taxis? Regarding the media influence at local level, as the following quote highlights, a wide but responsible participation and support from the media was also identified as an important and crucial step in the successful implementation of RTI interventions and their enforcement:.
Well, journalists are very important, but they shouldn't be so sensationalists, they should be more responsible when they release the news.
Only in that way they are going to help really to improve the problem of road crashes. Finally, the interviewees consistently pointed out that none of the interventions have systematic monitoring and evaluation activities, and no clearly identified process and impact indicators to evaluate progress along the way. When they exist, they are most often related to the measurement of administrative goals, rather than to the impact of the implemented interventions themselves or to identification of aspects needing improvement. This reveals that monitoring and evaluation limitations are far more dramatic than we had assumed in our theoretical framework:.
It is hard to talk about monitoring, you know, especially when implementation of activities is still limited. We have explored this, however, and facilitators should be trained, so they can monitor and offer counseling. But actual evaluation of results and monitoring, no, because it is not sustainable. Probably in the future, after massive training of drivers.
We have a supervisor who tries to ensure a small evaluation at the end. We were thinking to give an incentive to schools that show improvement in road safety. We would like to perform a theater representation. Although they acknowledged the existence of various information systems in the police, Ministry of Transport, Ministry of Health, and the private sector, which could be used as useful information sources for monitoring and evaluation efforts, participants expressed concern about the quality and completeness of the assembled information, the limited access to the existing information, and the lack of coordination and integration of these different information systems.
This finding, which we did not specifically include in our theoretical framework, highlights that a construction of a functional and integrated information system for RTIs remains a pending task at national and local level:. The police, the judicial system, the insurance companies and other sectors have agreed. But there are no concrete results. For example, we wanted to use the district police information, but they didn't want. They said that there was not superior approval.
Product details
We need written agreements, maybe a law. We need to improve the quality of the information. It needs to be complete. If there are not data, we don't have monitoring and evaluation, you know. A Coordination Committee has been established to allow sharing of information on violence and road traffic crashes.
We have met to coordinate. We hope to have access to different information sources. The meeting generated a wide array of ideas and suggestions applicable to the design, implementation and evaluation phases of the policy process. Many of them were consistent with those provided in the in-depth interviews presented above. The most important or recurring ones are highlighted here. First, the participants emphasized the need to promote intersectoral alliances both for normative and operational aspects of RTI interventions, as otherwise overlapping, and sometimes-competing efforts will hamper the impact and effectiveness of different interventions.
Second, they insisted on the need to strengthen enforcement of road safety regulatory strategies. In particular, they emphasized that the police needs to be supported from inside and outside to improve its social surveillance role. This would include strengthening of all aspects of the police system, including audit and accountability mechanisms, training, logistic and budgetary aspects.
Evaluation of interventions on road traffic injuries in Peru: a qualitative approach
The lack of availability of even basic supplies such as alcohol measurement devices and of combustible fuel for the patrols were highlighted as dramatic examples that jeopardize an effective participation of police in the implementation of RTI interventions such as control of speed limits and alcohol-impaired driving. Third, the participants highlighted the importance of sustaining and improving technical training and regular supervision of human resources involved in different aspects of RTI implementation efforts, at both national and local level.
Fourth, they felt that improvement of quality and accessibility of information related to RTIs is a pending accomplishment. Although several information systems are in place, there is a critical need for a thorough process to improve the structural and functional aspects of each information system, to identify core indicators, to have easy access to meaningful data, and to effectively coordinate and integrate the generated data to get reliable and updated epidemiologic profiles of RTIs at national and sub-national level. The brainstorming session and email discussions proved to be a very instructive and useful exercise that can be easily repeated for other interventions.
Zero Tolerance inspections are currently limited to main departure points in Lima and a few large cities of the country. Buses and drivers undergoing inspection are not chosen randomly or systematically. While inspection of vehicles and drivers should occur with the joint participation of Ministry of Transport officers, police officers and a public prosecutor, in reality, they are performed sporadically by a Ministry of Transport officer, who is often short of the necessary equipment and public support for an effective enforcement of the corrective measures. Early in its implementation, unrealistic expectations for the potential impact of Zero Tolerance were raised both by the media and politicians.
It was burdened, therefore, with ample discredit and wide public criticism that resulted in decreased political commitment and limited allocation of resources. The most important perceived negative system-wide effects were unplanned diversion of human resources and unfulfilled aims to setup a functional information system for monitoring and evaluation. Overall, the brainstorming exercise revealed that although there is room for substantial redesign of Zero Tolerance at different building blocks of the health system and the different sectors involved, policymakers may also wish to reconsider whether it deserves continued implementation, due to its very narrow target group only public inter-provincial buses , and thus the remote likelihood of any measurable impact on the overall rate of RTIs.
As a result of the different activities and findings described above, supplemented with additional relevant literature [ 29 - 32 ], we identified a list of key programmatic components that we believe are critical for the successful implementation of any policy intervention, including but not limited to RTIs:.
The qualitative approach we used for appraising the RTI problem revealed key programmatic aspects that should be taken into consideration when designing any RTI policy intervention. First, candidate interventions should be identified and selected through a wide consultation process with stakeholders representing all relevant actors, and after careful appraisal of the scientific evidence.
.svg/1200px-Flag_of_Peru_(state).svg.png)
Then they should be carefully designed, with defined objectives, target groups, as well as inputs, outputs, outcomes and impact indicators to be assessed through a functioning monitoring and evaluation system [ 32 ]. A summary of key recommendations emerging from this process for selecting and planning RTI policies follows:. It is reassuring that our proposed general and RTI-specific programmatic recommendations are generally in agreement with various experiences in other low- and middle-income settings with regard to identifying and applying key aspects for successful implementation of RTI interventions, but considering also context factors such as migration trends, urbanization planning, quality of road network, and vehicles importing regulations.
Take for instance the experience of Curitiba in Brazil, or of Bogota, the capital city of Colombia. A Bus Rapid Transit BRT system combined with a long-term land use planning was established in Curitiba since as part of a Master Plan to convert it in a livable city calling for a cultural, social and economic transformation of the city [ 33 ]. A traveller survey was performed in, showing a reduction of about 27 M auto trips per year, 28 percent reduction of particular car users, use of about 30 percent less fuel than other Brazilian cities of its size, resulting in one of the lowest rates of ambient air pollution in the country.
Currently Curitiba BRT serves more than 1. Eighty percent of travelers use the express or direct bus services, and citizens spend only about 10 percent of their income on travel, much below the national average. In Bogota, a road safety programme combining land use and transport measures was implemented since , with careful planning, the necessary budget allocation, and with an active and sustained involvement of central and local governments and of the community [ 35 ]. The emphasis of the programme was put on meeting the needs of Bogota's non-motorized road users and to improve public transport.
Building on Brazil and Colombia experiences, a programme of provision of a rapid massive bus transport system was implemented in Lagos Nigeria in , adapted to the local context, together with other road safety regulatory enforcement measures and investment in infrastructure needs for an enabling environment, with the critical role of commitment and leadership at the highest political levels as an essential success ingredient [ 37 ]. Although an impact evaluation on reduction of road traffic injuries is still pending, this programme has already shown an increase in cheaper, safe, comfortable and efficient transportation for over commuters, as well as a 40 percent reduction in journey time and a 35 percent reduction in passengers waiting time.
Another illustrative example comes from Thailand, where strong political leadership, responsibility delegation as part of the decentralization process, use of information from RTI databases, positive media influence and multisectoral participation including community groups, NGOs, and advocacy groups , allowed the development of a comprehensive RTI prevention policy [ 38 ]. Interestingly, while the provinces themselves were responsible for preparing the provincial level plans for action, they were developed by local multisectoral public agencies, and at the same time they were largely in agreement with national goals and objectives.
The plans cover law enforcement, public education, traffic engineering, emergency services and information systems. Although initially financed on an ad-hoc basis, by the end of a regular budget was dedicated to road safety. This RTI prevention policy was launched in Thailand building on lessons learned from previous efforts that had not led to measurable impact on RTIs, because they had disregarded various programmatic aspects that were only incorporated in the current initiative [ 38 ].
Likewise, it is reassuring that the main programmatic issues raised by participants in our study are well in line with those recommended by the WHO for a better planning, implementation and evaluation of RTI interventions [ 39 ]. The Make Roads Safe Campaign of the Commission for Global Road Safety has also defined the programmatic building blocks for a concerted global effort to reduce the rising toll of road deaths and injury [ 40 ]: At the same time, our results also highlight particular, context-specific issues that our national and local policymakers should consider when selecting RTI interventions, such as the need to think on alternative public transport vehicles, other than moto-taxis , which are too fragile and hazardous, and alternatives other than sub-standard quality vans and microbuses.
It is encouraging that very recently various local governments municipalities have pushed forward a renewed impetus to interventions addressing those specificities.
These specificities, similarly to the key programmatic issues raised by the interviewees, are also mentioned by WHO programmatic recommendations [ 40 ], as key policy aspects to take into account at national and local levels. In this paper we described the process and the utility of using a systems approach to address policy questions related to RTIs. Strengthening of key programmatic aspects of the different interventions and systematic consideration of potential system-wide positive and negative effects are crucial through the whole policy process, from design through to implementation and evaluation phases.
Without involvement of key stakeholders from all sectors involved, it is unlikely that any of these interventions will reach an impact, since coordination and collaboration between sectors are crucial to the successful implementation and enforcement of chosen strategies. Lessons learnt from this process were summarized in a series of key programmatic aspects and principles for consideration in reviewing and revising the existing RTIs policy interventions plan.
While this is as far as we can go with this process, we hope that the findings from this study, and more importantly the process itself that was richly informed and strengthened by the involvement of stakeholders, will help moving this agenda forward. We particularly hope that key stakeholders involved in the design and implementation of RTI policies in Peru will be inclined to take the evidence generated from this study several steps forward, to increase the likelihood of successful implementation of these interventions, and eventually to reach a measurable impact at population level.
LH conceived the paper and wrote the first draft and the final version of the manuscript. All other authors made substantial contributions to the content of the final version. All authors read and approved the final manuscript. The sponsor had no role in the design, analysis, and interpretation of the literature, nor in the writing of the report and the decision to submit for publication. We gratefully acknowledge all those involved in the PIAT Working Group, field coordinators, and field workers that participated in the different study locations and in particular to those who volunteered as participants in the study.
We are also grateful to all persons that supported the study in its different phases: National Center for Biotechnology Information , U. Published online Jan Brazil , Lima LI05, Peru. Received Sep 3; Accepted Jan Abstract Background Evaluation of interventions on road traffic injuries RTI going beyond the assessment of impact to include factors underlying success or failure is an important complement to standard impact evaluations.
Methods We performed in-depth interviews with policymakers and technical officers involved in the implementation of RTI interventions to get their insight on design, implementation and evaluation aspects. Results Key programmatic components were consistently identified that should be considered when designing and implementing RTI interventions.
Conclusions Our approach revealed substantial caveats in current RTI interventions in Peru, and fundamental negative effects on several components of the sectors and systems involved. Background Road traffic injuries RTIs are currently one of the leading causes of death at global level. Open in a separate window.
Ranking of selected Latin American countries by quality of roads. Methods Qualitative evaluation of RTI interventions Qualitative studies focused on policy makers and implementers may allow understanding of underlying success or failure factors operating during the planning and implementation phases. Stakeholders meeting We then held a meeting with stakeholders representing in a balanced way the different sectors involved in planning or implementation of road safety interventions in the country, as recommended [ 25 , 26 ].
Brainstorming sessions with stakeholders Additionally, a role-play brainstorming session was organized with a group of participants representing key policymakers and implementers from different sectors. Results and discussion Qualitative evaluation of RTI interventions Based on our preconceptions reflexivity principle and on the iterative discussions performed until the theoretical saturation point was reached, the analysis issues were consolidated eventually into three main categories, namely driving forces for selection of RTI interventions, programmatic characteristics of interventions, and monitoring and evaluation aspects.
Selecting RTI interventions in Peru There was a general consensus that the main motivation for selecting RTI interventions varied in different settings, although a salient driving force highlighted was the pressure exerted by the media and the community. As an interviewee explained: An interviewee put it this way: Technical officer of Road Safety Education, Lima Interestingly, the interviewees agreed with the media on the importance of strengthening current students and drivers' education interventions despite informing them about available evidence from Peru suggesting otherwise [ 27 ].
As a policymaker explained: Policymaker, Technical Secretariat of Road Safety Council, Lima At regional level, the need to comply with municipal or regional level mandates; efforts by NGOs focusing on road safety; and the decentralization process at regional and local governments were all considered as important driving forces. Main programmatic drawbacks related to RTI interventions Interviewees consistently identified programmatic drawbacks in the policy process of design and implementation of RTI interventions in Peru, which include: Coordinator, Violence Observatory, Universidad Peruana Cayetano Heredia, Lima b Lack of clear planning and implementation guides, which lead to a high degree of improvisation when selecting potential RTI interventions, and when implementing them.
Coordinator, Violence Observatory, Universidad Peruana Cayetano Heredia, Lima c Lack of regular, planned training and supervision activities; and scarcity of dedicated staff to implement and enforce RTI interventions. One technical officer explained: Technical officer, Training Unit, Municipality of Pucallpa, Ucayali d Ineffective coordination between the different sectors involved, which is particularly important when designing and implementing complex, crosscutting RTI interventions.
A policymaker put it this way: Central level policymaker, National Strategy for Road Traffic Injuries Ministry of Health, Lima e Insufficient community participation, which reveal the need to consider more seriously the opinions and the active participation of community members when planning and implementing the interventions. Responsible, Road Safety Program, San Borja Municipality, Lima These perceptions were broadly in agreement with our theory framework, although interviews revealed further context-specific aspects not anticipated, such as lack of adequate regulations for local transport vehicles.
As local policymakers explained: Responsible, Training Program for moto-taxis, Coronel Portillo Municipality, Pucallpa With regard to suggested actions to improve the implementation of current RTI interventions, the most important ones were the need for a stronger multisectoral coordination of scaling-up activities; and the need to develop strategies to overcome the cultural, social and geographical diversity of the country, and to address the specificities and modalities for enforcing regulations in the different regions.
As a local policymaker explained: Policymaker, Road Safety Training Unit, Ministry of Transport, Ucayali Regarding the media influence at local level, as the following quote highlights, a wide but responsible participation and support from the media was also identified as an important and crucial step in the successful implementation of RTI interventions and their enforcement: Monitoring and evaluation Finally, the interviewees consistently pointed out that none of the interventions have systematic monitoring and evaluation activities, and no clearly identified process and impact indicators to evaluate progress along the way.
This reveals that monitoring and evaluation limitations are far more dramatic than we had assumed in our theoretical framework: Policymaker, Communication Unit, Road Safety Decentralization Program, Lima It is hard to talk about monitoring, you know, especially when implementation of activities is still limited. Implementer, Safety Road Project, Ayacucho Municipality Although they acknowledged the existence of various information systems in the police, Ministry of Transport, Ministry of Health, and the private sector, which could be used as useful information sources for monitoring and evaluation efforts, participants expressed concern about the quality and completeness of the assembled information, the limited access to the existing information, and the lack of coordination and integration of these different information systems.
This finding, which we did not specifically include in our theoretical framework, highlights that a construction of a functional and integrated information system for RTIs remains a pending task at national and local level: Policymaker, Health District, Lima " Chief, National Police, Ayacucho. Stakeholders meeting The meeting generated a wide array of ideas and suggestions applicable to the design, implementation and evaluation phases of the policy process.
System-wide effects of zero tolerance The brainstorming session and email discussions proved to be a very instructive and useful exercise that can be easily repeated for other interventions. Table 2 Prioritized perceived system-wide effects of Zero Tolerance. Considerations for a revised RTI interventions plan using a systems approach General lessons for successful implementation of policies As a result of the different activities and findings described above, supplemented with additional relevant literature [ 29 - 32 ], we identified a list of key programmatic components that we believe are critical for the successful implementation of any policy intervention, including but not limited to RTIs: Programmatic considerations related to RTI interventions The qualitative approach we used for appraising the RTI problem revealed key programmatic aspects that should be taken into consideration when designing any RTI policy intervention.
A summary of key recommendations emerging from this process for selecting and planning RTI policies follows: Conclusions In this paper we described the process and the utility of using a systems approach to address policy questions related to RTIs. Competing interests The authors declare that they have no competing interests.
Maria-Elena Pinglo
Authors' contributions LH conceived the paper and wrote the first draft and the final version of the manuscript. Pre-publication history The pre-publication history for this paper can be accessed here: References World Health Organization. An opportunity for a different Peru. Prosperous, equitable, and governable. The World Bank; Traffic accidents scar Latin America's roads.
Country program evaluation for the World Bank Group, The World Bank, Washington, D. The Global Competitiveness Report Accessed on October 16, at: Pedestrians at risk in Peru. Commission for Global Road Safety. The United Nations, Washington, D. Reducing unintentional injuries on the nation's highways: J Health Care Poor Underserved. Bull World Health Organ. Instituto Nacional de Salud. Miranda JJ, Huicho L. Road traffic injuries in Peru: Assessment of the structure, dynamics and monitoring of information systems for road traffic injuries in Peru Gov't] ; 27 2: Road traffic injuries in developing countries: World report on road traffic injury prevention.
Don de Savigny, Taghreed Adam, editor. Systems thinking for health systems strengthening. Strauss A, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Basics of Grounded Theory Analysis. Stakeholder analysis for health research: Exploring health systems research and its influence on policy processes in low income countries. Utilization of research in policymaking for graduated driver licensing. Am J Public Health. Developing an evaluation framework for crosscutting policy goals: This report presents findings of an impact evaluation of a rural irrigation investment program with multiple components in coastal Peru.
It addresses three questions: The primary audiences for this report are development practitioners engaged in designing similar projects, evaluators interested in using similar methodology, and the general evaluation community. The rest of this report is organized as follows. Chapter 1 briefly explains how irrigation projects transmit their impacts, identifies potential areas of concern for this type of project, and illustrates these with experiences in water-related projects around the world.
Another section in this chapter provides some necessary background on the agricultural sector in Peru, past and present, and a brief description of the irrigation program and each of its components. Chapter 2 discusses the mixed-methods approach used in the impact evaluation, focusing on data description. Chapter 3 elaborates on the technical strategy used to identify impacts. Chapter 4 presents and discusses the findings. Chapter 5 presents concluding remarks. Independent Evaluation Group, Download full text from publisher File URL: Corrections All material on this site has been provided by the respective publishers and authors.
Help us Corrections Found an error or omission?