Results-Based Monitoring and
Evaluation System (RBME)
A Tool for Public Sector Management
             Madhawa Waidyaratna B. Sc. (Chem.), MBA , MSc (IT), MA, MCSSL, MSLIM
             Director Planning, Industrial Development Board of Ceylon
Overview of RBME
RBME is a powerful public management tool that can be
used to help policymakers and decision makers track
progress and demonstrate the impact of a given project,
program, or policy
There are growing pressures in developing countries to
improve performance of their public sectors
Overview of RBME [contd.]
Involves reform by tracking results of government or
organizational actions over time
Results-based M&E differs from traditional
implementation-focused M&E in that it moves beyond an
emphasis on inputs and outputs to a greater focus on
outcomes and impacts
New results-based approaches help to answer the “so
what” question
In other words, governments and organizations may
successfully implement programs or policies, but have
they produced the actual, intended results
New Challenges in Public Sector
Governments are increasingly being called upon to
demonstrate results. Stakeholders are no longer solely
interested in organizational activities and outputs; they are
now more than ever interested in actual outcomes.
  Have policies, programs, and projects led to the desired
  results and outcomes?
  How do we know we are on the right track?
  How do we know if there are problems along the way?
  How can we correct them at any given point in time?
  How do we measure progress?
  How can we tell success from failure?
The Power of Measuring
If you do not measure results, you can not tell success from failure

         If you can not see success, you can not reward it

If you can not reward success, you are probably rewarding failure


       If you can not see success, you can not learn from it


      If you can not recognize failure, you can not correct it


   If you can demonstrate results, you can win public support

                                                Adapted from Osborne & Gaebler, 1992
Reasons to Do Results-Based M&E
Provides crucial information about performance
Provides a view over time on the status of a
project, program or policy
Promotes credibility and public confidence by reporting
on the results of programs
Helps formulate and justify budget requests
Identifies potentially promising programs or practices
Reasons to Do Results-Based M&E
Focuses attention on achieving outcomes important to
the organization and its stakeholders
Provides timely, frequent information to staff
Helps establish key goals and objectives
Permits managers to identify and take action to correct
weaknesses
Supports a development agenda that is shifting towards
greater accountability for finances
International Initiatives
Millennium Development Goals (MDGs)
Paris Declaration on Aid Effectiveness
Accra Agenda for Action (AAA)
International Aid Transparency Initiative (IATI)
Highly Indebted Poor Country (HIPC) Initiative
Managing for results
Use the information to improve decision making and steer
development interventions towards clearly defined goals

   Traditional Management
   Approach
   • Focused mainly on inputs and
     activities.


             Results-Based Management
             Approach
             • Focus on the results obtained
               rather than just on the inputs
               used or the activities conducted
Critical Factors for Defining Results
 Socio-economic context: Results statement should reflect
 local needs and priorities
 Local Capacity: Existing skills, leadership, and
 management capacity will impact on what can be
 implemented to achieve expected results
 Resources: Level of resources will impact on what can
 realistically be achieved
 Timetable: Results framework must identify the results
 (changes) to be achieved in the life of the program
Monitoring & Evaluation
Results-based monitoring is a continuous process of collecting
and analyzing information on key indicators, and comparing
actual results to expected results
Results-based evaluation is an assessment of a planned,
ongoing, or completed intervention to determine its relevance,
efficiency, effectiveness, impact, and/or sustainability
  Monitoring: tracks movement of indicators towards the
  achievement of specific, predetermined targets
  Evaluation: takes a broader view, considering progress toward
  stated goals, the logic of the initiative, and its consequences
  Both are needed to better manage policies, programs, and
  projects
                                                  Adapted from OECD definition
Doing Things Right
Key Types of Monitoring
                   Impact
Results




                               Results Monitoring

                  Outcome



                   Output
Implementation




                  Activity   Implementation Monitoring
                             (Means and Strategies)


                    Input
Results-Based Monitoring
                 Goal (Impacts)
                                  • Long-term, widespread
                                    improvement in society
Results




                  Outcomes
                                  • Intermediate effects of outputs
                                    on clients


                   Outputs        • Products and services produced
Implementation




                  Activities      • Tasks personnel undertake to
                                    transform inputs to outputs


                    Inputs
                                  • Financial, human, and material
                                    resources
Results-Based Monitoring
                                  • Higher income levels of
                 Goal (Impacts)
                                    entrepreneurs ; Increased
Results



                                    global competitiveness

                  Outcomes
                                  • Increased productivity;
                                    reduction of production cost

                                  • Number of programmes
                   Outputs          conducted ; Number of trained
                                    personnel
Implementation




                  Activities      • Productivity enhancement
                                    programmes


                    Inputs        • Facilities, trainers, materials
The Essential Actions in Building
          M&E System
Formulate outcomes and goals
Select outcome indicators to monitor
Gather baseline information on the current condition
Set specific targets to reach and dates for reaching them
Regularly collect data to assess whether the targets are
being met
Analyze and report the results
Implementation Monitoring Links
     to Results Monitoring
                 Outcome




   Target 1       Target 2     Target 3



  Means and     Means and     Means and
   Strategies    Strategies    Strategies
  (Multi-Year   (Multi-Year   (Multi-Year
  and Annual    and Annual    and Annual
  Work Plans)   Work Plans)   Work Plans)
Performance Indicators
A variable that provides accurate and reliable evidence about the achievement of
a specific result
Indicators should be SMART or CREAM

Specific ; Measurable ; Attributable ; Realistic ; Targeted
Clear ; Relevant ; Economic ; Adequate ; Monitorable


                 What gets measured gets done


                 If you don’t measure results, you
                 can’t tell success from failure
Complementary Roles of
        Monitoring and Evaluation
         Monitoring                                Evaluation

                                         Analyzes why intended results were or
   Clarifies program objectives
                                        were not achieved

    Links activities and their           Assesses specific causal contributions of
   resources to objectives             activities to results
    Translates objectives into
   performance indicators and set
    targets                             Examines implementation process

    Routinely collects data on
   these indicators, compares
    actual results with targets         Explores unintended results

                                         Provides lessons, highlights significant
 Reports progresstotoproblems 
  and alerts them
                       managers          accomplishment or program
                                         potential, and offers recommendations
                                         for improvement
Examples of Results Chain


            Long Term             Outcomes             Outputs
            Goal                                       • Teachers trained
Education                         • Increased
                                    student
            • Increase literacy                        • Text Books
              rates                 completion rates     provided




            Long Term             Outcomes             Outputs
            Goal                  • Increased use of   • Doctors hired
Health      • Improved              health clinics     • Health workers
              maternal                                   trained
              mortality
Translating Outcomes to Action
Activities are crucial. They are the actions you take to
manage and implement your programs, use your
resources, and deliver the services of government
But the sum of these activities may or may not mean you
have achieved your outcomes
Question is: How will you know when you have been
successful?
Designing Good Evaluations

     Better to be approximately
     correct than precisely wrong
     • Paraphrased from Bertrand Russell
Designing, Building and Sustaining
            a RBME System
                                              Planning for
                   Selecting Key              Improvement
Conducting a       Indicators to              — Selecting               The Role of              Using
Readiness          Monitor                    Results Targets           Evaluations              Your
Assessment         Outcomes                                                                      Findings



    1          2        3              4           5            6          7          8            9            10


          Agreeing on              Baseline Data           Monitoring           Reporting Your              Sustaining
          Outcomes to              on                     for Results           Findings                    the
          Monitor and              Indicators—                                                              M&E System
          Evaluate                 Where Are                                                                Within Your
                                   We Today?                                                                Organization
Barriers to M&E
Do any of the following present barriers to building an
M&E system?
                                                               lack of an
                                               lack of a
                               lack of                         outcome-
 lack of fiscal resources                    champion for
                            political will                  linked strategy
                                              the system
                                                             or experience




           How do we confront these barriers?
In Conclusion

  We are what we repeatedly do.
  Excellence, then, is not an act, but
  a habit
  • Aristotle




Questions? Comments? Views?

Results Based Monitoring and Evaluation

  • 1.
    Results-Based Monitoring and EvaluationSystem (RBME) A Tool for Public Sector Management Madhawa Waidyaratna B. Sc. (Chem.), MBA , MSc (IT), MA, MCSSL, MSLIM Director Planning, Industrial Development Board of Ceylon
  • 2.
    Overview of RBME RBMEis a powerful public management tool that can be used to help policymakers and decision makers track progress and demonstrate the impact of a given project, program, or policy There are growing pressures in developing countries to improve performance of their public sectors
  • 3.
    Overview of RBME[contd.] Involves reform by tracking results of government or organizational actions over time Results-based M&E differs from traditional implementation-focused M&E in that it moves beyond an emphasis on inputs and outputs to a greater focus on outcomes and impacts New results-based approaches help to answer the “so what” question In other words, governments and organizations may successfully implement programs or policies, but have they produced the actual, intended results
  • 4.
    New Challenges inPublic Sector Governments are increasingly being called upon to demonstrate results. Stakeholders are no longer solely interested in organizational activities and outputs; they are now more than ever interested in actual outcomes. Have policies, programs, and projects led to the desired results and outcomes? How do we know we are on the right track? How do we know if there are problems along the way? How can we correct them at any given point in time? How do we measure progress? How can we tell success from failure?
  • 5.
    The Power ofMeasuring If you do not measure results, you can not tell success from failure If you can not see success, you can not reward it If you can not reward success, you are probably rewarding failure If you can not see success, you can not learn from it If you can not recognize failure, you can not correct it If you can demonstrate results, you can win public support Adapted from Osborne & Gaebler, 1992
  • 6.
    Reasons to DoResults-Based M&E Provides crucial information about performance Provides a view over time on the status of a project, program or policy Promotes credibility and public confidence by reporting on the results of programs Helps formulate and justify budget requests Identifies potentially promising programs or practices
  • 7.
    Reasons to DoResults-Based M&E Focuses attention on achieving outcomes important to the organization and its stakeholders Provides timely, frequent information to staff Helps establish key goals and objectives Permits managers to identify and take action to correct weaknesses Supports a development agenda that is shifting towards greater accountability for finances
  • 8.
    International Initiatives Millennium DevelopmentGoals (MDGs) Paris Declaration on Aid Effectiveness Accra Agenda for Action (AAA) International Aid Transparency Initiative (IATI) Highly Indebted Poor Country (HIPC) Initiative
  • 9.
    Managing for results Usethe information to improve decision making and steer development interventions towards clearly defined goals Traditional Management Approach • Focused mainly on inputs and activities. Results-Based Management Approach • Focus on the results obtained rather than just on the inputs used or the activities conducted
  • 10.
    Critical Factors forDefining Results Socio-economic context: Results statement should reflect local needs and priorities Local Capacity: Existing skills, leadership, and management capacity will impact on what can be implemented to achieve expected results Resources: Level of resources will impact on what can realistically be achieved Timetable: Results framework must identify the results (changes) to be achieved in the life of the program
  • 11.
    Monitoring & Evaluation Results-basedmonitoring is a continuous process of collecting and analyzing information on key indicators, and comparing actual results to expected results Results-based evaluation is an assessment of a planned, ongoing, or completed intervention to determine its relevance, efficiency, effectiveness, impact, and/or sustainability Monitoring: tracks movement of indicators towards the achievement of specific, predetermined targets Evaluation: takes a broader view, considering progress toward stated goals, the logic of the initiative, and its consequences Both are needed to better manage policies, programs, and projects Adapted from OECD definition
  • 12.
  • 13.
    Key Types ofMonitoring Impact Results Results Monitoring Outcome Output Implementation Activity Implementation Monitoring (Means and Strategies) Input
  • 14.
    Results-Based Monitoring Goal (Impacts) • Long-term, widespread improvement in society Results Outcomes • Intermediate effects of outputs on clients Outputs • Products and services produced Implementation Activities • Tasks personnel undertake to transform inputs to outputs Inputs • Financial, human, and material resources
  • 15.
    Results-Based Monitoring • Higher income levels of Goal (Impacts) entrepreneurs ; Increased Results global competitiveness Outcomes • Increased productivity; reduction of production cost • Number of programmes Outputs conducted ; Number of trained personnel Implementation Activities • Productivity enhancement programmes Inputs • Facilities, trainers, materials
  • 16.
    The Essential Actionsin Building M&E System Formulate outcomes and goals Select outcome indicators to monitor Gather baseline information on the current condition Set specific targets to reach and dates for reaching them Regularly collect data to assess whether the targets are being met Analyze and report the results
  • 17.
    Implementation Monitoring Links to Results Monitoring Outcome Target 1 Target 2 Target 3 Means and Means and Means and Strategies Strategies Strategies (Multi-Year (Multi-Year (Multi-Year and Annual and Annual and Annual Work Plans) Work Plans) Work Plans)
  • 18.
    Performance Indicators A variablethat provides accurate and reliable evidence about the achievement of a specific result Indicators should be SMART or CREAM Specific ; Measurable ; Attributable ; Realistic ; Targeted Clear ; Relevant ; Economic ; Adequate ; Monitorable What gets measured gets done If you don’t measure results, you can’t tell success from failure
  • 19.
    Complementary Roles of Monitoring and Evaluation Monitoring Evaluation Analyzes why intended results were or  Clarifies program objectives  were not achieved Links activities and their Assesses specific causal contributions of  resources to objectives  activities to results Translates objectives into  performance indicators and set targets  Examines implementation process Routinely collects data on  these indicators, compares actual results with targets  Explores unintended results Provides lessons, highlights significant  Reports progresstotoproblems  and alerts them managers accomplishment or program potential, and offers recommendations for improvement
  • 20.
    Examples of ResultsChain Long Term Outcomes Outputs Goal • Teachers trained Education • Increased student • Increase literacy • Text Books rates completion rates provided Long Term Outcomes Outputs Goal • Increased use of • Doctors hired Health • Improved health clinics • Health workers maternal trained mortality
  • 21.
    Translating Outcomes toAction Activities are crucial. They are the actions you take to manage and implement your programs, use your resources, and deliver the services of government But the sum of these activities may or may not mean you have achieved your outcomes Question is: How will you know when you have been successful?
  • 22.
    Designing Good Evaluations Better to be approximately correct than precisely wrong • Paraphrased from Bertrand Russell
  • 23.
    Designing, Building andSustaining a RBME System Planning for Selecting Key Improvement Conducting a Indicators to — Selecting The Role of Using Readiness Monitor Results Targets Evaluations Your Assessment Outcomes Findings 1 2 3 4 5 6 7 8 9 10 Agreeing on Baseline Data Monitoring Reporting Your Sustaining Outcomes to on for Results Findings the Monitor and Indicators— M&E System Evaluate Where Are Within Your We Today? Organization
  • 24.
    Barriers to M&E Doany of the following present barriers to building an M&E system? lack of an lack of a lack of outcome- lack of fiscal resources champion for political will linked strategy the system or experience How do we confront these barriers?
  • 25.
    In Conclusion We are what we repeatedly do. Excellence, then, is not an act, but a habit • Aristotle Questions? Comments? Views?