VA
Insights

There are three “90-10” rules in the field of training for commercial organizations. Over 90% of employees’ learning does not come from formal training but from outside the classroom. Over 90% of employees’ improvement do not owe to the training department but direct supervisors. Over 90% of a company’s problems cannot be tackled by training.

These statistics may embarrass those training agencies in the field of capacity building in the philanthropy sector: Is training useful? We do not attempt to answer this question in the article. However, we hope this article can serve as a guide to help examine capacity-building practices in the philanthropy sector from the perspective of impact evaluation. Our question then would be: how to assess and ensure the effectiveness of capacity building in the philanthropy sector?

How to assess the effectiveness of capacity building?

The effectiveness of training programs has long been a great concern of funders and training organizers. From the perspective of impact evaluation, capacity-building programs are also faced with difficulties of evaluating non-qualitative and untraceable impact within a short period. Nowadays, it is a prevailing trend for organizers to distribute pro-training questionnaire. The biggest problem with this is that questionnaires are mostly all related to the satisfaction aspect of the courses but lack of actual associations with the objectives of the course.

Kirkpatrick’s training evaluation model, commonly used in commercial training, focuses on 4 aspects of the training programs: ”reaction” (what they thought and felt about the training), ”learning” (increase of knowledge or capability because of training), ”behavior” (the extent of behavior and capability improvement and application), “result” (the effects on the business or environment resulting from the trainee’s performance). “Reaction” and “learning” show a clear positive correlation and so do “behavior” and “result”. But there is no obvious correlation between these two levels. In other words, trainees’ satisfaction toward the training (“reaction”) can prove the knowledge they obtain (“learning”), but cannot ensure that they can apply what they have learned to improve work efficiency (“behavior -result”).

There are two approaches to further assess the effectiveness of capacity building training: 1) Beyond satisfaction survey, follow-up work can be enhanced to strengthen the knowledge and skillset transfer and trainee feedback can be collected on a continuous basis; 2) Similar to other evaluation programs, without impact indicators, we could validate the program theory (the logical chain from input to impact) to ensure the anticipated effects after project implementation. In a training program, if there are scientific needs analysis and goal setting, learning “response” can naturally lead to learning “effect”.

Obviously, the latter is a more radical strategy. Without needs analysis, follow-up work is just a waste of time. If we clearly know what the program wants to achieve (NGOers’ specific business requirements, working skills, rather than abstract terms such as “project management” and “fundraising capacity”), and also design the training course in a scientific way, trainee satisfaction can reflect training effectiveness to the greatest extent.

How to ensure the effectiveness of capacity building?

There lie two problems in needs assessment in NGO capacity building:

• When NGO practitioners say they need training on project management, human resource management, and marketing capacity, etc., training organizers do not really understand what they really mean, what skills they need to improve, what they need to learn
• A bigger issue is that just as trainees’ satisfaction level may not necessarily imply the effectiveness of the training, the needs described by the trainees may not be what they actually need to improve their performance. Trainers usually design courses and provide cases studies based on their professional knowledge, funders’ expectations and NGOers’ declared demands, but not what NGOers truly need. The drive for most training is external benchmark, rather than internal needs related to job performance.

The needs for capacity building should be based on existing problems in the working environment and organizational operation. To figure out the needs requires diagnosing the bottleneck of an organization. Therefore, we propose the following system for needs assessments:

• Help employees find out their real needs. Talk to NGOers and ask questions like “Anything happening or not happening in the working environment can lead you to this kind of training need?” or “How can you know if it works after attending this training?”

• Explore and analyze the internal benchmark: 1) Identify model individuals, projects and institutions; 2) Understand their performance and assess reasons for the gap (including aspects of skills, attitudes, procedures and division of responsibilities)

• Analyze the organizational objectives and problems: 1) mission and goal; 2) specific works for attaining the objectives; 3) strengths and weaknesses included in these works

• Identify the necessary knowledge and skills and set up training performance indicators

It can be seen from the above that there is a close connection between needs assessment and organizational evaluation. There are three levels in the process: 1) Macro-level: organization operation patterns, governance structure and strategic planning; 2) Meso-level: staff structure, incentive mechanism and management system; 3) Micro-level: employee skills and working environment. Capacity building is not (only) about solving problems, but more importantly, about the process of problem solving.

Prospect: Strategic Learning and Evaluation System

When the organizational evaluation is performed by the organization itself with the support of external specialists (not as trainers, but as facilitators) and is aligned with strategic planning, which leads to an internal mechanism involving self-evaluation, internal sharing, sustainable learning and systematic feedback, then the organization has achieved the so-called “Strategic Learning and Evaluation System”. The ultimate goal of NGO capacity building is to equip NGOs with a sustainable systematic learning mechanism and help them move forward to a learning organization. Evaluation, therefore, is the footstone of a systematic and in-depth capacity building practice.