Evaluation and Feedback

The purpose of a social marketing program is to effect some change in the target audience's attitudes and behaviors. The true test of the effectiveness of the program is not the number of PSAs which were aired, but whether it contributed to improving people's health and lives. However, there are a number of distinct levels to evaluation of the program, and each is important for different reasons. These can be divided into three basic types: process, outcome and impact evaluation. Process evaluation determines whether the intended target audience was reached. If so, outcome evaluation looks at whether they engaged in the desired behavior after being exposed to the message. Finally, impact evaluation judges whether performing the behavior induced the desired change (i.e., a reduction in related mortality and/or morbidity). Kotler and Roberto (1989) add another layer--ethical evaluation. This type determines the answer to the question, "Did the program ethically cause a right or desirable change in a right or desirable way?" It looks at both the means and the end of the program from an ethical perspective.

Process Evaluation

While the social marketing program is in effect, process evaluation should take place intermittently in each part of the program. As detailed in the implementation phase, it includes media monitoring and analysis, as well as evaluation of program activities. Most important, however, is tracking the same population that was surveyed in the initial baseline KAP study. The first wave of tracking should occur six months to a year after the product introduction, and ideally should be conducted at least annually. If possible, the methodology and wording of the questions should be identical to the first survey in order to maintain comparability of the results.

The key measurement areas to track include awareness of the product, advertising awareness and recall, knowledge level, attitudes and perceptions, images of product and users, experience with the product, and behaviors (trial and repeat). The new questions should be very specific about the particular product or campaign, in addition to the earlier, more general questions about attitudes and behaviors regarding the topic.

Based upon the standards set in the program objectives, the results should focus on levels of awareness, trial and continued/repeat usage. The interpretation of these measures provide direction for improvements and areas upon which to concentrate in the future. If the survey indicates low levels of awareness of the product or campaign, the program should investigate whether the media vehicles are, in fact, reaching the target audience effectively and appropriately, and whether the communications materials are memorable, understandable and consistent with the program objectives. If there is high awareness, but low trial, this indicates that the message is reaching the audience, but other elements of the mix may be weak. These problems may be with the distribution system, price, product image or packaging, poorly conceived promotion activities or competition from other sources. Finally, both awareness and trial may be high, but the target audience does not sustain the behavior. In this case, there may be problems with the product itself, or consumers may be unmotivated to continue usage. This type of process evaluation can be very helpful in diagnosing problems or indicating success.

Outcome Evaluation

The follow-up survey will help to identify the extent of attitude and behavior change in the target population, and tie it to their exposure to the campaign or use of the product. A "user profile" can be compiled, either from the KAP survey or from additional studies of users and non-users. For example, those who use condoms can be compared with those who do not on a number of attributes. These would include demographics, contraceptive history, lifestyle factors, sexual behavior, brand use, advertising awareness and attitudes toward the particular social marketing product. Evaluation efforts can also utilize secondary sources to determine changes in behavioral measures. These include the annual National Health Interview Survey (NHIS) and the state-based Behavioral Risk Factor Surveillance System (BRFS) conducted by the Centers for Disease Control. However, these general studies do not necessarily contain the information that is relevant to a particular social marketing program. If the organization is a government agency, it may be possible to add one or two relevant questions to the surveys.

Impact Evaluation

The actual impact of the social marketing program is often difficult to assess accurately. Can one public service announcement cause a drop in morbidity and mortality from heart disease? Probably not, but many such efforts combine synergistically and may be a contributing factor in health improvements. Educational efforts are relatively transient, and gone long before changes can be seen. Because campaigns change so quickly, it is impossible to determine the effect of a particular spot on overall trends. However, we can at least compare mortality and morbidity rates before and after implementation of the program in many cases.

The most effective way of establishing a cause-and-effect relationship between health marketing efforts and the changes in behavior and health outcomes is to conduct an intervention study in one or more communities, with matched communities as controls. Assuming that there are no significant differences between the intervention and control communities, marketing activities may be linked to changes in the communities with precision and reliability. If conducting the study in a number of communities, the effectiveness of various elements of the program can be tested by including them or excluding them from certain markets. Without this type of scientific methodology, it is not possible to claim that improvements were a direct result of the social marketing program.

Ethical Evaluation

Kotler and Roberto give strong emphasis to the importance of ethics in social marketing. Whenever marketing behavior change, it is imperative to acknowledge the need for responsibility and accountability to the people in the target audience. Although in the end, the results of the program are the final measure of success, the means to that end are just as important. People should never be coerced into a behavior, even though it may be "for their own good." Programs may also have side effects and unintended consequences, which can be harmful in the long run. For example, a program intended to empower minority women to use condoms resulted in a number of beatings by their male partners, who perceived their insistence on condoms as an insult. Programs which offer incentives for behavior change may encourage materialistic values and a disinclination to do anything without an extrinsic reward. Ethical criteria must be considered from the beginning in selecting the target audience, designing research and determining a social marketing mix, and must also be used in evaluation throughout a program to make sure that it "does no harm." Every element of the program must be ethically sound.

Prior Page