Home
About IGWG
Guide
Presentations
Keynote Addresses
Participants
Resources
Contact Us
Print this Chapter Print the Entire Guide

IMPLEMENTATION GUIDE

VIII. Tools and Programs for Monitoring and Evaluation

Examples of questions to include in a Knowledge, Attitudes and Practice Survey (KAPS) are included in Annex I. KAPS are useful tools for assessing program needs and evaluating program achievements.

The programs outlined is this guide show the value of program evaluation and acknowledge the complexity of measuring changes in knowledge, attitudes, and practices, i.e., factors programs aim to change. Quantifiable outcome indicators require a clear definition of desired results in terms of women's health and gender equity. Indeed, measuring outcomes goes beyond defining desired outcomes, it includes developing scales to measure results and establishing norms on timing assessments, given that change takes time. These programs recognize that quantifying outcomes plays an enormous role in persuading donors and policy makers of a program's success in accomplishing its goals and meriting longer term funding, as well as in determining which interventions are effective and should be replicated.

To inform how indicators are developed and how to use them to quantify program achievements, two presentations focus on these separate objectives. One informs about the development of indicators to measure men's gender equitability, Measuring Gender Equitability Among Men (Brazil); and the other, Preliminary Findings from the New Visions Program Pilot Evaluation in Egypt, demonstrates the steps taken to measure the achievement of a pilot project with boys and young men in Egypt. The former suggests a scale that could be adapted to any male involvement project that mainstreams gender while the latter provides examples of quantitative indicators and how to apply them to assess improvements in RH knowledge and changes in behaviors, attitudes, and "gender awareness sensitivity."

UNFPA recently developed and published a program advisory note, It Takes 2, which includes a section on "outputs and output indicators for programming." These were not presented at the conference but merit mentioning as they include: a) program outputs for the different levels of society a program aims to impact; b) output indicators to measure achievement; and c) suggested activities to achieve intended outputs.

Measuring Gender Equitability Among Men (Brazil)

Implementing Agency: HORIZONS program of the Population Council and Instituto PROMUNDO.

1. Background
There is an increasing awareness that gender role socialization puts men and their partners at health risk (WHO, 2000), and evidence to that effect, such as a study which indicates that young men who support 'traditional' versions of manhood are more likely to use drugs, be violent, practice unsafe sex (Courtenay, 1998). To understand risk and prevention behaviors, we should investigate both men and women's views about gender roles, as well as evaluate the impact of programs . Unfortunately, to date there are few quantitative studies that measure "gender-equitable" norms and behaviors. Most research about men and the role that gender plays in affecting their health is qualitative, which does not easily permit comparisons across different people or groups. Of the quantitative studies, few use psychometrically evaluated and culturally-specific measures. This presentation focuses on the development and evaluation of a measure for gender-equitable norms and behaviors for men, the Gender-Equitable Men (GEM) Scale.

2. Objectives
The objectives of the study are to:

  • Develop and evaluate a measure for gender-equitable norms and behaviors (GEM Scale);
  • Determine the relationship between gender norms, and violence and HIV/STI risk;
  • Follow with an intervention study to test the impact of a program that promotes gender-equitable norms and behaviors among young men, and investigates if violence and HIV/STI risk decreases.

3. Implementation
Stakeholders and their roles: Instituto PROMUNDO, and Instituto NOOS, both Brazilian NGOs, conducted the initial baseline, with technical assistance from the Horizons Program, a USAID-funded global HIV/AIDS Operations Research, led by Population Council and partners. The MacArthur Foundation provided financial support for the initial baseline. The follow-on intervention study is being implemented by Instituto PROMUNDO, with technical and financial assistance from the Horizons Program. Instituto PROMUNDO, PAPAI, ECOS, and Salud y Genero, designed the program under evaluation.

Cost: The intervention study cost $180,000.

Challenges and Opportunities: This was a first attempt by the research and intervention team to develop such a measure for the Brazilian context and it was unclear whether the measure would be sensitive enough or would be applicable in other cultural settings.
However, there was substantial local and international interest in developing such a measure. Creating a scale was appealing because it would be a useful tool for evaluating the impact of Program H, to explore how gender dynamics are related to violence/HIV risk, etc., and if to explore if these dynamics can be influenced with interventions similar to Program H. As there were no appropriate measures available, particularly for this cultural context, HORIZONS and PROMUNDO worked together to develop a scale. An evaluation of the program, using the scale, is now ongoing

Defining "gender equitable men": The definition of whether or not men are gender equitable is based on the results of intensive qualitative ethnographically-based research with young men in Brazil. The research involved observing and interacting with 25 young men in a favela, a low income neighborhood, in Rio de Janeiro. The methodology required following these young men (ages 15-21) two days a week for one year, interviewing family members of some of the young men, conducting life history interviews, focus group discussions and key informant interviews (Barker, 2001). The study concluded that gender-equitable men have the following characteristics:

  • Support relationships based on respect, equality, and intimacy rather than sexual conquest;
  • Are involved fathers, both financially and in care-giving;
  • Take some responsibility for reproductive health and disease prevention;
  • Oppose intimate partner violence.

Based on this local research (Barker, 2001) plus various qualitative studies worldwide that discerned similar characteristics, the GEM scale was developed to include gender norms around the following domains:

  • Home and child-care
  • Sexual relationship
  • Sexual and reproductive health
  • Violence
  • Homophobia and relations with other men

The scale items were administered to 749 men between the ages of 15-60, with an over sampling of young men (15-24 year olds) to allow for more analysis of the youth. The questionnaire was administered via a household survey to a random sample of men in three neighborhoods. The households were selected using census tract data, and one man was interviewed per household. The research team, which was all male, used house-hold community surveys and conducted face to face interviews in low and middle income communities in Rio de Janeiro. The surveys included 35 gender-equitable norm items and questions about key health related behaviors, such as partner violence and condom use.

Questions that are theoretically related to gender-equitable norms, such as socio-demographic status, history of physical violence, and current safer sex behaviors were used to analyze statistical associations with the gender norms.

Building a GEM scale with findings in the statistical analysis: Factor analyses and coefficient alpha tests were utilized to determine which of the gender norm items would be included in the GEM Scale. Analysis supported the hypothesis that there are multiple important domains within gender norms that should be taken into account, and the resulting scale maintains good reliability. In addition, the GEM Scale is associated with key outcome behaviors. For example, those young men who reported more equitable norms were significantly less likely to report partner violence. This supports the validity of the scale.

LESSONS LEARNED

  • The GEM Scale appears to be a useful measure for gender-related norms and behaviors among both youth and adults in Brazil. (It is being tested elsewhere to see if it is universally applicable.) It seems to be sensitive to assessing men's attitudes about gender norms. Furthermore, the scale provides a way to measure attitudes at baseline and whether there are changes due to an intervention.
  • The study provides evidence that holding inequitable gender norms is associated with reporting risk behaviors such as partner violence and lack of condom use. Support for more gender-equitable norms are associated with less partner violence among youth and more condom use among adults.

    Condom use at last sex with primary and second partner was measured. For adults, condom use with a secondary partner was significantly associated with the gender norms scale (more gender equitable, more condom use). For youth, partner violence (have you ever...hit, kicked, etc…with your current or last primary partner) was associated with the gender norms scale (more gender equitable, less violence).
  • Adults are significantly more likely than youth to support gender-equitable norms. Younger men reported more inequitable norms than older men, possibly because they have less real experience in relationships and do not realize they have to negotiate or compromise.
  • To develop the scale, questions were based on the qualitative research that was done with young men in the community, and a community survey was developed and tested before applying it to the young men who participated in the intervention.

    The community survey provided evidence that gender norms (as measured by this scale) are associated with HIV risk behaviors, such as reported violence and condom use.

References

Barker, G. 2000. What about boys? A Review and Analysis of International Literature on the Health and Developmental Needs of Adolescent Boys. World Health Organization, Geneva.

Barker, G. 2001. Peace Boys in a War Zone: Identity and Coping among Adolescent Men in a Favela in Rio de Janeiro, Brazil. Doctoral dissertation, Erikson Institute (Loyola University-Chicago).

Courtenay, W.H. 1998. Better to die than cry? A longitudinal and constructionist study of masculinity and the health risk behavior of young American men. (Doctoral dissertation, University of California at Berkeley). Dissertation Abstracts International, 59(08A), 232 pp. (Publication number or AAT 9902042).

Chapter 7 | Chapter 9
Table of Contents