2011 Evaluation
Heritage Keepers Abstinence Education is 1 of the 41 programs now identified by the US Department of Health and Human Services as effective. It is the first abstinence until marriage program in the nation to be recognized on the current federal teen pregnancy prevention program list.
Since 2009, the U.S. Department of Health and Human Services (HSS) has contracted with Mathematica Policy Research and Child Trends, to conduct an independent, systematic review of programs effective in reducing teen pregnancy, STIs, and related sexual risk behaviors. The review identifies, assesses, and rates the rigor of program impact studies and describes the strength of the evidence supporting different program models. The findings were used to identify program models meeting the criteria for the HHS List of Evidence-Based Teen Pregnancy Prevention Programs.
41 South Carolina schools were involved in the study (posted below) which was comprised of 63% African American students, grades 7-9, both males and females. The study demonstrated a positive behavioral change in Heritage program students initiated sexual activity at a rate that is 1/3 that of students who were not in the program., one year after the Heritage Keepers program was complete.
This is the third study re-affirming the success of the Heritage Keepers Abstinence Education program.
Read the full study: Testing a Predictive Model of the Heritage Keepers Abstinence Education Program Stan Weed, et al. 2011
Top Of Page
2007 Evaluation
In 2007, Mathematica Policy Research published a study of the Heritage Keepers Life Skills Education program. This study was NOT on the Heritage Keepers Abstinence Education program.
The study began in 2001 and ended in 2006. The primary purpose of the study was to determine if the Heritage Keepers Life Skills component had any added benefit on the abstinence education program in regards to abstinence behaviors. Therefore, every student in the study had the Heritage Keepers Abstinence Education program. Program students also had the Life Skills component, while the Control students only had the Heritage Keepers Abstinence program. Therefore, the only difference between the control group and the program group was that the program group had the added Life Skills program. This study did not draw conclusions on the success of the entire Heritage program nor did it study our abstinence program directly.
Three cohorts were studied:
* 2001 Middle School Cohort (had the program for 5 years and averaged age 15.9 at the final follow-up—old enough to answer behavior questions)
* 2001 High School Cohort (had the program for 5 years and averaged age 18.9 at the final follow-up —old enough to answer behavior questions)
* 2003 Middle School Cohort (had the program for 3 years and averaged age 13 at the final follow-up —NOT old enough to answer behavior questions).
Important Note: The third cohort received a revised Life Skills curriculum. Early data from other students in the Life Skills program indicated that the program was not having as much impact on intentions to abstain as Heritage would have hoped. Therefore, Heritage revised the Life Skills curricula to include direct correlations to abstinence in each chapter. The third cohort received the new Life Skills curricula from the start. Also, Mathematica considered the third cohort to be too young (13 yrs old) at the final follow-up to answer questions about their sexual behavior. Therefore, this group was only given questions that reflected their attitudes towards abstinence.
For the first 2 cohorts (2001 Middle School and 2001 High School), there were not any differences in initiation of sexual activity between the Program and Control groups. However, it is very important to remember that BOTH the Program and Control groups received the Heritage Keepers Abstinence program. As indicated in the chart below, a very high percentage of students in both the Program and Control groups were already practicing abstinence, so it would have been difficult to improve the Program kids much further.
When both the Program and Control groups were compared to national and state (South Carolina) averages of students in their same age range, the rates of abstinence for both the Program and Control groups (who all participated in the Heritage Keepers Abstinence Education program) are well above average.
|
Age at Follow Up |
Virginity Rates HCS Program/Control Groups |
Virginity Rates SC Average 9th-12th Grade |
Virginity Rates SC Average Same Age |
Virginity Rates National Average 9th-12th Grade |
Virginity Rates National Average Same Age |
|
|
*2001 Middle School Cohort |
15.9 |
73%/72% |
48% |
57.2% (10th grade) |
53% |
48.2% (10th grade) |
|
***2001 High School Cohort |
18.6 |
47%/46% |
48% |
**26.5% (high school seniors) |
53% |
**36.9% (high school seniors) |
|
All state and national statistics are from the Center for Disease Control |
||||||
* 2001 Middle School Cohort: There are indications that a substantial percentage of the young people may have recommitted to abstinence, as the surveys indicate the percentage that had abstained during the year prior to the survey was 82%/78%.
** 2001 High School Cohort: Some of these students in the program group had likely dropped out of high school, and as such were probably at higher risk than the closest comparison group available, high school seniors.
***2001 High School Cohort: There are indications that a percentage of the young people may have recommitted to abstinence, as the surveys indicate the percentage that had abstained during the year prior to the survey was 50%/54%.
The 2003 cohort, who received the revised curricula, could not be tested for their sexual behavior because of their age. However, this Program group reported attitudes that were significantly more positive towards abstinence then the Control group. The 2003 Middle School Cohort: 1) reported views more supportive of abstinence and less supportive of teen sex than did their counterparts, 2) reported greater confidence than did their counterparts in their skills for refusing sex, and 3) reported higher expectations to abstain than their counterparts. According to proven social science research, adolescents with strong positive intentions towards abstinence are more likely to abstain than adolescents that do not have strong positive intentions towards abstinence.
Another Important Note: Across all cohorts, the Control group and the Program group did not differ in their perceptions in the effectiveness of condoms in preventing pregnancy and preventing STDs. They were also no less likely than their counterparts to engage in unprotected sex. Therefore, it is a false assumption to believe that authentic abstinence programs will cause students to have negative contraceptive attitudes.
Heritage believes that any program that is showing an abstinence rate of 72% amongst 15.9 year olds and an abstinence rate of 46% amongst 18.6 year olds, especially when compared to the rates of abstinence for the same ages in South Carolina and the United States, should be recognized as a program with noteworthy behavioral results.
Top Of Page
2005 Evaluation
“A year after Heritage Keepers Abstinence Education, program students reported initiating sex at a rate half that of similar non-program students.”
Stan Weed, Ph.D of the Institute for Research and Evaluation published a study of the effectiveness of the Heritage Keepers Abstinence Education program in 2005. The Department of Health and Human Services, Office of Population Affairs provided oversight to the peer-reviewed process of publication.
In order to encourage the distribution of meritorious research and evaluation, the US Department of Health and Human Services, Office of Population Affairs and the Agency for Children and Families planned the Abstinence Education Evaluation Conference “Strengthening Abstinence Education Programs Through Scientific Evaluation” in 2005. A “call for papers” requested completed evaluative research for presentation. A rigorous system for review was established, utilizing some of the most recognized experts in adolescent and sexuality education (from National Institute of Health, Centers for Disease Control, Mathematica, AHRQ, ASPE and leading universities). The process for review was significantly more intense than that used by most journals. The results were impressive, with over forty submissions sent from throughout the United States. Heritage Keepers was among the seven selected manuscripts, and the manuscript was approved as a good example of a thoughtful program, a carefully designed evaluation and results indicating effective outcomes. The review process, the expert reviewers and the quality of the selected materials are comparable to or exceed the publication standards of most journals. The validity of this process cannot be dismissed unless you would similarly dismiss comparable processes used by the CDC and other government entities which set standards, use experts in the field and recognize worthy programs and evaluative research.
Program students received the Heritage Keepers Abstinence Education program. The Control students did NOT receive Heritage Keepers Abstinence Education or any other Heritage programs (students in the Control Group were from different schools than those participating in the program). One year after the program, 13.9% of Program students had initiated sex, and 25.6% of Control group students had initiated sex. Therefore, a year after Heritage Keepers Abstinence Education, program students reported initiating sex at a rate half that of similar non-program students. This rate held true when comparing the entire set of Program and Control students, as well as in comparisons between subsets of the Program and Control students (subsets included only African American students, only Caucasian students, only males, and only females).
The study of the Heritage Keepers Abstinence Education program was a quasi-experimental design because Heritage Program students were compared to similar Control students from other schools (because the Heritage program is designed to impact an entire school and community, an experimental design using Control students from the same school would not have been a strong research design). The students were matched by various variables (race, age, gender, etc), but were not randomly assigned.
It is important for Program and Control students to be matched on these external variables, but it is more important to also include matching by internal factors, known to the Heritage staff as Dr. Weed’s predictors of sexual activity.
From Dr. Weed –
“A persistent, sometimes formidable, challenge in doing field research is to come up with an adequate comparison group — A reasonably good match between program and comparison groups on the variables that really matter. And to then maintain the integrity of those groups over time. Often, random assignment is not feasible or practical. For example, in this project, we could not find a good way to do random assignment within the school, and eliminate group intermingling, contamination, and ensure group permanence in the school setting.”
Conclusion from Dr. Weed
“The results of this study suggest that a carefully developed abstinence-centered education program can lower the rate at which virgin youth initiate sex. The Heritage program produced a significant and substantial delay in sexual initiation 12 months after the intervention. In addition, a better understanding of the mechanisms that produced this change was realized by including hypothesized mediating factors in the program design and evaluation. These findings support the premise that primary prevention (risk avoidance) efforts to influence teens towards sexual abstinence are a viable strategy.”
The publication is provided below.
An Evaluation of the Heritage Keepers Abstinence Education Program
Stan Weed, Ph.D. November 3, 2005
Tables and graphs are provided below from Dr. Weed’s powerpoint presentation at the HHS Conference.
Health and Human Services Conference – Testing a Risk Avoidance Model
Stan Weed, Ph.D. November 3, 2005
Heritage CEO, Anne Badgley has also provided a report summarizing the findings and the publication process:
Report on an Evaluation of the Heritage Keepers Abstinence Education Program
Anne Badgley, MEd. November 22, 2005