This IMPACT Evaluation Resource is designed to assist campus professionals with the design and implementation of the evaluation of their CHOICES Alcohol Education efforts. Specific guidance and resource assistance is offered for preparing meaningful and appropriate outcome and process evaluations. This resource helps project leaders as they monitor their progress and document their success. Its aim is to
enhance evaluation efforts, as well as to prepare for sharing insights and lessons. The ultimate impact desired through the use of this resource is to increase the breadth and quality of campus-based programs and initiatives, with a focus on developing and enhancing connections across campus with athletics.

• IMPACT Evaluation Resource (pdf)
• IMPACT Materials, Resources, and Documents

Community Evaluation Planning Process:
TUNE UP

This resource complements a training activity and extensive resource for community leaders. The focus is on assisting organization and project personnel, typically with a small budget, with designing and implementing evaluation activities. Emphasizing six areas - Tools, Understanding, Negotiating, Effort, Us and Presentation – this resource assists with evaluation design through the use of worksheets and background materials.

• Tune-Up Booklet Community Evaluation Planning Process (301K PDF)

Qualitative Evaluation
(Qualitative Research: The What and Why)

Emphasizing the important role of qualitative evaluation, this booklet focuses on three methodological strategies: focus groups, interviews and observations. For each of these approaches, background information, suggestions, limitations, skills, and tips are offered; in addition, worksheets assist with preparing individuals to implement these strategies.

• Qualitative Evaluation Booklet (58.3 k PDF)

Step by Step Evaluation Guide

This evaluation resource guide, organized around 14-steps, assists leadership personnel with three phases of evaluation: project preparation, evaluation design, and evaluation implementation. Emphasizing feedback loops to enhance a project’s impact and value, this guide prepares organizers with the background and rationale for each of the steps, including worksheets for preparation and planning.

• A Step-by-Step Guide to planning and implementing evaluation strategies. (709k PDF)

Training on “Doing What You Want With Your Data” and using SPSS-X

This resource is prepared to help community and campus leaders with managing their data. Typically using SPSS-X or other statistical packages, program evaluation data can be analyzed and reviewed using skillful approaches; this requires appropriate preparation and design. Also emphasized are strategies that help in data management activities.

• Training on “Doing What You Want With Your Data” and using SPSS-X. (PDF)

Evaluation and Accountibility

Main Objectives:
• To provide project staff with some broader perspectives about evaluation
• To stress the role that accountability has with evaluation
• To identify specific ways for enhancing a projects’ accountability through evaluation
• To enhance the desire for being reflective about the public trust placed in us
• To provide greater science and grounding regarding whether our programs are making a difference

View Presentation (2.5MB PPT)

BACCHUS & GAMMA PEER EDUCATION NETWORK

• COLLEGIATE TOBACCO PREVENTION INITIATIVE is a multi-pronged approach to evaluate campus-based tobacco prevention and policy implementation efforts has been conducted. This includes revision of an existing student attitude and behavior survey, data collection from students on multiple colleges in Colorado using an on-line survey, e-mail notification of students, follow-up with non-respondents, data analysis, report preparation, and interpretation of longitudinal findings. The evaluation also includes the development, compilation, and reporting of campus and community strategies using an Environmental Scan instrument. Further, the evaluation incorporates interviews with efforts and community leaders. The synthesis of these various evaluation strategies results in an overall report of efforts and monitoring of progress. See also http://www.tobaccofreeu.org
• IMPAIRED DRVIVING PREVENTION INITIATIVE project entails working closely with campuses striving to reduce impaired driving among their students. It includes the development of a clear logic model, assistance with development, preparation, and implementation of a student survey, review of findings, implementation of an Environmental Scan addressing policies, programs, and services, and overall implementation of grounded initiatives to reduce impaired driving. See also http://www.friendsdrivesober.org/

BOAT PEOPLE SOS

On Sunday, June 8, 2008

The Boat People SOS

presents

The Four Seasons (pdf)

at 3:00PM at

George Mason University's

Center for the Arts, Concert Hall

For more information, click here!

Co-sponsored by
George Mason University's
Center for the Advancement of Public Health

• TEENS WAIT. This includes overall evaluation of an abstinence education project designed for Vietnamese adolescents. Evaluation activities include overall evaluation, design, assistance with development and analysis of the event, pre-test post-test results, and long-term implementation of a use attitude survey regarding abstinence and related issues. On-going staff consultation and training is integral to this project. See also http://www.bpsos.org/
• PALMA. We assist evaluation with this community-based project focusing on substance abuse prevention through intermediary groups and organizations serving the Vietnamese community. This includes the development of an organizational planning process whereby community groups are reviewed and monitored on a three-tier system that promotes greater self-efficacy and skills to systematically address substance abuse issues. The project also incorporates on-going training and consultation with staff. See also http://www.bpsos.org/
• STEP. We assist in the evaluation of STEP (Survivors Torture Empowerment Program) . The STEP program is focused on older adults and family members as well as others in the Vietnamese community regarding health and mental issues. The program also includes in-depth training of peer companions in which CAPH personnel is involved in planning, implementing, and evaluating. http://www.bpsos.org/

FAIRFAX COUNTY PUBLIC SCHOOLS

• COMMUNITIES THAT CARE SURVEYis a project involved working with school and county personnel on the implementation of a youth survey in the school setting with 8th, 10th and 12th grade students. This included instrument review, training of staff, implementing the survey, program observations, preparation of data packets, collection and secure storing of student surveys, and review of findings. See also http://www.fcps.k12.va.us/DSSSE/SDFY/safe_drugfree.htm

INOVA KELLAR CENTER

• OUTCOME STUDY. This includes review of adolescents involved in two of the Center’s programmatic services: day treatment and substance abuse services. The study included training of the Center staff that conducted the interviews, as well as interviews with prior youth participants. The data analysis and recording incorporated review of results gathered from case file reviews and the telephone interviews with prior participants and/or their parents or guardians. See also http://www.inova.org/

MANASSAS CITY PUBLIC SCHOOLS

• STUDENT SURVEY. On a biannual basis, this student-based evaluation addressed alcohol, drugs and violence issues among middle and high school students. We developed the instrument in conjunction with school personnel, making revisions each time the instrument was implemented by school personnel. We coded, stored, analyzed, interpreted, and recorded on results gathered from over 2,000 students each survey administration. This also included talking with staff in the interpretation of findings, which were then used for program and policy review. See also http://www.manassas.k12.va.us/

MARSHALL UNIVERSITY

• PROJECT IMPACT. Evaluation incorporated the development, implementation, analysis and reporting of two campus-based student instruments. One instrument was for pre-test post-test design implemented with freshmen seminar courses using comparisons among four types of classroom settings with first-year students. The other instrument was designed, implemented, analyzed and reported for students campus-wide to assess the longer-term impact of the project implementation. The campus has incorporated training of staff and intermediaries on evaluation strategies. See also http://www.marshall.edu/

NORTHERN VIRGINIA AIDS MINISTRY

• HIV/AIDS AND SUBSTANCE ABUSE PREVENTION AMONG YOUTH. Developed multiple strategies and protocols to monitor outcomes and processes among youth involved in HIV/AIDS and substance abuse prevention program. This includes development of a logic-based, grounded instrument to assess impact regarding knowledge, attitudes, behavior, and intentions among youth leaders participating in training. All resources developed were culturally- and age-appropriate. It also includes analysis, interpretation and reporting of longitudinal data among youth participants. Further, data collection and reporting among workshop attendees is conducted. Normally, focus groups, staff interviews, and observations of activities are conducted. See also http://www.novam.org/

NORTH-AMERICAN INTERFRATERNITY CONFERENCE

• ALCOHOL SUMMITS. As evaluators of this project designed to change the campus culture around alcohol through proactive leadership by fraternity and sorority members, CAPH personnel designed and implemented an outcome and process evaluation. Initial funding for the project was provided by the U.S. Department of Transportation’s National Highway Traffic Safety Administration. The design included stakeholder interviews, staff discussions, observations, pre-test/post-test instrumentation, and compilation of insights and recommendations for program implementation. See also http://www.nicindy.org/

OHIO TOBACCO PREVENTION INITIATIVE

• Using the student survey prepared for the BACCHUS & GAMMA Peer Education Network, CAPH hosted on-line data collection for Ohio colleges and universities. This includes notification of students, follow-up with non-respondents, hosting an on-line survey, data cleaning, and summary reporting. See also http://www.tobaccofreeu.org

The PRINCE WILLIAM HEALTH PARTNERSHIP AUTHORITY (PWPHA)

• PWPHA SKIP. The Prince William Health Partnership Authority SKIP program focuses on youth regarding their perceptions and behaviors on a wide variety of topics.  These include the use of alcohol, tobacco, and other drugs, as well as self-esteem and the relationships the youth have with their parents or caregivers. CAPH services as the evaluators for this project on the various topics the SKIP program addresses.

THE CENTURY COUNCIL

• PROMISING PRACTICES: CAMPUS ALCOHOL STRATEGIES. Specific evaluation with college campuses included telephone interviews and follow-up assessments to determine ways in which campuses have implemented strategies and used resources affiliated with the Promising Practices project. See also http://centurycouncil.org/

U.S. CENTER FOR SUBSTANCE ABUSE PREVENTION (CSAP)

• CULTURAL COMPETENCE ASSESSMENT. Through use of written surveys, focus groups, staff interviews and dialogue with leadership, an overall assessment of cultural competence issues and issues was prepared for CSAP in conjunction with a local firm. This resulted in training initiatives undertaken with staff and the identification and provision of resource material. See also http://prevention.samhsa.gov/

U.S. DEPARTMENT OF EDUCATION

• EVALUATION. Training on evaluation and accountability including keynote and workshop sessions on promoting outcome based strategies and meaningful results. The emphasis has been upon promoting greater value with projects, and enhanced attention to accountability with projects and activities. See also http://www.ed.gov/about/offices/list/osdfs/index.html

VIRGINIA DEPARTMENT OF ALCOHOLIC BEVERAGE CONTROL

• UNDERAGE DRINKING PREVENTION EFFORT. Strategies to address underage drinking have been evaluated with a community-based focus for localities throughout the state. This includes an overall “menu” of strategies and potential measures for these. Also developed have been an online community opinionnaire, pre/post tests for law enforcement, assessments of merchant activities and training, and youth surveys were developed. These are compiled in a resource for communities entitled “Evaluation of Community Underage Drinking Efforts.” See also http://www.abc.state.va.us/
• COLLEGE CONSORTIUM. Campus leaders are engaged in a set of consortium activities with an evaluation of their programmatic effectiveness. Strategies included a campus leader survey, leadership discussion groups, individual interviews, and review of program effectiveness. See also http://www.abc.state.va.us/
• COMMUNITY EVALUATION EFFORTS. A range of community-based tools and resources was prepared for communities addressing drug and alcohol issues, particularly communities with limited evaluation budgets. The evaluation efforts also included the preparation of “TUNE UP” training and resources. See also http://www.abc.state.va.us/

VIRGINIA DEPARTMENT OF MOTOR VEHICLES

• COMMUNITY ASSESSMENTS. Needs assessments and potential strategies to address traffic safety issues in regions throughout the state were implemented. These included discussion groups, telephone interviews and written assessments. See also http://www.dmvnow.com/

YOUTH FOR TOMORROW

• EVALUATION PROJECT. This outcome and process-based evaluation is designed to assist project staff with a residential treatment program for youth. It includes interviews with graduates and attendees regarding their experience with the program including a review of the academic, counseling and overall experience with the program. It also includes interviews with parents or guardians of prior participants, as well as staff interviews and discussions. See also http://www.youthfortomorrow.org