Weertz                                 Collecting Evaluation Data with Focus Groups

24/F                                        9:45 AM                                                                                Allen

 

Type_Presentation:         60-minute Concurrent Session

PresentationTitle:         Collecting Evaluation Data with Focus Groups

ProgramStrand_Primary:     Research and Evaluation

ProgramStrand_Secondary:   None

 

Presentation_description:

For those involved with program evaluation, the presenter offers a method of gathering qualitative data using focus groups. Included will be descriptive context regarding basic elements of a focus group, how to incorporate qualitative research into program evaluation, and sample protocol which you may tailor to your own program.

 

Session_summary:

This Powerpoint presentation supports evaluation/research using a mixed method design approach -- combined qualitative and quantitative evidence -- which shifts the measurement paradigm from input/output-oriented evaluation to participant-oriented evaluation. The presentation purpose is to expound the initial rationale for program evaluation and why a mixed methods approach produces more powerful evaluation results.

 

The session begins with a review of the three inherent reasons for program evaluation: To judge program merit or worth, provide direction for improved program effectiveness/ efficiency, or offer rationale for financial support.  Rather than simply go through the motions of an evaluation process (because that is what is expected or what has been done in the past) itŐs important for program administrators to understand exactly why they conduct program evaluations. Depending on the status of the participantsŐ programs -- whether they have a new program just getting off the ground or a more established program -- they may be evaluating for any one or a combination of the above reasons.

 

The majority of program evaluations within higher education produce a fair amount of quantitative evidence (expressed with numbers and percentages). Despite successful and impressive objective data, this kind of research says little about the actual students the program seeks to help. The inclusion of qualitative evidence (expressed in writing or words), however, enhances the evaluation research in several ways. First, it tests the consistency of other research instruments; the qualitative data backs the quantitative data. Second, it strengthens aggregate data, clarifying and illustrating the quantitative results. And third, a mixed methods design approach aids program evaluation, generating insight and points of view regarding various program aspects, some possibly never considered until this point. In essence, qualitative evidence adds authority to the original evaluation purpose.

 

My goal is to offer a clear, concise method for gathering qualitative data using student focus groups. Participants will be introduced to focus group conventions with an in-depth explanation of quantitative and qualitative evidence, as well as an opportunity to view samples of how both evidence types may be presented in an evaluation summary report. A good portion of the presentation will focus on the nature and basic elements of a focus group, preparation, and technique, as well as a thorough description of the documentation plan which answers the five W questions (who, what, when, where, and why).  Lastly, participants will not only get a chance to listen to some of my focus group interviews, they will also receive a sample script and protocol which they are welcome to tailor to their own program evaluation needs.

 

My presentation is unique because it advocates an additional research method for program evaluators. I believe focus groups are an efficient way to collect qualitative evidence, because this method allows you to reach a fairly large group of students in a relatively short period of time. After attending this 60-minute concurrent session, participants will be able to do the following:

 

1)             Extol the specific reason(s) for their own program evaluation

2)             Point out the difference between qualitative and quantitative evidence

3)             Explain the benefits of a mixed methods evaluation design

4)             List the basic elements of a focus group

5)             Determine whether this research method is right for their program

 

PresenterBio:

Sara received her BA in English from Wayne State University (Detroit, MI) and a certificate in Publishing from the University of Denver (Denver, CO). For many years Sara worked in publishing, researching and writing reference books which were sold mainly to schools and libraries. From the publishing world, Sara worked as a technical writer and methods analyst, producing training and instructional manuals for Kelly Services World Headquarters. She ultimately ended up back at Wayne State where she received her M.Ed. in Instructional Technology with an emphasis in performance improvement. SaraŐs graduate work eventually led to her position as Assistant Coordinator for Tutoring and Supplemental Instruction at Wayne State University. In 2005, Sara moved to San Angelo, TX, to start the Supplemental Instruction Program for Angelo State University.

 

In closing, I have an M.Ed. in Instructional Technology with an emphasis in program improvement (particularly in the areas of program assessment and evaluation). For the past six years I have worked with Supplemental Instruction and Tutoring (three years at Wayne State University in Detroit, MI, and three years at Angelo State University, San Angelo, TX). I have continually used my IT expertise to improve these academic assistance programs. Fall 2007 I researched and piloted focus groups for SI. Much of my presentation is about my research and the pilot.