January 17, 2011 Assessing Student Goal Attainment »
Part I in a Series on Small & Mid-size Ways to Improve Quality & Efficiency. This series explores executive level innovations designed to improve institutional quality and efficiency in programs designed for working adult students.
For the purpose of this Briefing, "students" are adults attending adult-centered and professional programs. For the most part these are students who are engaged in meaningful and career-centered work; have acquired adult social, civic, and family responsibilities; are responsible for personal financial decisions larger than their educational expenses; and are informed consumers.
Along the way and at the conclusion of their course of study, they will appraise the quality of their experiences with your institution, especially the extent to which your program was instrumental in attaining their goals.
Adult Students Have Adult Goals
Working adult and professional students enroll in your program for specific reasons. These reasons are tied to concrete personal and professional goals. Whether we would agree with the judgments of a particular student, their legal and social status as adults confers a sense of finality on these judgments and an equal sense in which they are removed from our purview. How important are students’ goals to those who provide education services to them? Judging by our behavior, they are not at all important. We seldom capture them and, if we do, we fail to do so in a way that is measurable or plays a material role in the services we deliver to them.
How important are students’ goals to the programs that serve them? Judging by our behavior, they are not important.
On the other hand, we pay a great deal of attention to our goals, some of them quite trivial, and insist that students pay attention to them.
Goal Attainment Assessment
What if we were to determine and objectify each student’s goals around the time of matriculation? What if we were to track the achievement of these goals incrementally from matriculation to graduation? What if we were to report progress at defined milestones (calendar and event driven) and do it in a way that was meaningful to students and other stakeholders, especially the instructors, advisors, and retention counselors? What if we were to provide for the opportunity of mid-course corrections to accommodate not only progress to date but the refinements that students make in their goals as they progress in their courses of study? What if we were to assign final “Goal Attainment Scores” upon graduation, and deliver that that information to the student in the form of a “Final report to our client?” What if we also aggregated this valuable information as evidence of institutional quality and as a form of guidance to improve programs, services, instruction, and much more?
What if we were to assign final “Goal Attainment Scores” upon graduation, and deliver that that information to the student in the form of a “Final report to our client?”
Rationale for Goal Attainment Assessment
I can illustrate the executive rationale in the fewest words by using an example that you will immediately grasp and generalize to your own setting. Consider a working adult in a Business Economics course and the two principle stakeholders: the student and the professor.
Stakeholder #1 (the student) – his goal for this particular course is to better hold his own in weekly managerial meetings in which he participates at his place of work. His manager majored in Economics and tends to cast business problems in economic terms. Knowing nothing of economics, but being an otherwise capable employee, this student wants to demonstrate that he has a better grasp of economic constructs.
Stakeholder #2 (the professor) – his goal for that student (one of them) is, in his terms, having a “grasp” of the principles of negative price elasticity. He has no knowledge of or concern with the student’s goals but he has determined that the student will not pass the course without demonstrating this “grasp.” It seems clear that Stakeholder #1’s and Stakeholder #2’s goals are related, even interdependent. But, which one is subordinate to the other? Higher education’s youth-centered canon, which is applied without consent to adult-centered programs by accrediting bodies and the Department of Education, decrees that the professor’s goal is the only relevant consideration. Is this unexamined practice valid on ethical grounds? How about empirical, including pedagogical, grounds?
Validation of Goals Attained
Now, let’s look at the means of goal validation or verification used by the two stakeholders.
Stakeholder #1’s validation consists of being able to “hold his own” in economically-centered discussions in the workplace. His validation and verification will take place in an authentic context, over time, and at the hands of his manager, his colleagues, and his own self-appraisal to which he will apply increasingly higher standards.
Stakeholder #2’s validation consists of three multiple choice questions on a 100 question mid-term examination and two multiple-choice questions on a 150 question final examination. If these two infrequent high-stakes tests (which violate every precept of modern measurement science) are typical, only 45% of the multiple-choice questions will be scientifically valid. Applying this ratio to the five questions means that the determination of goal attainment will rest on two multiple-choice questions, five questions if the professor is among the very few who carefully validate each test item on each test group.
The bottom line here is not that the professor’s goal is irrelevant – far from it. Rather it is that one can make a strong socio-moral case for giving equal or greater weight to the student’s goal. To give appropriate weight to the student’s goals, they must be determined, operationalized, and actively managed within the institution’s system.
The bottom line here is not that the professor’s goal is irrelevant – far from it. Rather it is that one can make a strong case for assigning equal or greater weight to the student’s goal.
Caveats
Implementing a Goal Attainment Assessment System is a manageable process that will deliver many forms of positive return on your investment. Here are the potential problem areas.
Operational Definitions or Scaling – You need to be mindful of the “garbage in/garbage out” risk inherent to this process. The typical student will have five or more goals. Training and structure (templates and instructions) are required to identify and frame those goals so they can be measured for incremental attainment. Once trained, it will take an admissions or retention counselor 10 to 15 minutes to finalize the student’s goals in a face-to-face or virtual meeting. Advance material can – and should – be developed so that the student arrives at the goal setting session having done much of the work in advance. I recommend placing the Goal Attainment work products on each student’s web portal so that the goals and progress toward them are easily tracked and refined along the way. Tracking should include the ability to see refinements of these goals. Students should be encouraged to revisit and refine goals at defined milestones because this refinement is evidence of a successful educational experience.
Goal Migration and Calibration Error – We’re brushing up against measurement science here but you need to know that, left unattended, students will revise their recollection of the original goals as they accommodate their educational experiences. While this is a good thing, it can lead to an understatement of attainment if these changes are not recognized and tracked. On the other hand, if recognized and managed, these errors form another source of evidence that the student has refined his world view in important ways.
Nature of Goals
Encourage an open mind in this area.
Some goals – such as passing a RN NCLEX, the CPA or the Bar – are easily defined. Gaining comfort and confidence in the workplace requires a little more work, primarily by teasing out the specifics of what this means. Some adult students will limit their goal statements to financial metrics. I suggest efforts to look behind these goals, not to replace them, but to supplement them with the underlying goals that likely exist.
Executive Action Items
Become one of the very few institutions that can state with confidence, and in detail, how well your students achieve their goals.
Assemble a team of can-do individuals and come up with a plan to define, measure, and manage the achievement of students’ goals. Incorporate the process in your assessment of student and institutional outcomes. Incorporate it as well in your marketing and in your referral and alumni-marketing programs. Become one of the very few institutions that can state with confidence, and in detail, how well your students achieve their goals.
Questions?
Drop me an email if you have specific questions related to implementing a Goal Attainment Assessment.
Coming Soon
Other small to mid-size initiatives to improve quality and efficiency including Measuring and Managing the Delivery of Instruction, Random Exit Interviews, Initial Experience Assessments, Staying in Touch with Students, Validating Your In-class Assessment Instruments, and more.
Robert W. Tucker, President and CEO of InterEd, Inc., has been leading innovation in higher education since 1986.
He can be reached through this forum.
Reader Comments (1)
This is really a helpful article. We have been learning about incoming students' goals for some time in the enrollment process but the information has not been carried forward. We do see, as you say, how students' goals become more sophisticated as they progress in their program but had not thought about this change as an outcome. Thanks for that as well.