August 25, 2010 « Do-It-Yourself Mystery Shopping »
This Executive Briefing assumes basic institutional experience with front-end analysis, including so called "mystery shopping." It is designed to provide guidance of potential use when developing or refining shopping procedures to test and stress your enrollment function. This Briefing provides empirically tested solutions. To keep this Briefing within a manageable word count, some subtle issues have been ignored. I would be happy to discuss them with you offline.
If you are a colleague, you already know how much importance I place on monitoring all aspects of service delivery, from marketing and enrollment to the management of alumni relations, and adjusting those functions based on what you learn through the monitoring processes.
Continuous quality improvement is not possible without continuous quality monitoring.
While I know of no one who disagrees with this statement, very few colleges and universities (I can count them on one hand) have implemented comprehensive monitoring systems. As important as they are perceived to be, the development of these systems ends up pushed aside by the latest crisis or even the press of daily business.
I am reminded of Covey’s parable of the out of breath lumberman. The dullness of his saw has caused him to fall behind in his work, leaving him no time to sharpen it.
One of the most important areas to monitor (the other being instructional processes), is enrollment. The recent spate of activity in which the GAO, ABC News, and others shopped private sector colleges and universities called renewed attention to the importance of examining how students get into your seats, virtual or otherwise.
Effective monitoring of the enrollment function will generally entail one or more methods to test, flex, and systematically stress, the system to ensure that it:
- It is functioning in accordance with institutional policy and design specifications.
- It complies with applicable regulations.
- It can effectively and appropriately adapt to changing student populations while securing students who have the ability to benefit.
Unacceptable Variance in Enrollment Processes
I have been conducting front-end analyses, including mystery shopping variants, since 1995. In that time I have never seen evidence of willful negligence on the part of the institution’s leadership.What I see daily, however, is failures of training and monitoring to ensure fidelity to institutional standards and practices, and applicable regulations.
The leaders of these colleges and universities are generally shocked to learn how an outlier enrollment advisor has interpreted his role and training to fill in missing gaps with “facts” of his own creation.
Benefits of Shopping
Going forward, many private sector schools will be shopping their enrollment processes in response to the soon to be added layers of compliance mandated by the Department of Education. If you are a private sector institution, you have the choice of outsourcing this work or doing it yourself. Either way, this information may be of help to you.
It would be a shame, however, if only private sector colleges and universities undertake this increased level of monitoring. If you care about continuous quality improvement, specifically getting the right prospective students enrolled in the right programs when the time is right for them, you will monitor the performance of your enrollment function.
The benefits of shopping far exceed basic compliance issues. Among the deliverables of properly executed shopping are increased understanding of:
- Early identification of outlier employees or “enrollment cowboys”
- The impact of training and supervision
- Areas of need for additional training and supervision
- Expectations created by your website and/or third party lead sources
- Congruence between training and the presenting needs of prospective students
- Specific ways to decrease decision cycles and increase conversion rates
- Better ways to assist the student in making the right decision
- Issues that may affect retention
- Appropriate performance standards for enrollment related roles
- Potential for new programs
Ten Tips for Setting Up Your Own Shopping Function
While some institutions will prefer or insist on the third-party imprimatur and other benefits of outsourcing this process, you can do it yourself. Start with the following considerations.
- Identify the goals of the shopping function. Beyond ensuring simple compliance (i.e., not misrepresenting the institution or its deliverables), what do you want to accomplish over the long run by establishing this process? Once you have determined these goals, ensure that they are informed by the shopping process and its metrics.
- Isolate the function organizationally. Create a shopping team consisting largely of individuals who are not connected with the enrollment function. Unless you are a large entity, this function will represent a few hours work per month for a half dozen individuals.
- Report broadly. Enrollment connects to pre- and post-matriculation services, instruction, program development, and other functions. Create reports that are and can be widely disseminated.
- Develop a suite of prospective student personae. This is one of the most difficult challenges of all. You don’t want to follow the lead of the GAO when their fictitious students asked enrollment advisors if they had to report their $250,000 inheritance (a question that has probably never been asked in a real life enrollment process). You do want to present a variety of situations including the messy inputs of having multiple credits from multiple schools and having restrictions on your time and resources. I recommend developing at least a half dozen shopping personae before beginning the process. Systematically vary conditions such as incoming credits, marital status, financial aid requirements, program interests, GPA and test scores (if applicable), and interest in online, blended, or classroom-based delivery.
- Develop adequate collateral. Along with individual prospective student personae, you need to develop collateral to support them. This means email accounts, phone numbers where voicemail can be left, even PO boxes, if appropriate.
- Test and stress all entry points. Most institutions respond differently to phone, email, and in-person inquiries. Test each as appropriate to your normal methods of enrollment.
- Shop at random days, times, and channels. With few exceptions, your enrollment system will function better at certain times of the day and evening or days of the week. Test them all over time.
- Don’t overdo it. Excessive shopping is counterproductive. Divide your enrollment universe into logical or functional segments (online/on-ground, traditional/adult, etc.) and rotate your shopping activities. Rotate across personnel more often than you rotate across other segments.
- Report/revise/report. Set reporting intervals around expectations for change. A typical reporting interval may be 90 days with expectations for change set for the next 90 days. Respond to clear infractions of important rules or policies separately. Set incremental, not wholesale, targets for improvement.
- Do not kill the messenger! With very few exceptions, findings should be taken as an opportunity to improve training, strengthen supervision, revise policy, and improve other aspects of the enterprise. Findings should not be taken as an opportunity to set the most offending enrollment counselor straight. Doing the latter will guarantee that your shopping system will become ineffective. It would be difficult to overemphasize this point. When we do shopping for our clients, leaders always agree with the premise until the finings come in at which time they beg to be told the name of the offending party. This phenomenon, I have concluded, is because leaders are not prepared to place the blame where it belongs – on their training and supervision.
These ten steps will get you started. If you have questions, I would be happy to discuss them with you offline. Be sure to describe what you have done so far (send material if applicable) and what problem you are experiencing.
Good luck!
Robert W. Tucker, President and CEO of InterEd, Inc.,
has been leading innovation in higher education since 1986.
He can be reached through this forum.
The expression of differing views by other leaders is welcomed.
Reader Comments (1)
This is very helpful. Do you have any more suggestions for developing the different "personalities" that test the admissions process?