Do we really need mystery shoppers in health care?

I was a little bit shocked by this AP item in the San Franscisco Chronicle (AMA to consider endorsing undercover patients)

Lori Erickson-Trump has faked headaches and back pain. She’s had physicals and MRIs she didn’t need and she gets paid for it,” all to evaluate the performance of doctors and their staffs.

Hospitals and health clinics are increasingly turning to these undercover patients to grade the health care experience being offered.

Now the ethics council of the American Medical Association is pressing the doctors group to endorse such practices. AMA delegates are expected to vote on the proposal, along with dozens of others, during their five-day meeting beginning Saturday.

I admit to being somewhat ignorant of the mystery shopping/undercover patient concept but my initial reactions are negative:

  • Unnecessary tests and treatment can be dangerous and should be avoided
  • Regular patients should be just as good as mystery shoppers as evaluating health care providers, especially if they are given a bit of coaching beforehand
  • I don’t mind if my doctors thinks I might evaluate them, but I sure don’t want them to think I might be a mystery shopper faking my symptoms!
  • By definition a mystery shopper lacks a relationship with the provider being evaluated. Therefore s/he can’t assess important aspects of the provider’s performance, such as continuity of care, communications with other providers and family, and so on. Just evaluating the initial encounter doesn’t sound so great to me

We already have good patient experience surveys (especially in Massachusetts, through Massachusetts Health Quality Partners) and there are survey firms that can interview patients in more depth.

Maybe I’m missing something but I think the AMA should say no to this idea and send the mystery shoppers back to the hotels, department stores and burger joints where they belong.

June 13, 2008

14 thoughts on “Do we really need mystery shoppers in health care?”

  1. Hi David,
    I thought I would take a moment to reassure you and perhaps address a few of your concerns, as in your comments you indicated you are in unfamiliar territory.

    I have been a healthcare mystery shopper for about five years now. My health and best interests have never been “endangered” by what I have encountered in appointments, and the award-winning firm who hires me would NEVER put me or any other shopper—or any other patient, for that matter—in an at-risk situation.

    Second, “regular patients” are not getting paid to memorize names, make mental notes of details, and later fill out thorough evaluations that include ratings of each employee and the facility encountered and a detailed three to five page accompanying report. In other words, everyday patients aren’t going to remember, much less be able to evaluate, those things the healthcare client wishes to know about. The information collected from mystery shop reports is specific to what those healthcare providers have requested and the numbers and info provided allow them to measure results easily over the long term. We only rate them on what they have requested to be rated.

    As to your concern about not wanting your doctor to think you are a mystery shopper, I would ask you: why not? Medical staff would do well to wonder if each and every patient is a shopper–they would then be certain to consistently deliver the best of care to each, wouldn’t they? How can the finest care possible be a “bad” thing?

    Finally, I would point out that, inasmuch as I am a human being, I am a healthcare consumer. I don’t “lack a relationship” with the doctors, I’m just like any other new patient. As a mystery shopper, I’m not “out to get” doctors or staff. As a matter of fact, I am always on the lookout for those who are doing it RIGHT and, in my reports, often point employees deserving of awards.

    We are not heartless spies dedicated to getting people in trouble; actually, it’s the opposite—we are trained professionals hired by the healthcare providers to give them a true glimpse of the patient experience through the patient’s eyes. We’re there because they WANT us there. If staff were forewarned we were coming to specific appointments, the systems or clinics or CEO docs “in the know” wouldn’t get an accurate reflection of what the everyday patient “initial encounter” is like. Surveys cannot provide information like this because the detail of each individual experience is missing.

    David, I hope I’ve helped to assuage your fears and showed you the good points about healthcare mystery shopping. Our work helps to show the healthcare providers who hire us where they are getting it right, where they can improve, and how they can beat the competition. It is win-win for everyone involved, and it is making healthcare in America better overall.

  2. My primary objection to healthcare mystery shopping stems from the fact that physicians learn through experience. Their decision-making processes are an integration of every patient they’ve ever had. “Fake” patients who complain of odd symptoms alongside conflicting test results not only waste the physician’s already limited patient time, but they also become part of the physician’s medical experience, thus affecting his or her ability to accurately assess “real” patients.

  3. Lori,

    Thanks for your informative comments. I am confident that you are a competent professional and that your clients receive value from the services you provide. I don’t think you’re out to get anyone and I’m sure you’re not heartless.

    I’m still uncomfortable about widespread use of mystery shoppers in health care, though:

    1. I think it’s fine that my doc knows I may evaluate him but I don’t want him to suspect I’m faking my condition.
    2. You may have the same relationship as any new patient, which means you don’t have much of a relationship and can’t evaluate long-term experience.
    3. Since you’re not being put at risk you’re missing out on a lot of what goes on in health care, i.e., testing and treating.
    4. Although I’m sure you’re much more observant and organized than any regular patient, I’d rather have feedback from dozens or hundreds of patients rather than a single mystery shopper

    Thanks again for sharing your perspective!


  4. Lori & David,

    Thank you for your succinct comments. I’m certain they will help many have a better understanding of healthcare mystery shopping, the need and value for all involved.
    To further define Medical Mystery Shopping for David and others, I’d like to point out additional areas trained healthcare mystery shoppers are valued.

    Specialty or “fee for service” practitioners such as plastic surgeons, ophthalmologists offering Lasik surgery, cosmetic dentists and dermatologists use healthcare mystery shopping to measure their level of patient care. They offer elective procedures and recognize the need and value in mystery shopping their facilities. This practice should not offend anyone as the shopper is there for a noninvasive consultation and cannot be viewed as endangering anyone or compromising the “regular patients” care. These professionals have done their research and know for decades mystery shopping has proven to increase referrals, client/patient retention and reduce employee turnover.

    Nursing homes and assisted living communities also employee healthcare mystery shoppers to tour the facilities and make observations. They recognize areas in need of improvement or as Lori mentioned “catch them doing it right”.

    I agree patient surveys are an excellent means to obtain information from a larger population and they work well with mystery shopping to ensure exceptional patient care however the mystery shoppers do have excellent observation and writing skills as clearly demonstrated above. In addition, no emotional attachment to the provider allows for a subjective, unbiased evaluation.

    It matters not if it is a family practice, hospital, nursing home or plastic surgeon, the practice of healthcare mystery shopping will ensure a better patient experience for all and conducted professionally will not compromise the real patients’ care.

  5. My initial gut reaction was that this practice was unethical as it seemed that patients were getting MRIs, tests and x-rays that weren’t needed.

    If it is just a matter of evaluating how one is approached as a patient, that would be different.

    I also wonder how a “mystery patient” can get into a closed practice or get a timely appointment when it often takes three months to get in to see a specialist (dermatologist, Ob-Gyn for example), if you are a new patient.

    Is this limited to specialty practices and long term care facilities?

    The one advantage I can think of is that a “mystery patient” doesn’t have the level of anxiety that a “real” patient would have and could easily remember names and titles. Someone who is undergoing an MRI, for example, is more concerned about what might be found than the name of the technician.

    Who hires the firm(s) that provide the “mystery patients”? What kind of firm is it? Marketing? Legal?

    I’d also like to know what it takes to be hired as a “mystery patient” – I would think that a fair amount of acting ability would be helpful – seriously! : )

  6. Many of my unhappy experiences with medical services have come after sequences of treatment. So I don’t see how the “shopper” can really get the best evaluation given that he/she is a “new” patient.

    In my experience the first visit is usually the best, and after that they throw you to the mercy of the staff.

    So this approach might be good for some criteria, but to really get the lowdown on how a system works I think you need to survey somebody who had been through the mill for a couple of months.

  7. As as long as profit motive is what fuels our healthcare system, those who profit from the system will always have an incentive for maximizing that profit. In a doctor’s office, that includes things like number of patients per hour, efficiency of tests ordered and, of course, marketing. If you can show some data that shows your doctors are somehow “better” or “more compassionate” or something, you can develop a brand to sell.

  8. Firstly, I’d like to make a correction to my earlier post “an objective, unbiased evaluation.” I apologize for any confusion.

    As an owner of a healthcare mystery shopping company, I’d like to take the time to address some of the questions above. Kim, with regard to getting a timely appointment it varies from practice to practice but it is usually not a concern. The provider or managed care group hire us so they understand they may have to wait for the information we provide. There are also other means to gather valuable information without the need for an appointment.

    As for who hires us, it is usually the providers themselves to measure and improve their patient care. My firm has also been hired by medical marketing firms, practice management consultants, nursing home management and medical manufacturers that offer healthcare mystery shopping as “added value” service or a marketing tool for their clients. The providers are aware a mystery shopping program has been implemented they simply don’t know when or whom it might be.

    We look for experienced, MSPA certified mystery shoppers and train them specific to healthcare. We require excellent writing and observation skills; dependability is also a must. Acting skills don’t hurt however most often the shopper is seeing a provider with a valid aliment or consultation that won’t encourage further invasive testing. We carefully examine the office procedures and predetermine the appropriate scenarios.

    Ann, you are absolutely right; it is very important to survey patients during treatment and in addition to shoppers portraying new patients we conduct actual patient surveys. We provide the questions we will be asking and ask they review them so they may be somewhat prepared. We of course have signed permission to contact the patient and we interview them at different intervals of their treatment. It is simply a phone call for the patient; we provide the written evaluation to the client. Their identity is strictly protected so in the event they have developed a relationship with the provider(s), they will not be uncomfortable being completely candid. The calls are handled with privacy, kindness and compassion. The patients understand they are contributing to the betterment of their treatment and healthcare overall.

  9. One of the offshoots of the mystery patient debate has been the animosity generated by some to the point of creating derogatory terms to describe the practice such as “sham patients.” I am amazed at how we got to this point. How can mystery shopping be such a threat if the ultimate goal is to improve the patient experience? I can attest to the fact that from the AP wire story, Perception Strategies has received hundreds of inquiries from prospective shoppers; most of whom share a desire to be a part of changing the healthcare environment for the better.

    I started Perception Strategies ten years ago because, as a hospital marketing director, I realized that in order to move market share at my hospital I needed to have a stronger sense of how people perceived us. If what we were advertising was not consistent with the experience patients/consumers were receiving, we were simply throwing money away.

    So I first had to ask myself, can I get it done internally. For instance, as an employee could I walk up to the front desk and be treated like anyone else? Probably not. Could I rely on an honest self-assessment by staff or department management? Not without them feeling exposed. I had to find an objective means of giving us feedback. Borrowing from the retail-concept of mystery shopping, we were able to develop a research model that allowed the organization to focus on the specific behaviors and processes that most affected patient perception. It is the only way, short of a detailed letter to the CEO of the organization, to identify the few “chinks in the armor” that mean the difference between a loyal patient and a lost patient.

    I realize that the greatest concern is over the means by which we conduct mystery shopping. As Jodi mentioned, we are hired by the hospital, practice, clinic, etc. to help them improve their delivery of care, and there are various ways to do this that doesn’t tie up staff and resources. But even if it did, isn’t it a small price to pay to insure that healthcare organizations take corrective actions that result in improved patient experiences?

Leave a Reply

Your email address will not be published. Required fields are marked *