Know when it's time to fire your doctor

Leaving your physician can be a difficult decision
It's OK to consider leaving if your doctor doesn't like questions, doesn't listen
Before bolting, try to express your dissatisfaction, using "we," not "you"

ATLANTA, Georgia (CNN) -- Dr. Jerome Groopman knew he needed to break up with his doctor.
Five years ago, when he started seeing his internist, everything was fine. But Groopman says that in time, the internist became more popular -- and hence more busy and harried -- right when Groopman needed him most.
"I have a strong family history of high cholesterol and heart disease. Every male in my family has had a [heart attack] in his 50s and 60s," he says. "I was moving into middle age, and I just didn't feel that my doctor was looking at me as an individual, and taking those factors into account."
But Groopman -- a physician and author of four books about doctors and patients -- found it difficult to leave his internist of five years. "It sounds strange, but I didn't want to insult him."
Groopman is not alone. "I really think it's a fear of the unknown," says Robin DiMatteo, a researcher at the University of California at Riverside who's studied doctor-patient communication. "But if the doctor isn't supporting your healing or health, you should go."
Here are five ways to know when it's time to think about leaving your doctor, and the best way to do it.
1. When your doctor doesn't like it when you ask questions
Groopman says after the publication of his book "How Doctors Think," a reader contacted him with her story. "She was seeing an orthopedic surgeon for back pain, and when she asked a question, his response was 'Since when did you get an M.D.?'" Groopman says. "That kind of response is just about a deal breaker."
2. When your doctor doesn't listen to you
Debra Roter, a behavioral scientist at Johns Hopkins and co-author of "Doctors Talking with Patients," says it's a red flag when your doctor doesn't pay attention to what you have to say. "A doctor suggested my friend take a certain drug, but she'd taken it before and she told him it hadn't worked for her," she says. "But her doctor wanted her to try it anyway. He didn't give her any credibility."
3. If your doctor can't explain your illness to you in terms you understand
"It's really important that a physician be able to communicate in plain speak and plain language," Roter says. "A doctor has to be able to explain things so you can put the information to use to take good care of yourself."
4. If you feel bad when you leave your doctor's office
DiMatteo says sometimes you just have to go with your gut. "For example, if a patient says, 'My pain is still there,' and the doctor says, 'It shouldn't be -- this treatment works for other people,' and you walk out of the office feeling badly, I don't think you should stay."
5. If you feel your doctor just doesn't like you -- or if you don't like him or her
"Sometimes there's chemistry and people click right away, and there are some people you don't click with," Roter says. "If your gut says you're not crazy about your doctor, they probably aren't crazy about you, and that's not good."
Groopman agrees. He says a doctor who doesn't like a patient often stereotypes him or her. "I was terribly guilty of this as a young doctor. One of my patients said she had indigestion, and I got very irritated with her, and thought she was a whiner and a complainer," he says. "It was catastrophic because she actually had a torn aorta."
The woman died. "I have never forgiven myself for failing to diagnose it," he writes in "How Doctors Think." "There was a chance she could have been saved."
So once you've decided it might be time to divorce your doctor, how do you do it? First of all, make sure whatever's bothering you isn't just a one-time thing. "Make sure it's not just a quirk of the doctor's day," Groopman says. "Maybe they're just having a bad day."
If the problems continue, Groopman, Roter, and DiMatteo agree it's best to try to express your dissatisfaction instead of just bolting. "Use the first person plural, such as 'We're not communicating well' as opposed to 'You seem distracted or irritable with me,'" suggests Groopman. "That may cause cause the physician to stop and reflect and shift gears."
When it doesn't, you can be sure it's time to get another doctor, Roter says. She described two friends who wrote letters to their doctor saying they were unhappy with some of the treatments they'd received. "The both got back letters saying, 'Good luck with your new doctor.'"

Elizabeth Cohen is a correspondent with CNN Medical News.