Beware of ‘search satisfaction,’ a common cognitive error

18 may 2008--Daniel Ginsberg, FACP, related to us the case of a 60-year-old woman who presented to an urgent care clinic with lower abdominal pain and urinary frequency. Despite extensive evaluation, the cause of her symptoms was not recognized until four months later. We talked to Dr. Ginsberg about this case and the cognitive errors that likely led to the initially missed diagnosis.
Dr. Ginsberg saw the patient a few days after she visited the clinic. Test results from the clinic visit showed that a urine dip stick was negative for leukocytes and nitrites but positive for occult blood. She was treated empirically with ciprofloxacin but urine culture came back negative. Her past medical history was remarkable for esophageal reflux, bilateral oophorectomy and vaginal hysterectomy, as well as a laparoscopic cholecystectomy, but she was otherwise in good health.
The woman reported feeling better when she first saw Dr. Ginsberg but several days later, her symptoms recurred and a repeat urine analysis again demonstrated microscopic hematuria. A CT scan showed a small mass in the inferior pole of the right kidney, and it was noted that the appendix was enlarged at the takeoff from the cecum, without surrounding inflammatory changes. The radiologist felt the appendix was a normal anatomic variant. Urology was consulted.
About one month later, the patient underwent a laparoscopic right nephrectomy. Pathology showed a benign angiomyolipoma. When she returned to see Dr. Ginsberg three weeks later, she said that the surgery had not resolved her abdominal pain. On examination, she had some tenderness in the epigastrium, the right upper quadrant and the hypogastrium. Dr. Ginsberg attributed this to postsurgical pain.
A week and half later, the symptoms were no better and she had developed diarrhea. The woman was concerned about celiac disease, a condition her daughter suffered from, and was referred to a gastroenterologist. An endoscopy showed minimal chronic gastritis and two gastric polyps, but no evidence of celiac disease. She also underwent colonoscopy, which showed congested mucosa in the cecum; a biopsy of this area revealed non-specific inflammatory changes. She continued to experience intermittent abdominal pain which she described as “gas pains.”
Assuming that her discomfort was related to her prior surgery, the patient did not come back to Dr. Ginsberg’s office until about six weeks later, when she complained of two days of “sharp and gas-like” worsening abdominal pain located in the periumbilical area. On examination, she had mild hypogastric tenderness. A urine dipstick performed in the office showed 0-2 white blood cells and 10-20 red blood cells. A urine culture was sent and the patient was started on levofloxacin.
Two days later, she presented to the emergency room with increased abdominal pain, nausea, and vomiting. In the emergency room, she was afebrile. Tests revealed the following:
Her white blood cell count was normal with a left shift: WBC 9300 with 82% neutrophils
Serum transaminases were elevated, AST 88, ALT 188, and alkaline phosphatase was 265
A repeat CT scan, now four months after the initial CT scan, showed a 5.4 x 3 x 4 cm cecal mass
Malignancy was suspected and she underwent laparotomy and resection of the mass. Final pathology demonstrated an appendiceal abscess.
Dr. Ginsberg described the patient as a very pleasant person, and that he had cared for her and her entire family for some time. She was an accurate and reliable historian. During the course of her illness, he was convinced that something was wrong but was not sure what it was.
sIFR_callback_2_DoFSCommand(info, args);
Discussion: Seeing (all of) what’s there
We as clinicians are strongly guided by technology. Most of us, receiving the information that the radiologist gave Dr. Ginsberg, would have eliminated consideration of appendicitis in this case. But it is worth keeping in mind the cognitive challenges that radiologists face when reviewing images. Ehsan Samei, PhD, of the Advanced Imaging Laboratories at Duke University Medical Center, analyzed results from a variety of radiological procedures based on published data in the literature. “Currently, the average diagnostic error in interpreting medical images is in the 20% to 30% range. These errors, being either of the false-negative or false-positive type, have significant impact on patient care,” his study concluded (J Am Coll Radiol; Vol. 3, p. 400, 2006).
The workload for radiologists has increased significantly over the past decade. It is estimated that radiologists in private practice evaluated from 12,000 to 15,000 cases a year in 1997, but the workload now has nearly doubled. From 1998 to 2002, the number of CT scans in the U.S. increased by 95%, MRI by 51%, and ultrasound by 30% during traditional work hours, and each increased by 15% during on-call off hours (J Am Coll Radiol; Vol. 3, p. 433, 2006). Some cases, like chest X-rays, generate only a few images, but CT and MRI scans involve hundreds or thousands. Herbert Kressel, MD, a radiologist at our institution who is an expert in MRI, pointed out that how the CT or MRI scan is reviewed in a cine (akin to a movie) format can importantly influence the identification of subtle findings. That is, moving too quickly through the images can cause the radiologist to overlook an abnormality.
The difficulties in identifying abnormalities have been the subject of study by several researchers, including E. James Potchen, MD, at Michigan State University in East Lansing (J Am Coll Radiol: Vol. 3, p. 423, 2006), who has studied performance in reading chest X-rays. More than 100 certified radiologists participated in a research study where they evaluated a series of 60 chest X-rays that included duplicates of some of the films. When the radiologists were asked, “Is the film normal?” they disagreed among themselves an average of 20% of the time. This phenomenon is called “interobserver variability.” Remarkably, when a single radiologist re-read on a later day the same 60 films, he contradicted his earlier analysis up to 10% of the time, a tendency referred to as “intraobserver variability.”
One of the most striking findings in Dr. Potchen’s research came from the same study where one chest X-ray out of the 60 was of a patient who was missing his left clavicle. Here, 60% of the radiologists failed to identify the missing clavicle. When clinical data were added to the exercise, informing the radiologists that the 60 chest X-rays were obtained as part of an annual physical examination, 58% of the radiologists still missed it and scored the film as normal. However, when they were told that the chest X-rays were obtained from patients with cancer, then only 17% of the radiologists failed to identify the missing clavicle. The important conclusion is that specific clinical information from the referring physician can substantially improve the performance of the radiologist.
Harold Kundel, MD, of the University of Pennsylvania, has studied the physiology of image perception by tracking the eye movements of his fellow radiologists. The radiologist sits with an apparatus on his head that looks like a bicyclist’s helmet. In addition to a visor, the apparatus has a miniature camera. As the doctor examines a series of images, a beam of invisible infrared light is trained on his pupil, tracking his gaze. By this means, one can distinguish between abnormalities on an image that are not looked at and those that are not consciously processed despite the eye gazing at the site (J Am Coll Radiol, Vol. 3, p. 402, 2006).
These and other studies demonstrate a common error among radiologists called “satisfaction of search.” This means that once an abnormality is identified by the radiologist, his mind is “satisfied” and does not register a second abnormality. In Dr. Ginsberg’s case, the finding of a renal mass may have led the radiologist to dismiss the second finding of an enlarged appendix as merely a normal variant.
Dr. Ginsberg later discovered that the urologist had done a second CT scan. A different radiologist read this scan as follows: “1.7 cm right inferior pole renal tumor, possible renal cell carcinoma. Thickened appendix; this finding is non-specific but may have progressed since prior examination and could represent early appendicitis. Correlate clinically.” Unfortunately, the urologist made no mention of the appendix in his consultation note, focusing only on the renal mass, another example of search satisfaction.
Dr. Ginsberg told us that he viewed this case as “a horse masquerading as a zebra.” Why didn’t we see the horse? To help answer the question, Dr. Ginsberg sent us this picture of how the mind may not perceive a visual abnormality.