Evidence-Based Medicine: Do Clinical Practice Guidelines Contribute to Better Patient Care? -- An Expert
Interview With Drs. Joseph O. Jacobson and Antonio C. Wolff
Editor's Note: In a field as complex and rapidly changing as oncology, clinical practice guidelines
(CPGs) are an important tool in applying evidence-based medicine to patient care. However, adherence to practice guidelines
varies widely,[1] and some physicians have concerns that guidelines characterize a rigid or oversimplified practice
of medicine.[2]
The rapid increase in requirements to provide accountability for medical care delivery to oversight and regulatory
agencies as well as payers underscores the need to reduce variability in cancer care. Programs such as the American Society
of Clinical Oncology (ASCO) Quality Oncology Practice Initiative (QOPI) represent an important effort to evaluate oncologists'
concordance with standardized practice guidelines and to identify clinical practice areas in need of improvement.
Kate Evans, a contributing writer for Medscape Oncology, had the opportunity to speak with Joseph O. Jacobson, MD,
Chair of the ASCO Quality of Care Committee and the QOPI Steering Group, and Antonio C. Wolff, MD, member of the Cancer Guidelines
Committee of the National Comprehensive Cancer Network (NCCN) and the ASCO Quality Care Committee, about the role and optimal
implementation of oncology practice guidelines in improving patient care.
Dr. Jacobson is Chair of the Department of Medicine and an Oncology Hospitalist at North Shore Medical Center in Salem,
Massachusetts. Dr. Wolff is Associate Professor of Oncology and member of the Breast Cancer Research Program at the Johns
Hopkins Sidney Kimmel Comprehensive Cancer Center in Baltimore, Maryland.
Medscape: What defines quality care in oncology, and what defines evidence-based care? How do these concepts intersect?
Dr. Wolff: Ultimately, quality cancer care is treatment that improves survival and/or enhances quality
of life. It is offering the right care to the right patient, correctly providing care, and, most importantly, treating the
patient and not just the disease.[3] To me, accomplishing those goals in a safe setting where there is some standardization
of management, controlling for unnecessary variation, would be considered quality care.[4]
As far as what defines evidence-based medicine, David Sackett[5] provided a classic definition: "The conscientious,
explicit, and judicious use of current best evidence in making decisions about the care of individual patients." The practice
of evidence-based medicine "requires integration of clinical expertise with the best available external clinical evidence
from systematic research." It is based on a high level of evidence, which is preferably, but not exclusively, derived from
randomized clinical trials.
Dr. Jacobson: It may be best to look at quality of care from 2 vantage points. The first is the individual
patient standpoint: patients expect and are entitled to the highest level of care. There is also quality of care relative
to a population, which is measured differently and has different connotations and consequences.
An example of population-based care is adherence to an evidence-based CPG across a population, where healthcare systems,
hospitals, or insurers would be held accountable for quality of care. In contrast, care at the individual patient level is
best measured according to the performance of an individual clinician or team of clinicians.
Dr. Wolff: We should note that evidence-based medicine isn't necessarily the best medicine. Some payers
look at this as a way to control costs, and we have to be careful because sometimes practicing the best care possible may
actually increase costs up front. Those who are thinking about guidelines need to be very cautious, especially from a regulatory
standpoint, about whether their true motivation is trying to enforce some kind of system of concordance with these guideline
documents.
Medscape: Based on their survey of the medical oncology literature, Smith and Hillner[1] concluded in
2001 that the use of oncology CPGs has been "moderately successful," and that there has been wide variability in adherence.
What is the role of CPGs in improving patient care in cancer treatment?
Dr. Wolff: As a practicing medical oncologist, what made me become interested in CPGs and health services
research was the obvious variability in patient care and the whole issue of dissemination of new information in the clinic.
Oncology is a field in which most of us are accustomed to the idea of high-level evidence and randomized clinical trial
applicability. In clinical practice, most practitioners undergo a great deal of training; they have a lot of clinical experience,
they share information, they read the literature -- but they don't necessarily do the best or right thing for their patients.
Perhaps more importantly, just because one practice intervention is widely adopted doesn't necessarily mean that it's the
right thing. The perfect example is the widespread application of high-dose chemotherapy plus bone marrow transplant for breast
cancer that occurred in the mid-1990s although there was no randomized evidence showing a survival benefit -- except for a
study that was ultimately retracted.[6] Nonetheless, many patients demanded it, and some insurance companies gave
in after dealing with so many lawsuits. It took forever for the clinical trials to be done, but by 1999, randomized evidence
finally became available showing no benefit, and almost immediately the use of high-dose chemotherapy with autologous marrow/stem
cell rescue outside of the context of investigational studies came to a halt and died.
CPGs can be viewed as useful mechanisms to break down complex data sets into more manageable pieces, allowing busy clinicians
to use them effectively for individual patient care -- even though they may not have access to the primary data sources when
they are in busy clinics. That forms the basis of interest in CPGs by many professional organizations like ASCO as well as
some policy bodies.
Medscape: What are some concerns that oncologists might have about applying CPGs in their own practices?
Dr. Wolff: There's increasing interest from healthcare administrators, regulators, and policy-maker payers
in CPGs as standards of care. That's why practitioners fear using CPGs as "cookbook medicine," or as the only correct recipe
for cancer care.
CPGs don't replace individual clinical expertise, and they cannot be used in the absence of information specific to the
patient being treated. Factors such as comorbidities and individual preferences need to be considered. Many patients have
ideas about their care, and what they want to do may differ from clinical recommendations depending on individual goals. In
applying CPGs, it is also important to take into account the characteristics of the system in which one practices and what
resources are available at both the individual and systemic levels.
When I talk about CPGs, I try to put clinicians at ease by saying that just because you are not concordant with a guideline
doesn't mean that you are wrong; you may have good reasons for a clinical decision not to practice in full agreement with
the recommendation. Nobody who develops CPGs expects 100% concordance.
Dr. Jacobson: A concept Brent James[7] describes may be relevant. In his opinion, American
medicine is at a crossroads where 2 generations are clashing. My generation -- physicians over the age of 45 -- were trained
in what James would call the "craft-based" era. Those more newly trained constitute the "profession-based" era.
Craft-based physicians undertake patient care by bringing all their knowledge (medical school and residency training, journal
reading, CME courses, and clinical experience) to the bedside and reaching a clinical decision. The decision, by definition,
will be highly individualized (and variable) since no 2 physicians share exactly the same training and experience.
Profession-based medicine is very different. It is team-based. It is the acknowledgement that medicine is changing so quickly
that no single physician can stay current on any single area of medicine, and a physician must rely on a collection of tools
to ensure that he or she enters the room as up-to-date and evidence-based as possible. With a profession-based approach to
medicine, CPGs are a natural tool because multiple experts have come together to provide a road map or guideline for care.
We are at a point where some physicians will try to avoid CPGs, using pejorative terms like "cookbook medicine." The incoming
generation, raised on the Internet and technologically savvy, will embrace them much more easily and find that they improve
care.
Medscape: What are some of the factors that predict a higher likelihood of success with CPGs?
Dr. Wolff: In the past, guidelines were narrative reviews of the literature, which are subject to all
kinds of ascertainment bias, vs the systematic review methodology we now use. I think guidelines can be very successful when
they address more controversial topics, because those are the issues where you're likely to have more variability in opinions.
Another factor is the credibility of the developer. If the guideline is developed by an entity that has a relationship,
financial or not, with individuals who have a lot to gain from specific recommendations, those documents are likely to have
less credibility. For this reason, ASCO has been very strict about issues of conflict of interest.
A feedback mechanism is important. As [Dr. Jacobson] said, medicine is increasingly a team effort, and there must be representation
not just from the end users -- clinicians in multidisciplinary practice -- but also from patients. Dissemination methods are
also important, because most clinicians don't have time to read long documents. You need to develop tools and educational
materials that the clinician and patient can use efficiently in the clinical setting.
Dr. Jacobson: One of the best ways to improve compliance is to provide physicians with timely feedback
regarding their compliance with CPGs. Physicians are highly competitive. If CPG compliance data are provided along with a
peer comparison, most of us will be sure that we do not fall below the mean.
Medscape: On that note, can you provide some background on QOPI? Why was it established, and how does it assess
the dimensions that constitute quality care in oncology?
Dr. Jacobson: QOPI was established in late 2002. It was based on the vision of a single person, Dr. Joseph
Simone, a pediatric oncologist who was the chair of the National Cancer Policy Board of the Institute of Medicine at the time.
He was struck by the fact that, in contrast to pediatric oncologists, adult medical oncologists seemed incapable of measuring
the care that they provided. He reached out to a group of community oncologists around the country, and the 7 who were willing
to tackle the problem became the QOPI alpha group.[8] QOPI began with limited ASCO administrative support or enthusiasm,
but within 4 years it became the ASCO-recognized program for measuring quality of care. More than 550 practices have now enrolled
in QOPI.
Quality is often measured in 3 domains, which were defined by Avedis Donabedian[9] more than 40 years ago: structure,
process, and outcome. While we are certainly interested in quality outcomes and structural components of care, it was not
within our reach to measure these domains in the community cancer setting where QOPI had its roots. Hence, QOPI's initial
focus was on process. Processes of care are the actual care steps that span the gamut from diagnosis to treatment decision
to treatment implementation and follow-up care.
QOPI measures were developed on the basis of many sources. Some came from an ASCO-sponsored project called the National
Initiative on Cancer Care Quality (NICCQ), some came from the NCCN, and many came from established ASCO CPGs. Some were derived
from a consensus process that we created early on at QOPI; these tend to be common-sense, intuitive measures that are not
evidence-based.
Medscape: Based on the assessments completed to date, what are some of the most common individual and systemic
practice shortcomings that QOPI has identified?
Dr. Jacobson: Let's begin with the good news. QOPI practices consistently demonstrate high rates of concordance
for those measures that assess compliance with high-impact, evidence-based CPG recommendations. These include recommendations
to provide adjuvant chemotherapy for patients with breast, colorectal, and non-small-cell lung cancers.[10] QOPI
results replicate the findings of the earlier ASCO-sponsored NICCQ trial.
At the other extreme, measured concordance rates are lower and far more variable in the end-of-life module. Too few patients
are enrolled in hospice, and a significant minority of patients receives chemotherapy within the last 2 weeks of life. We
believe that QOPI has identified important opportunities for medical oncologists to provide better end-of-life care.
Medscape: Who can benefit from using QOPI, and what are some specific ways that medical oncologists can get the
most out of using the assessment program?
Dr. Jacobson: There are many reasons for a practice to enroll in QOPI. First and foremost, it is a tool
to measure the care we provide to our patients. QOPI lets us see how we are doing, and, importantly, it lets us see how we
are doing compared with our peers.
Oncologists who were board-certified after 1991 must renew their certification every 10 years. In order to be eligible
to take the recertification examination, medical oncologists must undertake a practice improvement program. Participation
in QOPI has been accepted by the American Board of Internal Medicine as a means to obtain the 20 points needed for practice
improvement.
Medscape: As you mentioned earlier, oncology is a field that changes particularly quickly. In that context, how
can clinicians implement standardized CPGs and still appropriately apply new data and clinical judgment?
Dr. Jacobson: I'll refer back to Brent James,[11] with whom I was fortunate to train last year.
According to James, medicine should follow the path that industry has taken; the term he uses is "mass customization," meaning
that most processes can be standardized but few are perfectly suited for an individual patient. Brent James reminds us that
no CPG is perfectly suited for any patient, nor is any patient a perfect fit for a practice guideline.
James argues that, in general, 80% of the care we provide should be standardized, but we rely on an astute, highly trained,
experienced, up-to-date oncologist to provide the final 20% that shapes care and that results in treatment that is both standardized
and individualized.
Dr. Wolff: [Dr. Jacobson] was talking earlier about physicians over age 45 vs those who are younger. I'm
actually 45, and I interact with the new generation of trainees in the first 10 years of their careers. My sense is that they
are very comfortable with the whole idea of CPGs. It also becomes so much easier to use them in specialties that have benefited
from a high level of evidence.
For example, I treat only breast cancer; I think we're spoiled by the amount of available data and systematic reviews like
the Oxford Overview to guide treatment recommendations. So I could be a little bit biased, but at least among the professionals
I spend time with, I don't hear concerns about "cookbook medicine" from younger people. Even the older people are resigned
to it.
Dr. Jacobson: A minority of the older generation will resist CPGs, unwilling to believe that the direction
medicine has taken is the correct one.
I worry a little that the pendulum could swing too far as the next generation emerges -- that they may too easily embrace
standardized care approaches and fail to attend to the nuances of care that define any patient encounter. The degree to which
very newly trained doctors rely upon online review references as their sole knowledge base for medicine is sometimes alarming
to me.
Dr. Wolff: I think you have a point. The use of these review articles rather than primary data sources
is pervasive here at Johns Hopkins University among trainees. We are all human, and we see this with patients and sometimes
with physicians as well -- anything that's available online must be true.
If you're just using CPGs and not using good judgment, you'll be in trouble. Although we're trying to minimize variability,
I don't think the art of medicine will ever die. You need to have common sense, good clinical training, and intuition -- the
latter of which may be simply a summation of knowledge plus experience plus mileage. The importance of those things will never
go away.
Medscape: Can you speculate about the evolution of CPGs and the future of measurement-based quality improvement
tools in cancer care?
Dr. Jacobson: The most effective way to improve compliance with CPGs is to automate the process. Oncology
electronic health records are an ideal vehicle to accomplish this. Decision support tools could trigger clinicians to select
evidence-based treatments based on disease- and stage-specific parameters. In addition, the electronic health record could
be used to track compliance with guideline adherence for an oncology practice or for individual providers.
Dr. Wolff: The key is to make things easier. With electronic medical records, if you have easy access
to high-quality, well developed guidelines and decision-making tools, I think you'll see more and more people comfortable
with using them.
Dr. Jacobson: QOPI currently employs a retrospective chart abstraction methodology that samples practice
quality. The next step for QOPI is to collect data prospectively, to create a comprehensive registry that can capture all
patients treated in a practice rather than just a sample. This will become feasible as more and more practices adopt electronic
health records. We envision a day when QOPI data will be electronically transmitted to ASCO, which will greatly reduce the
cost, inefficiency, and errors associated with manually collected data.
Medscape: Recent news articles have expressed concerns about the provenance of some practice guidelines.[12]
How should clinicians evaluate the credibility of CPGs for cancer care?
Dr. Wolff: There is a concern about a proliferation of CPGs, many of which are being developed by organizations
that don't have a track record or a predefined process, who are maybe too close financially to commercial entities that have
a clear interest in the outcome of the documents. So end users need to ask themselves: Who developed the guideline? Why and
how was it developed, and what was the process?
Those who create CPGs shoulder a great deal of responsibility. I'm biased, but I will say that for the most part I'm very
proud of what ASCO has done in this regard.
Medscape: What is the take-home message for oncologists about the use of CPGs today?
Dr. Wolff: When we're talking about oncology, there are 2 main organizations developing CPGs. One is ASCO,
and the other is NCCN. They have very different processes, different goals, and different missions. I actually happen to be
involved in both, and I'm not going to say one is potentially better than the other because, for the most part, I think they're
both pretty reasonable. In oncology, I think we're on safe ground with regard to the major CPGs.
The take-home message about CPGs is to think about them and embrace them, but always with a bit of caution. Never forget
to be a doctor, to emphasize clinical judgment, and to pay attention to the individual patient.
This activity is supported by an independent educational grant from Susan G. Komen for the Cure.
Disclosure: Antonio C. Wolff, MD, has disclosed that he has received grants for clinical research from Roche
Laboratories Inc.
Disclosure: Joseph O. Jacobson, MD, MSc, has disclosed no relevant financial relationships.
Disclosure:
Kate Evans has disclosed no relevant financial relationships.
