Evidence-based Nutrition Practice: Gluten-Free Diets
By Cynthia Kupper, RD, CD
Today’s Dietitian
Vol. 8 No. 6 P. 12

Evidence-based practice is a multistep, dynamic process that combines the best external data and clinical judgment. Evidence analysis, the first step in the development of evidence-based guidelines for practice, allows practitioners to critically evaluate the quality of studies and the way they translate to clinical practice. Evidence-based guidelines are used in many areas, including manufacturing and business. Evidence-based medicine (EBM) has existed for more than 50 years but has increased in use since the 1990s, largely because insurers and institutions need to standardize and rationalize care practices in the face of burgeoning research and data collection.

In my work with the Gluten Intolerance Group of North America and the American Dietetic Association (ADA), I have firsthand experience with the development of evidence analysis and evidence-based practice guidelines and have seen how this process can radically change the way dietitians think about and treat nutrition-related conditions. The current controversy about the inclusion or exclusion of oats and wheat starch in gluten-free diets (GFDs) is a prime example of how evidence evaluation can affect dietetic practice. Because it is the field with which I am most familiar, I’ll use it as an example to illustrate and explain the process.

Whether it is safe to include oats and wheat starch in the GFD has been debated for many years in the celiac community. In North America, the avoidance of oats and wheat starch in the GFD has been advocated, while in some European countries oats and wheat starch are accepted as part of the GFD.

During the 2004 National Institutes of Health (NIH) Consensus Conference on Celiac Disease, as an expert in the GFD, I recommended that the evidence on the inclusion of oats and wheat starch be reevaluated.1 Since then, the Canadian Celiac Association has changed its position on the use of oats, and numerous U.S. celiac organizations, research centers, and medical organizations are reevaluating their recommendations as well.2 The ADA recently published the results of its evidence analysis for oats and wheat starch in the GFD on the ADA Evidence Analysis Library Web site (www.adaevidencelibrary.com).3 This is the first step in developing evidence-based practice guidelines.

Evidence Analysis
Quality evidence analysis consists of at least four steps4:

• Problem identification: deciding what questions needs to be answered, such as, “How does the inclusion of oats in a dietary pattern for people with celiac disease impact effectiveness and acceptability of the dietary pattern?”

• Evidence search: finding the best evidence with which to answer the question.

• Critiquing the merit, feasibility, and utility of evidence and grading the evidence.

• Summarizing the findings into a conclusion statement of the evidence.

As professionals, we have been trained to trust peer-reviewed journals—perhaps too much. The fact that a report regarding a scientific study has been published in a peer-reviewed journal does not guarantee that the study design was strong, the study was well-conducted, the analysis of study data was performed correctly, or the results were interpreted properly.5

When an ADA evidence team decides to study a question, they determine inclusion criteria before searching for studies. Inclusion criteria can include numerous factors depending on the subject of the research. Age, ethnicity, time periods for study dates and length of studies, and the number of participants are typical inclusion criteria. With regard to celiac disease, studies not meeting the inclusion criteria were rejected; however, because the evidence team believed many still have an important role to play, their lack of inclusion led the team to add “study limitation statements” to the conclusions for each question addressed.

There are a number of quality evidence analysis processes. An example of evidence analysis can be seen in the detailed report prepared for the NIH Consensus Conference on Celiac Disease in June 2004.6 The ADA evidence analysis process design has been recognized by JCAHO as exemplary.

To date, the ADA evidence analysis team on gluten intolerance has published the conclusion statements on the use of oats and wheat starch in the GFD.3 The conclusion statements and statement of study limitations are included in Table 1.

To fully understand the results, it is important to read the entire report on each question, found on the ADA Evidence Analysis Library Web site. You can also find the complete listing of assessments of the research evaluated and a summary of the grading.

The strength of the research on oats received an overall Grade III, which is weak. The question on wheat starch also received an overall Grade III, which is given when:

• evidence comes from a limited number of studies with weak design for answering the questions;

• evidence from studies with strong designs are unavailable;

• no strongly designed studies have been done; or

• the studies done are inconclusive because they cannot be generalized, are biased, have design flaws, or use inadequate sample sizes.

In the oats question, the research included was relevant to the question asked. Analysis showed that most studies were of neutral quality and some were biased, based on lack of blinding to prevent bias in the participant inclusion selection process. Still, in all but one study, the outcomes were clearly defined, and the conclusions were supported by results of the studies, with biases and limitations taken into consideration.

In the wheat starch question, there were a limited number of studies, most of neutral quality. Participant inclusion bias was unclear in most studies. In three of the seven studies, the outcomes were either not clearly defined or the measurements were invalid or unreliable. Once again, despite bias, all study conclusions were supported by the results.

Developing Practice Guidelines
After the evidence analysis is completed, the next step is to turn the evidence into practice guidelines. Developing evidence-based practice guidelines is difficult and complicated—more difficult than analyzing the research. It involves two important considerations:

• Strength and quality of available evidence. This is important, but someone also has to consider the absence of evidence regarding the effectiveness or safety of a particular healthcare intervention. Absence of evidence does not mean the intervention is unsafe or ineffective.

• A rating of the strength of evidence regarding the effectiveness or safety of a particular intervention does not necessarily provide an insight into the magnitude of the effectiveness or safety of that technology.

For example, there is significant data showing that the inclusion of oats in a GFD is safe for patients with celiac disease and can improve the quality of life and compliance with the GFD. However, the strength of the evidence is weak. There are a large number of studies showing no negative impact of including oats in the GFD, even though the study designs themselves were not the strongest.

The importance of these two considerations is the difference between efficacy and effectiveness. A healthcare intervention is considered “efficacious” when there is evidence that the intervention is beneficial when administered by experts in a research setting. This evidence is usually derived from a controlled study with a narrowly defined population.4

A healthcare intervention is considered “effective” when there is evidence that the technology is beneficial when it is administered by a representative sample of clinicians in normal practice settings to all patients to whom the guideline will likely be provided in real life.

The Need for Guidelines
EBM guidelines have been available for more than 25 years for physicians and are accepted worldwide for a number of medical conditions. The necessity for guidelines arose from the fact that for many years, physicians were treating patients based on anecdotal data, reviews of small, short-term treatment studies, or the opinions of experts. Most recent guidelines are based on the results of well-designed, controlled, randomized, long-term outcome studies. The goal of guidelines is to provide an up-to-date review for practitioners, with the translation of the research evidence put into a practical application for use by practitioners.7

In many areas of traditional dietetics practice, anecdotal evidence, poorly designed studies, and opinions were used. Since there is always a great deal of misinformation being promulgated by unreliable sources, patients became easily misinformed, and practitioners needed to develop a way to rationalize the information.

As support organizations began to seek answers, disagreement over the interpretation of research led to even more confusion for patients, and different guidelines were used for similar situations. In addition, the guideline business has become competitive, with many organizations modifying major national recommendations.7 For more than 30 years, U.S. guidelines for dietary recommendations for celiac disease have been established by national patient support organizations, with and without the input of dietitians. These guidelines were based on anecdotal data, misinformation, or a lack of information, and have occasionally been found to be without merit. Sometimes, there is no way to track why the recommendation was made.

Evidence-based analysis is the answer, but it is not quick or easy. The ADA is beginning to develop new guidelines-based evidence analysis. The ADA Evidence Analysis Library lists practice guidelines for disorders of lipid metabolism and is working on practice guidelines for adult and pediatric weight management; critical illness; chronic kidney disease; chronic obstructive pulmonary disease; type 1, type 2, and gestational diabetes; heart failure; oncology; and nutrition in spinal cord injury.

It is important that dietitians clearly establish themselves as the professional experts in practice areas such as gluten intolerance. The work being done by the evidence analysis teams within the ADA adds the credibility needed to position dietitians as experts. The evidence analysis team for celiac disease anticipates being able to draft practice guidelines for the use of oats and wheat starch in the GFD in the near future. Hopefully, this work will solidify the guidelines for the use of oats and wheat starch in the American celiac patient’s diet, and a universal standard for the GFD will emerge to replace the hodge-podge of recommendations and guidelines now in place.

Turning evidence analysis into practice guidelines requires that someone consider both the scientific evidence and the practical utilization of guidelines in a clinical setting. Implementation of those recommendations allows reevaluation of their effectiveness over time. Thus, practice guidelines will always evolve in a continuing quest to provide the most effective medical care.

Is the System Flawed?
Some argue that while it alerts healthcare practitioners to the importance of high-quality research and helps apply pure research to practice, the evidence-based guideline system has flaws that cannot be ignored.

Genuis argues that evidence-based guidelines have resulted in a reductionist approach to research and illness. He points out that there is a considerable gap between research and the complexity of clinical practice and that evidence-based guidelines have been used to develop “standards of care” documents. These, he notes, have become regulatory tools for insurance companies and healthcare regulatory agencies, hampering the clinician’s ability to individualize care. He further argues that vested interests can impact the translation of knowledge by significantly influencing academia, researchers, medical publications, consensus conferences, and practicing doctors.8

Academicians have argued for many years that guidelines represent a cookbook approach to medicine whether they relate to the treatment of diabetes, dyslipidemia, or hypertension. Physicians, they assert, should not be told what they should or should not do in individual cases.7

While providing an algorithm or guideline may not be ideal, it has been repeatedly shown in carefully conducted clinical trials that a protocol or algorithm approach to management is highly effective. David L. Sackett, on the other hand, argues that “...EBM is not cookbook medicine. Because it requires a bottom-up approach that integrates the best external evidence with individual clinical expertise and patient choice, it cannot result in cookbook approaches to individual care.”9

EBM vs. Traditional Medicine
The key difference between EBM and traditional medicine is not that someone considers evidence while another does not—both take evidence into account. EBM demands better evidence than has traditionally been used. In a paper by White, Robert J. Flaherty, MD, a family physician at the Montana State University Student Health Service, who teaches courses in evaluating the medical literature, says: “Physicians have been using evidence to make medical decisions for over 50 years. The problem was that the research wasn’t always good. People tended to assume that, just because something was shown by a research study and published in a journal, it was correct and could be used in practice. Instead, what we found was that the quality of the information from these studies was quite variable, and we stumbled around for many decades really not knowing the best way to do medical research.”10

A guideline is a consensus document. The best guidelines are based on long-term, randomized, clinical trials. Guidelines should provide both treatment and evaluation advice—ie, what is new, what is practical, and what has been proven. Guidelines that present a proven approach to treatment are helpful in improving outcomes but should never replace individual clinical expertise in deciding whether the evidence applies to a particular patient’s situation and how that information should be used in the care management plan.7,9 In today’s managed care system, there is concern that clinicians are being rewarded not for thinking but instead for following the evidence-based guidelines.

Many guideline reports are too complicated and long. Ample evidence suggests that most physicians’ practices do not reflect the principles of EBM. Instead, their practices are based on tradition, their most recent experience, what they learned years ago in medical school, or what they have heard from their friends.

The average physician is said to read scientific journals approximately two hours per week, and most are probably overwhelmed by the volume of material confronting them. White points out that, on average, physicians spent less than two minutes seeking an answer to a question, relying mostly on fellow physicians, pharmacists, other individuals, drug references, and textbooks.10

It is not uncommon for health practitioners to skim the abstract or highlights of a paper or article to get the information they require. Evidence analysis reports and evidence-based practice guidelines will not be used if they are too long and complicated; guidelines need to be simple and practical. They should also allow for consideration of the patient’s understanding and values in development of a successful treatment plan.

The challenge lies in that it is important to not only find evidence of what works but what matters. As the ADA begins to develop evidence-based guidelines on the inclusion of oats and wheat starch in the GFD, how important is it that the evidence analysis is weak (Grade III) if the results are consistent? Should a patient’s quality of life factor into the decision to include oats and wheat starch in a GFD?

In formulating evidence-based guidelines for oats and wheat starch, the question, “How is including them going to change the patient’s overall compliance, prognosis, quality of life, and outcome?” must be addressed. If the answer is that it won’t change any of these things, then we shouldn’t include them. However, if it changes any of these things, then we must consider that. The patient is a factor in the guidelines, which complicates the process.

In 1994, the term patient-oriented evidence that matters (POEM) was coined by family practice physicians.10 POEM allows physicians to disregard much of the medical literature and focus only on what’s important, which simplifies EBM. They became adept at “information mastery,” the practical application of EBM, combining relevant evidence analysis with practical applications that work in a clinical setting. In the same way, some EBM knowledge is necessary but not sufficient to practice medicine in this age of information.

Using Evidence-based Guidelines
With so many evidence-based guidelines available for health practitioners, it is easier to use them in your daily practice. Here are a few tips for getting started10:

• Know where to look. There are a number of reputable evidence-based medical resources available today. The ADA is one such source. The Evidence Analysis Library topics are regularly updated to keep current of the advances in research.

• Have evidence-based guidelines resources readily available. Members of the ADA can easily access the Evidence Analysis Library. Nonmembers may access portions of the library at no charge.

• Know when to trust what you read. Not all research is created equal. Learn how to quickly identify critical factors of well-designed, strong studies. Read the abstract for hints about the quality of a study.

• Use evidence-based guidelines. At this time, the only practice guidelines available related to celiac disease and the GFD are consensus-based or opinions. However, in recent years, EBM has influenced recommendations made by health professionals and more professional groups are reviewing evidence to make better informed recommendations and guidelines.

A good source for clinical guidelines is the U.S. Preventive Services Task Force (www.ahcpr.gov/clinic/uspstfix.htm). Here you can find evidence-based guidelines on a number of conditions. “You can tell good guidelines at a glance because they grade the relevance and validity of the recommendations and the quality of the research that they’re based on,” says Flaherty.10

• Foremost, don’t lose sight of the patient. Clinicians must learn to integrate the evidence and their clinical expertise and then apply it individually to each patient. A broader definition of evidence-based guideline is, “Evidence-based clinical practice is an approach to decision making in which the clinician uses the best scientific evidence available, in consultation with the patient, to decide on the option that suits the patient best.”10

The role of the nutrition clinician is to determine the best treatment option for the patient. This is done by educating the patient about treatment options and engaging him or her in the decision making. Evidence-based decision making requires that consumers understand their diagnosis and are involved in making decisions about the treatment options available to them. They need to be educated to understand the benefits and risks associated with each option. Gone are the days of telling patients “what” but not “why.”

Using EBM gives you a framework for knowing when to adopt new therapies, when to discard old ones, and how to look for answers to the clinical questions that arise in everyday patient care.

Regarding celiac disease, the ADA has a big job ahead. The evidence is weak at best but strong in consistent results: Inclusion of oats and wheat starch in the GFD could improve compliance, palatability of the diet, and quality of life for many patients, but is it for everyone? These are not easy questions to answer, but in the age of information, we must pursue the answers.

— Cynthia Kupper, RD, CD, is the executive director of the Gluten Intolerance Group of America.

References
1. Kupper C. Dietary guidelines and implementation for celiac disease. Gastroenterology. 2005;128(4Suppl1):S121-S127.

2. Canadian Celiac Association New Position Statement on Oats. Available at: http://www.celiac.ca/Articles/PAB%20oats%20060313.html. Accessed March 2005.

3. American Dietetic Association Evidence Library. Available at: http://www.adaevidencelibrary.com. Accessed March 2006.

4. Oncology Nursing Society Evidence-Based Practice Resource Center. Available at: http://onsopcontent.ons.org/toolkits/ebp/definition/definition.htm. Accessed March 2005.

5. Steinberg E, Luce B. Evidence based? Caveat emptor! Health Aff. 2005;24(1): 80-92.

6. Rostom A, Dube C, Cranney A, et al. Evidence Report/Technology Assessment Number 104: Celiac Disease. Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. June 2004. Available at http://www.ahrq.gov

7. Moser M. Random thoughts on guidelines: How are they determined? Are they followed? J Clin Hypertens. 2004;6(7):360-362.

8. Genuis SK, Genuis SJ. Exploring the continuum: Medical information to effective clinical practice. Paper I: The translation of knowledge into clinical practice. J Eval Clin Pract. 2006;12(1):49-62.

9. Keckley PH. Evidence-based medicine in managed care: A survey of current and emerging strategies. MedGenMed. 2004;6(2):56.

10. White B. Making evidence-based medicine doable in everyday practice. Fam Pract Manag. 2004;11(2):51-58.

Examination

1. Evidence-based guidelines should:
a. include exhaustive testing to reach a diagnosis and treatment option.
b. reduce medical costs.
c. stabilize a condition with the least intervention.
d. improve a patient’s outcome, compliance, and quality of life.

2. Evidence-based decision making requires:
a. consumer knowledge and cooperation in treatment decisions.
b. a full review of current literature.
c. use of standardized guidelines for a specific condition.
d. oversight by an external monitoring agency.

3. The American Dietetic Association (ADA) evidence analysis for the inclusion of wheat starch in the gluten-free diet is:
a. Grade I.
b. Grade II.
c. Grade III.
d. Grade IV.

4. Good guidelines:
a. grade the study design, number of participants, and length of study.
b. grade the relevance and validity of the recommendations and quality of research.
c. grade the relevance of the research to the question asked.
d. grade the quality of the research.

5. Potential pitfalls in studies being high quality can include:
a. the study design.
b. how the study is conducted, analyzed, and reported.
c. the study design and how it is reported.
d. how the study results are manipulated to produce the desired results.

6. ADA evidence analysis for the inclusion of oats in the gluten-free diet:
a. is strong with consistent conclusions.
b. is strong with inconsistent conclusions.
c. is weak with consistent conclusions.
d. is very weak and based on opinion.

7. Evidence analysis defines the:
a. quality of study design.
b. quality of study results.
c. quality of study design and results.
d. merit, feasibility, and utility of evidence in the research.

8. Guidelines are:
a. a consensus, based on long-term, randomized clinical trails.
b. based on multiple types of clinical trials.
c. based on consensus of specialty practitioners.
d. based on case studies.

9. POEM means:
a. problem-oriented evidence medicine.
b. patient-oriented evidence that matters.
c. physician-experience oriented medicine.
d. patient-experience oriented medicine.

10. Defining the quality of evidence is an important first step to developing evidence-based practice guidelines.
a. True
b. False