Cross Design Synthesis

With the establishment of the Agency for Health Care Policy and Research in 1989, Congress launched an "effectiveness initiative" intended to improve the quality of health care through research-based development of national guidelines for medical practice. Credible guidelines require research results that are both scientifically valid and relevant to the conditions of medical practice. The elusiveness of such information, however, is becoming increasingly apparent, and the task facing the new agency is a difficult one. This report assesses ways to determine how well medical treatments actually work and presents a new strategy--"cross design synthesis"--for medical effectiveness research.

GAO found that: (1) randomized studies and data-base analyses are complementary medical study designs; (2) randomized studies are designed to achieve a valid comparison of alternative treatments, but results may not be fully generalizable to the varied conditions of medical practice; (3) to minimize the risk of overstating a treatment's average effect in the patient population, results of existing randomized studies should be subject to an in-depth assessment of their generalizability; (4) data-base analyses are potentially weak because patients are not randomly assigned to alternative treatments; and (5) multiple assessments of comparison bias can minimize imbalances in data-base analyses and indicate and enhance the usefulness of data-base results. GAO noted that: (1) it devised a strategy that extends the logic of meta-analysis or quantitative overview, which specifies ways of combining results from similar studies; and (2) the new strategy, cross design synthesis, combines results from studies complementary designs in order to capture the designs' strengths and minimize the studies' weaknesses.