VA Patient Safety Program
A Cultural Perspective at Four Medical Facilities Gao ID: GAO-05-83 December 15, 2004The Department of Veterans Affairs (VA) introduced its Patient Safety Program in 1999 in order to discover and fix system flaws that could harm patients. The Program process relies on staff reports of close calls and adverse events. GAO found that achieving success requires a cultural shift from fear of punishment for reporting close calls and adverse events to mutual trust and comfort in reporting them. GAO used ethnographic techniques to study the Patient Safety Program from the perspective of direct care clinicians at four VA medical facilities. This approach recognizes that what people say, do, and believe reflects a shared culture. The focus included (1) the status of VA's efforts to implement the Program, (2) the extent to which a culture exists that supports the Program, and (3) practices that promote patient safety. GAO combined more traditional survey methods with those from ethnography, including in-depth interviews and observation.
GAO found progress in staff familiarity with and participation in the VA Patient Safety Program's key initiatives, but these achievements varied substantially in the four facilities we visited. In our study conducted from November 2002 through August 2004, three-fourths of the clinicians across the facilities were familiar with the concepts of teams investigating root causes of unintentional adverse events and close calls. One-third of the staff had participated in such teams, and most who participated in these teams found it a positive learning experience. The cultural support clinicians expressed for the Program also differed. At three of four facilities, GAO found a supportive culture, but at one facility the culture blocked participation for many clinicians. Clinicians articulated two themes that could stimulate culture change: leadership actions and open communication. For example, nurses need the confidence to disagree with physicians when they find an unsafe situation. Although VA has conducted a cultural survey, it has not set goals or explicitly measured, for example, staff familiarity and mutual trust. Clinicians reported management practices at one facility that had helped them adopt the Program, including (1) story-telling techniques such as leaders telling about a case in which reporting an adverse event resulted in system change, (2) management efforts to coach staff, and (3) reward systems. The Patient Safety Program Process shows how ideally (1) clinicians have cultural support for reporting adverse events and close calls, (2) teams investigate root causes, (3) systems are changed, (4) feedback and reward systems encourage reporting, and (5) patients are safer.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-05-83, VA Patient Safety Program: A Cultural Perspective at Four Medical Facilities
This is the accessible text file for GAO report number GAO-05-83
entitled 'VA Patient Safety Program: A Cultural Perspective at Four
Medical Facilities' which was released on December 15, 2004.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Secretary of Veterans Affairs:
United States Government Accountability Office:
GAO:
December 2004:
VA Patient Safety Program:
A Cultural Perspective at Four Medical Facilities:
GAO-05-83:
GAO Highlights:
Highlights of GAO-05-83, a report to Secretary of Veterans Affairs:
Why GAO Did This Study:
The Department of Veterans Affairs (VA) introduced its Patient Safety
Program in 1999 in order to discover and fix system flaws that could
harm patients. The Program process relies on staff reports of close
calls and adverse events. GAO found that achieving success requires a
cultural shift from fear of punishment for reporting close calls and
adverse events to mutual trust and comfort in reporting them.
GAO used ethnographic techniques to study the Patient Safety Program
from the perspective of direct care clinicians at four VA medical
facilities. This approach recognizes that what people say, do, and
believe reflects a shared culture. The focus included (1) the status of
VA‘s efforts to implement the Program, (2) the extent to which a
culture exists that supports the Program, and (3) practices that
promote patient safety. GAO combined more traditional survey methods
with those from ethnography, including in-depth interviews and
observation.
What GAO Found:
GAO found progress in staff familiarity with and participation in the
VA Patient Safety Program‘s key initiatives, but these achievements
varied substantially in the four facilities we visited. In our study
conducted from November 2002 through August 2004, three-fourths of the
clinicians across the facilities were familiar with the concepts of
teams investigating root causes of unintentional adverse events and
close calls. One-third of the staff had participated in such teams, and
most who participated in these teams found it a positive learning
experience.
The cultural support clinicians expressed for the Program also
differed. At three of four facilities, GAO found a supportive culture,
but at one facility the culture blocked participation for many
clinicians. Clinicians articulated two themes that could stimulate
culture change: leadership actions and open communication. For example,
nurses need the confidence to disagree with physicians when they find
an unsafe situation. Although VA has conducted a cultural survey, it
has not set goals or explicitly measured, for example, staff
familiarity and mutual trust.
Clinicians reported management practices at one facility that had
helped them adopt the Program, including (1) story-telling techniques
such as leaders telling about a case in which reporting an adverse
event resulted in system change, (2) management efforts to coach staff,
and (3) reward systems.
The Patient Safety Program Process in the figure shows how ideally (1)
clinicians have cultural support for reporting adverse events and close
calls, (2) teams investigate root causes, (3) systems are changed, (4)
feedback and reward systems encourage reporting, and (5) patients are
safer.
The Patient Safety Program Process:
[See PDF For image]
[End of figure]
What GAO Recommends:
To better assess the adequacy of clinicians‘ familiarity with,
participation in, and cultural support for the Program, VA should (1)
set goals, (2) develop tools for measuring goals by facility, and (3)
develop interventions when goals have not been met. VA concurred with
our recommendations and will develop an action plan.
www.gao.gov/cgi-bin/getrpt?GAO-05-83.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Nancy Kingsbury at (202)
512-2700 or kingsburyn@gao.gov.
[End of section]
Contents:
Letter:
Chapter 1: VA's Patient Safety Program:
Scope and Methodology:
Background:
Chapter 2: Progress in Clinicians' Familiarity with and Participation
in the Program:
Facilities Shared Safety Hazards but Not Program Familiarity and
Participation:
Summary:
Chapter 3: Measuring Cultural Support for the Program:
Varying Cultural Support:
Building a Supportive Culture:
Improving Assessment of, Familiarity with, Participation in, and
Cultural Support for the Program:
Summary:
Chapter 4: Promoting Patient Safety:
Using Storytelling to Promote Culture Change:
Deliberate Teaching, Coaching, and Role Modeling:
Rewarding Close Call Reporting:
Summary:
Chapter 5: Conclusions and Recommendations:
Measuring Clinicians' Familiarity with and Cultural Support for the
Program:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Content Analysis, Statistical Tests, and Intercoder
Reliability:
Content Analysis:
Ethnography:
Data Collection:
Data Analysis:
Significance Testing:
Intercoder Reliability:
Appendix II: A Timeline of the Implementation of VA's Patient Safety
Program:
Appendix III: Semistructured Interview Questionnaire:
Appendix IV: Comments from the Department of Veterans Affairs:
Appendix V: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Staff Acknowledgments:
Glossary:
Tables:
Table 1: Familiarity with and Participation in the Patient Safety
Program's Initiatives at Four VA Facilities, 2003:
Table 2: Number of Root Cause Analyses at Four VA Facilities, Fiscal
Years 2000-2003:
Table 3: Content Analysis: Achieving a Supportive Culture through
Aspects of the Work Environment:
Table 4: Nonparametric Multiple Comparison Results:
Table 5: Intercoder Reliability Assessment Results:
Figures:
Figure 1: A VA Patient Safety Poster and Its Story:
Figure 2: Model of the Patient Safety Program at Four VA Medical
Facilities:
Figure 3: Types of Adverse Event and Close Call Reporting at Four VA
Facilities, June 2002:
Figure 4: Familiarity with and Participation in the Program by
Facility:
Figure 5: Familiarity with VA's Program Compared with Trust and Comfort
in Reporting at Four Facilities:
Figure 6: Barriers to Staff Reporting Close Calls:
Abbreviations:
JCAHO: Joint Commission on Accreditation of Healthcare Organizations:
NASA: National Aeronautics and Space Administration:
NCPS: National Center for Patient Safety:
PSRS: Patient Safety Reporting System:
RAP: rapid assessment process:
RCA: root cause analysis:
VA: Department of Veterans Affairs:
United States Government Accountability Office:
Washington, DC 20548:
December 15, 2004:
The Honorable Anthony J. Principi:
Secretary of Veterans Affairs:
Dear Mr. Secretary:
This report on the Department of Veterans Affairs Patient Safety
Program examines the Program's status, the creation and implementation
of a culture that supports close call and adverse event reporting, and
practices that medical facility leaders have used to promote patient
safety. In our study, we used ethnography, a social science method that
includes qualitative and quantitative techniques developed within
cultural anthropology for studying communities and organizations in
natural settings.
We include recommendations aimed at strengthening the Patient Safety
Program by helping to build a more supportive culture and foster
patient safety.
We are sending copies of the report to appropriate congressional
committees and others who are interested. We will also make copies
available on request. If you have any questions about the report,
please call me at (202) 512-2700.
Sincerely yours,
Signed by:
Nancy Kingsbury, Managing Director:
Applied Research and Methods:
[End of section]
Chapter 1: VA's Patient Safety Program:
At the end of the 20th century, a report that the Institute of Medicine
issued estimated that up to 98,000 persons died each year from
accidents in U.S. hospitals. Before the institute published this
figure, the Department of Veterans Affairs (VA) had launched a Patient
Safety Program that included teams investigating the root cause of
medical close calls and adverse events and confidential staff reporting
systems. The Program's ultimate goal is to create a culture in which VA
can discover and correct unsafe systems and processes before they harm
patients.
VA has indicated that it is attempting through the Patient Safety
Program to introduce significant change in staff attitudes, beliefs,
and behavior so that health care professionals will report events as
part of their daily work. In testimony before the Congress in 2000, we
suggested that the Program could be more successful if greater
attention were paid to several leadership strategies the Institute of
Medicine has outlined, such as making patient safety a more prominent
goal and communicating the importance of patient safety to all
staff.[Footnote 1] In addition, we noted that:
"VA could also better ensure success if it prepared a detailed
implementation plan that identifies how and when VA's various patient
safety Programs will be implemented, how they are aligned to support
improved patient safety, and what contribution each Program can be
expected to make toward the goal of improved patient safety."[Footnote
2]
One of the most challenging aspects of VA's Patient Safety Program is
creating an atmosphere in which employees are willing to reveal system
problems and find system solutions to them. Traditionally, hospital
employees have been held responsible for adverse patient outcomes,
whether they stemmed from employees' mistakes or the health care
system. For example, a nurse might be blamed for administering the
wrong medicine, even when the system was at fault, as when two
medicines with similar names--one deadly, the other not--were stored on
the same shelf in similar bottles.
The poster and story in figure 1 show how complicated a day in the life
of a healthcare provider can be. In this instance, a VA nurse
recognized a potentially dangerous flaw in the system that could have
caused unintentional harm to patients. In June 2002, she reported the
close call, because she saw that the environment she worked in
encouraged reporting, and she was then rewarded with a gift
certificate.
Figure 1: A VA Patient Safety Poster and Its Story:
[See PDF for image] -graphic text:
PHARMACY WARNING Danger: Look Alike/Feel Alike:
EVENT:
Oral liquid KCL found in Acetaminophen storage bin.
Both containers are same size. Both containers are the same color. Both
containers have the same blue on white labeling.
ACTION:
Eliminate liquid KCL from stock since KCL is available in powder form.
The Close Call Story behind the Poster:
We visited an intensive care unit and talked to the nurse who reported
a close call of two look-alike drugs that were mixed together in the
same drawer. She said she reached for liquid Tylenol and found
potassium chloride concentrate also in the drawer. She told us that the
two drugs were very different-one could kill you and the other is a
mild analgesic. The two drugs were packaged similarly in containers
with pull-off lids, and since the same drug company made both
medications, the labels were similar. She told us she notified her
supervisor and the pharmacy. Since the medical facility had a reward
system for close calls, she received a gift certificate for the
cafeteria, and later it was determined that this close call was the
"pick of the month" This meant that her unit received a plate of
cookies. She said that she reported the close call not for the reward
but because she is a professional. When one day a poster appeared in
the hall alerting others to the two look-alike drugs, she wondered
whether the other medical facilities were notified. She wondered
whether she had made a difference in safety nationwide; a nurse rarely
has that chance.
Source: VA (poster).
[End of figure]
High-risk industries such as nuclear power and aerospace have found
that reliable safety organizations discover and correct system flaws.
In effective safety cultures, frontline workers trust one another and
report close calls and adverse events without fear of blame.
Healthcare, which traditionally employs a culture of blame, must place
a premium on learning from staff reporting of adverse events and close
calls.[Footnote 3] Experts in patient safety acknowledge that emphasis
on culture is important in preventing medical adverse events and close
calls and promoting patient safety.[Footnote 4]
To describe the culture in VA's medical facilities and to search for a
deeper understanding of patient safety from the viewpoint of VA staff,
we proposed to answer the following questions in the context of four VA
medical facilities:
1. What is the status of the Program's implementation at four medical
facilities?
2. To what extent do the four sites we studied have a culture that
supports the Program? What cultural changes can be stimulated?
3. What practices in the four facilities promoted patient safety?
Scope and Methodology:
To meet our study's challenges, we used several methods from
ethnography, and in certain cases we blended them with survey methods
to provide in-depth knowledge of organizational culture from the
perspective of VA's frontline staff--its physicians, nurses, and others
directly responsible for patient care.[Footnote 5] We intend this study
to complement our earlier reports on organizational culture and
changing organizations.[Footnote 6] We chose ethnography because
several of its techniques and perspectives helped us study aspects of
patient safety that would otherwise have remained overlooked or would
not have been observed, such as informal mores, and to assist GAO in
the development of new evaluation methods.[Footnote 7] These aspects
were ethnography's research traditions of (1) conversational
interviews, enabling interviewers to explore a participant's own view
of and associations with an issue of interest, (2) the researchers'
observations of real processes to further understand the meaning behind
patient safety from the natural environment of staff, and (3) systems
thinking.[Footnote 8]
Our study measures, at the facility level, the extent of familiarity
with, participation in, and cultural support for the Program, and it
complements a cultural survey VA conducted in 2000. VA expects to
resurvey staff in the near future, using its past survey data as a
baseline. VA's original, nonrandom survey contained questions regarding
shame and staff willingness to report adverse events when the safety of
patients was at hazard during their care. The VA survey did not
establish staff familiarity with key concepts of the Program,
participation in VA safety activities, or the facilities' levels of
cultural support for the Program.[Footnote 9]
Conversational Survey Interviews:
We recognized that a tradition of fear of being blamed for adverse
events and close calls might make staff reluctant to talk about their
experience of potential harm to patients. Besides breaking through an
emotional barrier, we wanted to understand the private views of staff
on what facilitates patient safety. To achieve the informal, open, and
honest discussions we needed, we conducted private, nonthreatening,
conversational interviews with randomly selected clinicians and other
staff in a judgmental sample. At each site, we chose one random and one
judgmental (nonrandom) sample of staff to interview in a conversational
manner, using similar semistructured questionnaires (see app. III).
For the first sample, we interviewed a random selection of 10
physicians and 10 nurses at each of the four facilities. While this
provided us with a representative sample of clinicians (physicians and
nurses) from each facility, the sample size was too small to provide a
statistical basis for generalizing our survey results to an entire
facility. To give us a better understanding of the culture and context
of patient safety beyond the clinicians involved in direct patient care
at each facility, we also interviewed more than a hundred other staff
in the four study sites, including medical facility leaders, Patient
Safety Managers, and hospital employees from all levels--maintenance
workers, security officers, nursing assistants, technicians, and
service chiefs. (Appendix I contains more technical detail about our
analysis.)
Reporting adverse events and close calls is a highly sensitive subject
and can successfully be explored with qualitative methods that allow
respondents to talk privately and freely. When staff did not recognize
a key element of the Program, our interviewers explained it to them.
(We were not giving the respondents a test they could fail.) Selecting
clinicians randomly at each of four facilities, and asking some close-
ended questions such as those expecting "yes" or "no" answers, allowed
us to analyze and present some issues as standard survey data. This
combined survey and ethnographic approach afforded us most of the
advantages of standard surveys while establishing an environment in
which the respondents could talk, and did talk, at length about the
cultural context of patient safety in their own facilities.
Clinicians responded to a standard set of questions, many open ended,
such as, To what extent do you perceive there to be trust or distrust
within your unit or team? Among the advantages these questions had were
that they allowed the clinicians to discuss issues spontaneously and
they allowed us to discover what facilitates trust from their point of
view. Thus, if clinicians thought leadership was important, we had an
opportunity to see this from their viewpoint rather than starting from
the premise that leadership would be important to them.
An important part of our approach was content analysis, which we used
to analyze answers to both the standard and open-ended questions.
Content analysis summarizes qualitative information by categorizing it
and then systematically sorting and comparing the categories in order
to develop themes and summarize them. We determined, by intercoder
reliability tests, that our content analysis results were trustworthy
across different raters. (See app. I.)
Observation:
We added another ethnographic technique in order to more completely
understand the culture within each facility. Since responses to surveys
are sometimes difficult to understand out of context, our in-depth
ethnographic observations of the patient care process gave us a more
complete picture of how the elements of the Patient Safety Program
interacted. They also gave us a better understanding of VA's medical
facility systems. We observed staff in their daily work activities at
each medical facility, which helped us understand patient safety in
context. For example, we attended staff meetings where the Program was
discussed and we attended RCA meetings, and we followed a nurse on her
rounds. We took detailed field notes from our observations, and we
analyzed and summarized our notes.
We reviewed files to examine data on adverse events, close calls, and
RCA reports. We read files from administrative boards, reward programs,
and patient safety committee minutes. And we interviewed high-level VA
officials.
Systems Thinking:
Finally, our ethnographic research approach was systemic. This was to
help us appreciate interactions between the elements of the Program and
the facilities' existing culture. Ethnography has traditionally been
used to provide rich, holistic accounts of the way of life of a people
or a community; in recent decades, it has also been used successfully
to study groups in modern societies. A systems approach casts a wide
net over the subject. In this case, we chose to study the Patient
Safety Program in relation to other aspects of culture in VA's medical
facilities that might affect its adoption, such as the extent to which
staff have mutual trust.
We also developed a model, or flow chart, to guide our study of the
Program and the culture of the facilities. The model, in figure 2,
helped us conceptualize the important safety activities within the
Program and analyze and present our results. We looked not only at the
Program's key elements, in the darkly shaded boxes in figure 2, but
also at what surrounds them--the context of the medical facilities'
culture--and whether the culture supports the adoption of the Program.
Our model illustrates that our primary focus was measuring clinicians'
supportive culture for reporting close calls and adverse events and
their familiarity with and participation in reporting programs and
RCAs. The model also depicts the interaction between clinicians'
receiving feedback and being rewarded and their desire to continue
reporting close calls and adverse events. It also allows us to describe
how clinicians' reporting close calls and adverse events, and the
subsequent investigation of the root causes of them, developed into
system changes that in turn resulted in patients being safer.
Figure 2: Model of the Patient Safety Program at Four VA Medical
Facilities:
[See PDF for image]
[End of figure]
We conducted the study at three medical facilities that VA had
recommended as being well managed. We selected a fourth facility for
geographic balance. Thus, the four facilities were in different regions
of the country. Using rapid assessment techniques, we conducted
fieldwork for approximately a week at each of two facilities, for 3
weeks at a third, and for 25 days at the fourth.[Footnote 10] We did
our work from November 2002 to August 2004 in accordance with generally
accepted government auditing standards.
Background:
The Patient Safety Goal:
In 1998, in an influential editorial in the Journal of the American
Medical Association, George Lundberg, the journal's editor, along with
Kenneth Kizer, then VA's Under Secretary for Health, and other patient
safety advocates and theorists, challenged the medical profession:
"to make health care safe we need to redesign our systems to make error
difficult to commit and create a culture in which the existence of risk
is acknowledged and injury prevention is recognized as everyone's
responsibility. A new understanding of accountability that moves beyond
blaming individuals when they make mistakes must be established if
progress is to be made."[Footnote 11]
This vision of making patients safe through "redesign . . . to make
errors difficult to commit" led to VA's National Center for Patient
Safety (NCPS), established to improve patient safety throughout the
largest health care system in the United States.[Footnote 12] To
transform the existing culture of patient care in VA's medical
facilities, VA's leaders aimed to persuade clinicians and other staff
in health care settings to adopt a new practice of reporting, free of
fear and with mutual trust, identifying vulnerabilities, and taking
necessary actions to mitigate risks.
The Under Secretary had recognized risk to patients during care and
that a focus on VA's existing culture could improve patient safety.
Related research shows that if complex decision making organizations
are to change, they must modify their organizational culture.[Footnote
13] Traditionally, clinicians involved in an adverse event could be
blamed or sued, but the roots of unintentional errors are now
understood as originating often in the institutions and structures of
medicine rather than in clinicians' incompetence or
negligence.[Footnote 14]
Several contextual factors influence how the Patient Safety Program is
experienced at the medical facilities we visited and show the
increasingly complex world of patient care. Our study's limitations
meant that we could not study these factors, but health care facilities
in general, as well as VA's, are experiencing difficulty in hiring and
retaining nurses, as well as potential staffing shortages. Patients
admitted to VA medical facilities have more multiple medical problems
that require more extensive care than in the past. VA's eligibility
reform allowed veterans without service-connected conditions to seek VA
services, leading to a 70 percent increase in the number of enrolled
veterans between 1999 and 2002.
The Patient Safety Process:
VA has provided funding of $37.4 million to NCPS for its Patient Safety
Program operations and related grants and contracts for fiscal years
1999-2004.[Footnote 15] In fiscal year 1999, NCPS defined three major
initiatives: (1) a more focused system for mandatory close call and
adverse event reporting, including a renewed focus on close calls; (2)
reviews of close calls and adverse events, including RCAs, using
interdisciplinary teams at each facility to discover system flaws and
recommends redesign to prevent harm to patients; and (3) staff feedback
on system changes and communication about improvements to patient
safety.[Footnote 16]
Close Call and Adverse event Reporting:
Starting with the NCPS program in 1999, reporting of close calls
increased dramatically as their value for patient safety improvement
was widely disseminated and increasingly recognized by VA personnel. A
close call is an event or situation that could have resulted in harm to
a patient but did not, either by chance or by timely intervention. VA
encourages reporting close calls and adverse events, since redesigning
system flaws depends on staff revealing them.[Footnote 17] VA's Patient
Safety Managers told us that only adverse events and not close calls
were traditionally required to be reported to supervisors and then up
the chain of command.
Under the Program, staff also have optional routes for reporting--
through Patient Safety Managers or a confidential system outside their
facilities. Staff can now report close calls and adverse events
directly to the facilities' Patient Safety Managers. They, in turn,
evaluate the reports, based on criteria for deciding which adverse
events or close calls should be investigated further. NCPS also has a
confidential reporting option--the Patient Safety Reporting System
(PSRS)--through a contract with the National Aeronautics and Space
Administration (NASA). NASA has 27 years of experience with a similar
program, the Federal Aviation Administration's Aviation Safety
Reporting System. Under the contract with VA, NASA removes all
identifying information and sends selected items of special interest to
the NCPS. NASA also publishes a newsletter based on reports that have
had their identifying information removed.
Root Cause Analysis Teams:
Working on interdisciplinary teams of usually five to seven
participants, staff focus on either one or a group of similar close
calls or adverse events to investigate their causes. Then they search
for system flaws and redesign patient care so that mistakes are harder
to make. Under the Program, NCPS envisioned that these teams would be a
key step to improving patient safety through system change and one of
its primary mechanisms of introducing clinicians to the
Program.[Footnote 18] In 1999, NCPS began RCA implementation.[Footnote
19] In this on-the-job training, Patient Safety Managers guide local
interdisciplinary teams in studying reports of close calls or adverse
events to identify and redesign system weaknesses that threaten
patients' safety. Teams are allowed 45 days to learn as much as
possible from a close call or adverse event or a group of similar close
calls or adverse events such as falls, missing persons, medication
errors, and suicides called aggregated reviews. Within the given time
period, teams are to develop action plans for system improvement.
Personal experience on interdisciplinary RCA teams investigating close
calls and adverse events at their home facilities is the clinicians'
key training experience. VA expected that the RCA experience would
persuade staff that VA was changing its culture by encouraging a
different approach to reporting.
Feedback Mechanisms:
Staff need to receive proof that the Program is working by receiving
timely feedback on their reporting. A feedback loop fosters and
perpetuates close call and adverse event reporting.[Footnote 20]
Without it, staff may feel the effort is not worth their time. NCPS
built in feedback loops at several levels of the system. For example,
individuals who report a close call or adverse event are supposed to
get feedback from the RCA team on actions recommended as a result of
their reports. Also, NCPS issues an online bimonthly newsletter that
reports safety changes.
In chapter 2, we measure clinicians' familiarity and participation in
the Program at the four facilities we visited. Chapter 3 is an
examination of whether the culture at the four facilities supports the
Patient Safety Program and chapter 4 provides examples of management
practices that promote patient safety. We asked VA to comment on our
report; VA's comments are in appendix IV. Our response to their
comments is in the conclusions located in chapter 5. VA also provided
some additional comments to emphasize that it believes that it has
taken steps to address the issue of mutual trust. VA describes those
steps in the report on page 67.
[End of section]
Chapter 2: Progress in Clinicians' Familiarity with and Participation
in the Program:
In general, we found progress in clinicians' understanding and
participation in the Patient Safety Program. Three facilities had
medium or higher familiarity with and participation in the Program's
core elements, and one had lower. At that facility, the staff were not
following VA's policy of reporting close calls and were not being
educated in the benefits of doing so. Examining the data across our
total random sample, we found that some clinicians were familiar with
several core concepts of the Program and were unfamiliar with others--
a picture NCPS officials said did not surprise them.
About three-quarters of clinicians were familiar with the concept of
RCAs (newly introduced in 2000) and the concept of the close call.
About half the clinicians recognized the new confidential reporting
process--another equally important program. One-third had participated
in an RCA or knew someone who had. NCPS staff told us that
participation in RCAs is crucial to culture change at VA, and
clinicians who were on RCA teams indicated that they experienced the
beginning of a culture shift.[Footnote 21] Of the staff who had
participated in RCAs, many indicated that it was a positive learning
experience, but facilities varied in ensuring clinicians' broad
participation.
Facilities Shared Safety Hazards but Not Program Familiarity and
Participation:
VA has made progress in familiarizing and involving clinicians with the
Program's key concepts. But while the facilities we studied shared
basic safety problems, three had made more progress than the fourth.
First, all four experienced similar hazards to patient safety. Second,
we report clinicians' familiarity with and participation in the Program
in two ways--grouped first by facility and then across the four sites.
Facilities' Share Common Safety Reporting Pattern:
The four facilities shared an overall pattern in the types of adverse
events they reported, reflecting their common safety challenge. To
establish the Program's context, we asked at the four facilities to
review documents related to close calls and adverse events reported
over a one-month period (June 2002). All the facilities reported falls
for this period, while two facilities or more recorded patients'
violence toward staff, patients' suicides and suicide attempts, missing
patients, and medication errors (see fig. 3).[Footnote 22] Although our
data reflect a limited time period, the highly overlapping types of
reporting at the facilities parallel those found in the wider VA
patient care system, as documented in an earlier review by the VA
Medical Inspector.[Footnote 23]
Figure 3: Types of Adverse Event and Close Call Reporting at Four VA
Facilities, June 2002:
[See PDF for image]
Note: Excludes reports in pharmacies, laboratories, and other areas of
VA facilities that had separate reporting systems. Facilities with
suicides not reported for June 2002 may have had suicides reported in
other months.
[End of figure]
Facilities' Differences in Participation and Familiarity with the
Program:
Staff at one facility had less familiarity with and participation in
the Program than staff at the three others (see fig. 4).[Footnote 24]
In the interviews with the random sample, we found Facility D had lower
familiarity with the Program's concepts than the other facilities and
lower participation in RCAs; this pattern was buttressed by additional
interviews at Facility D. For example, the quality manager who
supervised Patient Safety Managers at that facility did not realize
that close call reporting was mandated, and the education officer who
trained staff in patient safety told us that staff were generally not
acquainted with the concept of reporting close calls. Because knowing
that an initiative exists is often the first step to participation, the
lower familiarity with the Program at Facility D in the fifth year of
implementation was a likely impediment to the adoption of the Program
there.
Figure 4: Familiarity with and Participation in the Program by
Facility:
[See PDF for image]
Note: A summary code we created for each facility reflected a composite
score for answers to five questions about familiarity with the key
elements of and participation in RCAs: Do you know what a close call
is? Do you know what the Patient Safety Reporting System is? Do you
know what an RCA is? Have you participated in an RCA? Do you know
anyone who has participated? Coders analyzed all answers for each
individual random sample respondent with regard to expressions of
mutual trust and comfort in reporting. Then they created a summary
value rating of low, medium, or high for each individual. This summary
rating was then tested through rater reliability, and the scores were
determined acceptable. Individual summary ratings were averaged for
each facility. In each key elements question, we let "yes" equal 2 and
"no" equal 0, ensuring that an individual who knew each of the five
elements would achieve a composite score of 10. Finally, we averaged
composite scores to get an average score for each facility. Rather than
display these numbers, we used a scale of high, medium, and low for 10,
5, and 0 and placed the answers accordingly. (Appendix I describes our
methodology; appendix III reprints our questionnaire.)
[End of figure]
Differences in Facilities' Adhering to Close Call Reporting Policy:
The four medical facilities we studied also varied in their adherence
to close call reporting policies under the Program. We found three out
of four facilities followed the policy of reporting close calls. One
facility, in particular, showed a marked increase in the number of
close calls in a short period of time; close call reports were rare in
the 6 months before but numbered 240 in the 6 months after its leaders
told staff patient safety was an organizational priority and introduced
a simple reward system for close call reporting. However, one facility
we visited was not reporting close calls in the Program's fifth year.
Familiarity with and Participation in the Program across Four
Facilities:
We looked at interview responses with randomly selected clinicians
across all four facilities. We found that three-quarters of the
clinicians knew the meaning of close call--that is, when a potential
incident is discovered before any harm has come to a patient--but only
half were aware of the option of reporting close calls and adverse
events confidentially. (See table 1.) Close calls are presumed to occur
more often than adverse events, and reporting them in addition to
adverse events is central to the Program's goal of discovering and
correcting system flaws. Staff who do not recognize the close call
concept cannot bring to light system flaws that could harm patients.
Further, because changing from traditional blaming behavior to
reporting without fear can take time, staff familiarity with the
confidential reporting option is important. However, only half the
clinicians surveyed at the four facilities knew that they could report
adverse events or close calls confidentially under the NASA reporting
contract.
Table 1: Familiarity with and Participation in the Patient Safety
Program's Initiatives at Four VA Facilities, 2003:
Program: Root cause analysis;
Percentage of staff: 78%;
Indicator: Familiar with the concept.
Program: Root cause analysis;
Percentage of staff: 35%;
Indicator: Had participated.
Program: Root cause analysis;
Percentage of staff: 43%;
Indicator: Knew someone who had participated.
Program: Close call;
Percentage of staff: 75%;
Indicator: Familiar with the concept.
Program: Confidential report to NASA;
Percentage of staff: 51%;
Indicator: Familiar with the program.
Source: GAO analysis.
Note: Data, rounded to the nearest whole number, are from our
interviews with 81 randomly selected VA physicians and nurses. If staff
initially did not know of a concept, we explained it to them. If they
then recognized it, we accepted their answer as "yes." Therefore, when
we state that they are familiar with it, this means they either knew
the definition or recognized the term after an explanation.
[End of table]
Culture Shift through Root Cause Analysis:
Clinicians who had participated in interdisciplinary RCA teams found
that their participation enabled them to understand the benefits of
using a systems approach rather than blaming individuals for
unintentional adverse events and close calls. To understand the RCA
process from close call reporting to RCA team analysis, we provide an
example from fieldwork that shows how two misidentifications in a
surgery ward led to a reexamination of the preoperative process in an
RCA. (See "Developing Patient Safety from Examining Close Calls and an
RCA.")
While examining how many RCAs were conducted from 2000 to 2003 at the
four facilities, we found that the most active facility we studied had
performed twice as many RCAs as the least active. The RCAs have the
potential to promote a cultural shift from blaming staff for
unintentional close calls and adverse events to a rational search for
the root causes, but clinicians at the four facilities had inconsistent
opportunities to participate in the Program.
Illustrating the Steps from Close Calls to RCAs:
"Developing Patient Safety from Examining Close Calls and an RCA"
illustrates an RCA team's initial steps by following a series of events
involving two close calls of mistaken identity in surgery at one
facility.
Developing Patient Safety from Examining Close Calls and an RCA:
The Patient Safety Manager had an unusual visit from the Chief Surgeon.
He had come to report two recent instances of patients being mistakenly
scheduled for surgery. The identity mix-ups had been discovered before
the patients were harmed--a situation the surgeon recognized as fitting
the Program's mandate to report close calls in order to identify
hazards in the system. After each close call, he had filled out a form
and made a report to NCPS, which had called him back within 24 hours to
ask for more information and to offer some reengineering suggestions.
At the next weekly surgery preoperation meeting, the Chief Surgeon and
his staff discussed their schedule and details of coming surgeries,
using a matrix timetable projected for all to see. Then he discussed
the two close calls. In both cases, the correct patient had come to the
surgery preparation room, but the staff had been expecting someone
else. In one case, the scheduling staff had confused two similar names.
In the other case, the scheduling staff had, as usual, used the last
four digits of the Social Security number to help identify the patient
but had had two patients with the same last four digits. In the
meeting's discussion, the staff tried to understand how such mistakes
could happen.
The Patient Safety Manager convened an expedited RCA team of three
other VA staff to get at the root cause of such identification
problems. She opened the meeting by saying, "If we don't learn from
this [close call], we're all fools." She announced that the RCA would
be limited to two or three meetings rather than several weeks. After
introductions, the staff members explained their role in scheduling and
what happened in such cases. As they spoke, the staff tried to outline
the scheduling process: what forms were completed, whether they were
electronic or paper, how they moved from person to person, and who
touched the forms.
Several problems emerged. (1) Some VA patients might not always know
their identity or surgical site because of illness or senility or both.
Also, patients with multiple problems cannot always relay them to
staff, because they may focus on one problem while the appointment
scheduled is for another problem. (2) Two key VA staff may be absent at
the same time and a substitute may make the error. (3) In one case, two
patients' names differed only by m and n. (4) A scheduler noted that
scheduling is filled with interruptions and opportunities for
confusion. For example, it is not uncommon that scheduled patients have
overlapping numerals for the last four digits of their Social Security
numbers.
The RCA team's next meeting was scheduled. In future meetings, the RCA
team would consider various ways of preventing or minimizing similar
events.
[End of text box]
Clinicians' Belief in RCAs as a Positive Learning Experience:
Staff who had participated in RCAs told us that their experience was a
valuable and convincing introduction to the Patient Safety Program. In
lieu of giving clinicians formal training in the central concepts of
the Program, NCPS expected to change the culture of patient care one
clinician at a time by their individual experience in RCAs. NCPS
intended that experience on multidisciplinary RCA teams investigating
the underlying causes of reported close calls and adverse events at
their home facilities would be clinicians' key educational experience
and that it would persuade them that VA was taking a different approach
to reporting. All facilities are expected to perform RCAs, in which
local interdisciplinary teams study reports of close calls and adverse
events in order to identify and redesign systems that threaten
patients' safety.
Staff also reported that RCA investigations created a learning
environment and were an excellent way to introduce staff to redesign
systems to prevent harm to patients. Two doctors at one facility, for
example, told us that the RCAs they participated in were a genuine "no
blame learning experience" that they felt good about or found valuable.
Two nurses at another facility reported being amazed at the change from
a blaming culture to an inquiring culture as they experienced the RCA
process. However, staff also told us that the RCA process took too much
time or took time away from patient care. At another facility, where
trust was low and only 5 of 20 clinicians had a positive view of
reporting, each of those 5 clinicians had a positive experience with
RCAs under the new Program. "How Participating in RCAs Affects
Clinicians' Work" presents some clinicians' own stories of their
participation in RCAs.
How Participating in RCAs Affects Clinicians' Work:
Physician 1: I participated in an RCA through my work in the blood
bank. It taught me to look at errors systematically and not rush to
blame individuals. But if an employee were eventually found
responsible, then the Lab would hold that person accountable. [This
example reflects the decision leaders must make between personal
accountability and systemic change.]
Physician 2: RCAs are a good thing. It's fixing a potential disaster
before it can coalesce and become a disaster.
Nurse 1: I think RCAs are a good thing, because usually the problems
are system problems. I think if you fix the system, you fix the
problem. It seems to be that way in surgery. You try and concentrate on
the things you can fix.
Nurse 2: They used to have a process in psychiatry called "post
mortem." That process often led to the conclusion that a suicide could
not have been prevented. By contrast, in the new RCA process, we look
at how the RCA can promote system changes.
Nurse 3: RCA does a good job of identifying not only the actual adverse
event but also the contributing factors. This is very helpful because
it allows us to better understand what to do about an adverse event.
Nurse 4: RCA is a good system. It's a good way to share information and
avoid recurring error.
Nurse 5: My general impression is that RCAs are great. They're
especially important when teams look for results and action items.
[End of text box]
Variation in Facilities' RCA Activity:
Over the 4 years of the RCA implementation, the most active facility we
studied (Facility A) had performed twice as many RCAs as the least
active facility (Facility D). (See table 2.) The number of RCAs,
similar to the number of close calls and adverse events, does not
reflect the actual numbers of adverse events or close calls that
occurred or how safe the facility is; rather, it reflects whether
organizational learning is taking place, through increasing
participation in a core Program activity. Similarly, NCPS staff
recently reported to a facility leaders' training session that networks
of their facilities varied fourfold in fiscal year 2002 with respect to
number of RCAs conducted. Facility D's director told us that NCPS had
recently identified his facility as having too few RCA reviews.
Table 2: Number of Root Cause Analyses at Four VA Facilities, Fiscal
Years 2000-2003:
Fiscal year: 2000;
Facility A: 10;
Facility B: 9;
Facility C: 8;
Facility D: 1.
Fiscal year: 2001;
Facility A: 20;
Facility B: 14;
Facility C: 11;
Facility D: 9.
Fiscal year: 2002;
Facility A: 13;
Facility B: 9;
Facility C: 8;
Facility D: 5.
Fiscal year: 2003;
Facility A: 11;
Facility B: 6;
Facility C: 7;
Facility D: 8.
Total;
Facility A: 54;
Facility B: 38;
Facility C: 34;
Facility D: 23.
Source: GAO analysis.
Note: Includes only individual RCAs; excludes aggregate reviews. In
2002, VA began a program of aggregate RCAs, in which the most commonly
reported events, such as patient falls, were grouped and analyzed
quarterly. Thus, in 2003 we see a reduction in individual RCAs across
these facilities.
[End of table]
Inconsistent Opportunities to Participate in RCAs:
One facility was more successful than the three others at providing
busy physicians with the opportunity to participate in RCA teams by
adopting a mandatory rotation system.
RCAs have been required under the Program since 2000. About three-
fourths of the respondents were familiar with the RCA concept. Seventy-
five percent staff familiarity represents substantial learning, given
when the concept was introduced. However, only about a third had
participated in an RCA or knew someone who had. At one facility, we
found broad participation by physicians because management required it.
NCPS envisions RCA experience as central to changing to a culture of
safety, but many VA clinicians (approximately 65 percent) at the
facilities we studied had yet to participate in the nonblaming process
that NCPS's director told us he viewed as the most effective experience
for culture change: "We don't want professional root cause analysis
people doing this stuff. Then you don't change the culture."
We found a wide spectrum of methods being used to recruit physicians
into RCA teams. One facility had broad physician participation in RCAs
as its policy, and at another facility one unit had a rotational plan
that encouraged its own clinicians to participate, in contrast to the
whole facility. Administrators at three of the four had no policy
across the facility to ensure physician participation on the teams. At
two facilities, Patient Safety Managers told us it was difficult to get
physicians to participate because of their busy schedules.
Understandably, most of the clinicians we surveyed had not served on
RCA teams.
Summary:
We found progress but also variation in the range of clinicians'
familiarity with and participation in key elements of the Program.
Looking facility by facility, we found one of the four facilities had
lower familiarity and participation in the Program. Examining the
clinicians across the random sample, we also found that about three-
fourths were familiar with close call reporting but only half were
familiar with a confidential reporting system. Focusing on RCAs, we
found that about three-quarters of the sample knew the concept--that
is, staff teams investigate the causes for accidents--while one-third
had participated. Most of those who had participated thought that RCAs
were promoting a culture shift by investigating adverse events and
close calls in a no-blame atmosphere and redesigning systems so that
future problems could be prevented.
[End of section]
Chapter 3: Measuring Cultural Support for the Program:
Cultural support for VA's Patient Safety Program varied at the four
facilities we studied. While clinicians we surveyed at three facilities
had a more supportive cultural foundation for the Program,
significantly lower levels of mutual trust and comfort in reporting
limited the adoption of core Program activities at the fourth facility.
Further, our analysis indicated that low trust and fear of punishment
that characterize an unsupportive culture limit the adoption of the
Program and constitute a feature held by clinicians that does not
necessarily improve when they become familiar with the key concepts in
the Program.[Footnote 25]
The clinicians identified barriers to their participation in the
Program. However, they fundamentally agreed on workplace conditions
that can build the supportive culture and foster patients' safety.
Their most frequently articulated themes for building supportive
culture were (1) effective leadership; (2) good two-way communication,
including feedback on reports of adverse events and close calls; (3)
their professional values; and (4) workflow.[Footnote 26]
Varying Cultural Support:
Clinicians at three of the four facilities had medium or higher
cultural support for the Program. One facility had lower support, and
many clinicians indicated that they would not report adverse events
because they feared punishment.[Footnote 27] This suggests that the
Program will not succeed unless cultural support is bolstered. We
explored the cultural support from these four groups in two ways: (1)
by graphically comparing the groups' levels of mutual trust and comfort
in reporting close call and adverse events with their levels of
familiarity with the Program and (2) by graphically demonstrating the
barriers clinicians see as blocking their close call and adverse event
reporting, in conjunction with some elements of basic familiarity with
and cultural support for the Program.
Clinicians' Trust and Comfort in Reporting Varies by Facility:
In figure 5, we compare our findings on clinicians' mutual trust and
their comfort in reporting close calls and adverse events at the four
facilities. The levels of these components of a supportive culture
appeared to vary among the clinician groups.[Footnote 28] For example,
staff at Facility A had medium familiarity with the Program but had the
lowest levels of comfort in reporting adverse events and close calls
and mutual trust among the four facilities. Knowledge from specific
safety training or RCA participation was not sufficient for them to
readily change to safety practices under the Program if levels of
comfort in reporting and mutual trust were not high enough. Figure 5
contrasts information on the supportive culture (mutual trust and
comfort in reporting) with a measure of staff familiarity with the
Program from figure 4.
Figure 5: Familiarity with VA's Program Compared with Trust and Comfort
in Reporting at Four Facilities:
[See PDF for image]
Note: We reviewed all coded expressions of mutual trust and comfort in
reporting for each interview in the random sample, assessing the
preponderance of expressions and creating a summary high, medium, or
low value for each individual. Intercoder reliability testing found
coding consistency acceptable. We averaged these scores for each
facility. Finally, we created a summary code for each facility,
reflecting a composite score, using five questions about familiarity
with the key elements and participation in RCAs. Coders analyzed all
answers for each individual random sample respondent with regard to
expressions of mutual trust and comfort in reporting and then created a
summary rating of low, medium, or high values for each individual. This
summary rating was then tested through rater reliability, and the
scores were determined acceptable. For each facility, the individual
summary ratings were averaged.
We assigned numeric values, as customary in quantifying verbal answers.
For display and comparison purposes, we decided to let the maximum
individual knowledge, trust, and comfort levels be 10. Thus, in each
key elements question, we let "yes" equal 2 and "no" equal 0, ensuring
that an individual who knew all of the five elements would achieve a
composite score of 10. Finally, we averaged composite scores to get an
average score for each facility. In the trust and comfort summary
judgments, we let "high" equal 10, "middle" equal 5, and "low" equal 0.
Rather than display these numbers, we used a scale of high, medium, and
low for 10, 5, and 0 and placed their answers accordingly. (See app. I
for more on our methodology.)
[End of figure]
Many staff at Facility A were afraid of being punished, and they
mistrusted management and other work units. One staff member explained
why staff would not report adverse events: "We have a culture of back-
stabbing here. They are always covering themselves." Many other staff
members echoed this characterization of the atmosphere, linking the
lack of cultural support to their decision not to perform the most
basic of the Program's activities. Staff at that facility needed a
boost in supportive culture to fully implement the Program. In
contrast, Facility D, with the least familiarity with the Program, had
trust and comfort levels almost as high as any of the others,
indicating that if the Program were to be pursued with greater vigor
there, cultural support would not be a barrier to reporting close calls
and adverse events.
Barriers to Reporting:
In interviewing clinicians, we found that barriers remain to reporting
adverse events and close calls. Even for staff familiar with the
concepts, reporting required overcoming numerous remaining obstacles.
These staff indicated that reporting formally would be a time-consuming
diversion from patient care or, worse, "an invitation to a witch hunt."
In figure 6, we display the cumulative effect of the barriers to
reporting close calls that staff told us about, in conjunction with
familiarity with and cultural support for the Program.
Figure 9: Barriers to Staff Reporting Close Calls:
[See PDF for image]
Note: We asked VA staff "Do you know what a close call is?" If they
answered "No," we explained it to them; if they recognized the concept,
we accepted their answer as "Yes."
[End of figure]
Clinicians told us about barriers to their participation in reporting,
including (1) limited perceived value, (2) not knowing how to report,
(3) not having enough time to report, (4) fearing traditional blame or
punishment, (5) lacking trust that coworkers would not shame them, and
(6) lacking knowledge of the confidential reporting option. Staff at
all four sites reported such barriers in reporting both close calls and
adverse events. We present some of their views in "Clinicians' Barriers
to Reporting Close Calls and Adverse Events."
Clinicians' Barriers to Reporting Close Calls and Adverse Events:
Nurse 1: Some clinicians feel comfortable reporting adverse events and
close calls. I agree with the concept. It depends on the person. Some
would feel it would be used against them. I've seen nonreporting,
because, before, they got written comments such as "This is not a near
miss." "This is not a close call." We get shut down instead of worked
with. [By "shut down," she meant that management told her it was not a
close call and not to report it.] It happened to me. Management
generally discourages and does not empower staff to feel comfortable
reporting patient safety conditions. Instead, I reported and it was
used against me.
Physician 1: I can't remember if I've written a close call. That does
not happen here--only very, very rarely. Maybe I wrote one early on in
my career, but I'm not sure.
Physician 2: I thought I had a close call once and showed it to the
chief of staff and he told me that it was not a close call. I'm unclear
what the definition of a close call is.
Physician 3: I know what a close call is in other settings, but not in
the hospital setting. [Interviewer explains the definition.] They are
not reporting on close calls in this hospital.
Physician 4: Yes, I know what a close call is. I've not reported a
close call, but if I were to, I would go to a nurse supervisor and tell
her about it orally and have her report it. I would not use incident
reports to report a close call--only actual events.
Physician 5: I have not reported a close call. I'm removed from the
nursing communications.
Physician 6: I'm unsure if it is safe to report close calls without
punishment.
Nurse 2: If I saw a close call, I would go talk to the nurse who did
it. Writing up a close call on someone would be cruel. I would not
write up a close call or adverse event report on someone else. If
something happened to the patient, I would write it up. Writing up
another person would cause conflict. We need to help each other, and
writing each other up is not considered helpful.
[End of text box]
Additional Steps to Stimulate Culture Change:
The themes for work conditions that promote a supportive culture for
patient safety that clinicians articulated most often were (1)
leadership, (2) communication, (3) professional values, and (4)
workflow.[Footnote 29]
Building a Supportive Culture:
A few strong patterns emerged from the clinicians' responses to our
open-ended interview questions about what affects trust and comfort in
reporting close calls and adverse events. First, across the survey, the
clinicians said their leaders' actions were most likely to increase or
decrease comfort and trust. Attributes of communication were the second
most common aspect of their work that they said influenced their
comfort and trust. Third, and somewhat less commonly, clinicians
thought that the values and norms that they had developed in their
professional training and that had been reinforced on the job
influenced their culture, but they also thought that workflow could
support or undercut trust generally. In their view, trust literally
could be made or broken, depending on whether tasks shared between
individuals or between units went smoothly and cooperation was
maintained. Table 3 shows the results of our content analysis, listing
the clinicians' four top themes--leadership, communication,
professional values, and workflow--and how many times we found these
themes in our analysis.
Table 3: Content Analysis: Achieving a Supportive Culture through
Aspects of the Work Environment:
Aspects of work environment: four top themes: Leadership;
Culture element: Comfort in reporting: 22;
Culture element: Mutual trust: 25;
Number of times theme appeared in our analysis: 47.
Aspects of work environment: four top themes: Communication;
Culture element: Comfort in reporting: 13;
Culture element: Mutual trust: 25;
Number of times theme appeared in our analysis: 38.
Aspects of work environment: four top themes: Professional values;
Culture element: Comfort in reporting: 15;
Culture element: Mutual trust: 8;
Number of times theme appeared in our analysis: 23.
Aspects of work environment: four top themes: Workflow;
Culture element: Comfort in reporting: 0;
Culture element: Mutual trust: 12;
Number of times theme appeared in our analysis: 12.
Source: GAO analysis.
[End of table]
When we asked clinicians what affected a culture that supported comfort
in reporting and trust among the different professions, departments,
teams, and shifts they worked with, their most frequent answers were
effective leadership and good two-way communication. Moreover, the
clinicians told us that an unsupportive culture lacks these
characteristics. Clinicians gave us these same answers, whether we
asked about comfort in reporting or mutual trust. Further, we found
that the culture of blame and punishment traditionally learned in
medical training hampers close calls and adverse event reporting but
that mutual trust is developed more by workplace conditions.
Effective Leadership:
Leadership's role is important in fostering a supportive cultural
environment for the Program. Clinicians reported examples of leaders
facilitating comfort in reporting and mutual trust that enabled them to
participate in the Program. But at several facilities we also heard
about distrust of the Program that resulted from leaders' action or
lack of action.
Clinicians told us that some VA leaders had not focused sufficiently on
building the supportive culture that the Program requires. Staff
reported that in order to trust, they needed information and needed to
take part in decisions about their workplace and policies that affect
their work. For example, clinicians told us that they wanted to be part
of management's decisions or, at the very least, to be informed about
management's decisions when a number of changes were being introduced,
such as when medical supplies and software were purchased, clinicians
were assigned temporary rotations, and performance measures were
implemented. Their observations are in line with other studies that
show that leaders' making decisions without consulting frontline
workers can cause serious problems of trust.[Footnote 30]
In "Clinicians' Perspectives on Leaders' Supporting Trust," we
illustrate staff's positive attitudes toward patient safety and how
leadership is instrumental in developing mutual trust and comfort.
Clinicians' Perspectives on Leaders' Supporting Trust:
Nurse 1: I asked my staff what the role of leaders should be so I could
serve staff better. Many answered, "communication" and "knowing what is
happening at the facility is important.“
Physician 1: Leaders often bring up patient safety. They're "taking a
lead in making staff aware of patient safety." At my facility, they
hold staff meetings to review the patient safety goals of the Joint
Commission on Accreditation of Healthcare Organizations (JCAHO). The
chief of staff constantly brings up patient safety in meetings. The
administration takes the lead, not only "talking the talk" but also
"walking the talk.“
Nurse 2: Trust is sustained, in part, because of weekly meetings with
management, where they talk about patient safety.
Physician 2: It's leadership's responsibility to communicate that staff
are accountable for cooperation and coordination of patient care.
[End of text box]
Conversely, respondents said leaders' actions can diminish clinicians'
comfort and trust, as summarized in "Clinicians' Perspectives on
Leaders' Undercutting Trust." Physicians and nurses at different
facilities told us that trust is diminished when staff do not work in
stable teams. Some of the policies that clinicians told us were
obstacles to building a stable team include assigning floating or
nonpermanent supervisory personnel, rotating physicians on and off the
ward, and the monthly rotation of student nurses and doctors.
Clinicians' Perspectives on Leaders' Undercutting Trust:
Physician 1: For 20 years, there was nothing but "blame and train." In
the past, an adverse event or close call was associated with a person
you had to blame, and the "fix" was to train them.
Nurse 1: We have a panel of nurse managers who have discouraged adverse
event reports for medication errors. I vow to encourage reporting
errors without blame. We still have a way to go to be honest about
reporting.
Nurse 2: I know of instances when staff reported adverse events, they
were transferred, so that does not make staff comfortable reporting
them. There is no trust of management.
Nurse 3: Decisions that affect our work are made without talking to
staff or understanding our work situation.
Physician 2: If you don't know what's going on, you invent it.
Physician 3: The most critical change needed at this facility is in the
area of leadership. Leaders are ineffective because they are not good
at communication. We hear about reasons why we are blamed. This causes
a feeling of distrust.
Physician 4: Leadership has little grasp of patient care and, thus,
policy directives have little impact. If we're given a policy to spend
a maximum of 20 minutes per patient, including completing records, I do
what the patient needs. Management can just yell at me.
[End of text box]
Communication:
Staff indicated that communication in the workplace affects trust and
comfort in reporting. Further, they told us that communication is
challenging, since it involves coordinating tasks with and between
leaders and teams and their empowerment, all of which can be
problematic in the medical setting.
Some VA staff told us that unequal power relationships and hierarchical
decision making are often obstacles to patient safety. They also
elaborated on the kinds of communication that support patient safety,
including empowering staff so that they can be heard. Traditionally, a
nurse's status is lower than a physician's in hospitals, and some
nurses could find it difficult to speak up in disagreement with
physicians. For patients to be safe, however, nurses indicated they
wanted to be empowered to openly disagree with physicians and other
staff when they found an unsafe situation. For example, nurses told us
that they had to speak up when they disagreed with the medication or
dosage doctors had ordered. They also said that they had problems when
physicians telephoned nurses and gave directions orally when policy
stated that physicians' orders must be written.
The clinicians spoke to us about empowerment and their involvement or
lack of involvement in decision making. "Clinicians' Perspectives on
How Communication Promotes Trust" gives some examples of what they told
us about communication that they believed supports patient safety.
Clinicians' Perspectives on How Communication Promotes Trust:
Nurse 1: We interact with doctors and nurses in clinic. If something
happens, we share with one another about how we might have done it
differently. This goes on daily.
Nurse 2: The director of the medical facility is a good communicator.
he keeps us informed. He maintains a personal newsletter. Our nurse
manager is well rounded and she listens.
Nurse 3: Peers and coworkers communicating with one another supports
patient safety. For instance, sometimes we have patients who have a
history of violence. This information is reflected in the computer and
comes up when they "chart them in," but sometimes a nurse may still not
know of such a history. Therefore, in the nurses' reports, the history
of violence and the need for caution is passed on. Extra information
about the patients can also help them deescalate confrontations between
patients.
Physician 1: VA's Computerized Order Entry system [a computerized
method for ordering medications] promotes patient safety. Before, it
was hard to read the physicians' handwriting. The Computerized Order
Entry at least eliminated the legibility problem. They do not have
Computerized Order Entry at the university where I also work. VA also
got rid of using Latin abbreviations. Now everything has to be written
out.
Physician 2: Open communication promotes team buy-in and therefore
better customer service.
Physician 3: We have a good department because staff can communicate
their complaints.
Nurse 4: We do an RCA on our own close call or adverse event or those
from other sources, and then we present the results to the staff. I
brought a PowerPoint briefing to our staff meeting about another
hospital's wrong site surgery, so we could know what had happened. If
JCAHO published an adverse event, I put it in our staff notes and have
it discussed at the next staff meeting.
Nurse 5: Management is more involved with the workers. It seems that
they are listening more.
Physician 4: Within the unit, we have good trust. Outside the unit, the
administration has more trust and more communication. We're in the loop
more. In the clinic, we have good trust in nurse-to-doctor and doctor-
to-doctor relationships and with leadership.
Physician 5: I reported a close call recently and feared blame, but it
was not that way at all. It was a learning experience for all who heard
about it. I think it's wonderful that VA has created this open
atmosphere. Formerly, you might be a scapegoat, have backlash, and get
a poorer rating. Today, we don't feel we're going to be punished.
[End of text box]
In "Clinicians' Perspectives on How Faulty Communication Diminishes
Trust," we give clinicians' examples of management's undermining
patient safety by deciding policies without consulting them, as when
nurses were not included in decision making. Such policies sometimes
proved dysfunctional or were ignored.
Clinicians' Perspectives on How Faulty Communication Diminishes Trust:
Nurse 1: I have to double-check changes in supplies in order to
safeguard patients, because Supply often sends ABC instead of XYZ.
Since we're not included in decisions about product changes, we're
forced to continually double-check Supply to keep patients safe.
Nurse 2: We have poor communication between other units and the
radiology unit. They send incontinent or violent Isolation [contagious]
patients without notifying X-ray staff to be wary.
[End of text box]
Facility staff also wanted additional and more timely feedback on what
happens to their reports of close calls, adverse events, and the
results of RCAs. Some Patient Safety Managers often felt too busy to
provide feedback to staff because their jobs included a number of
activities, including facilitating RCAs. At one facility, Patient
Safety Managers routinely reported system changes back to staff who
made the reports, but at the other facilities, they did not have a
routine way of doing this. Many staff at the four facilities told us
that they did not know the recommendations of the RCA teams or the
results of close call or adverse event reports.
NCPS agrees that feedback to staff is necessary but inadequate, and it
plans to focus on the need for feedback at facilities in the near
future. NCPS's Web site publicizes selected results of RCAs and alerts
and system changes that result from reporting. Some of what VA's
leaders and frontline clinicians told us about the need for more
feedback is presented in "Facility Staff Concerns about Limited
Feedback.“
Facility Staff Concerns about Limited Feedback:
Nurse Manager: We do a good job of following up on close call or
adverse event reports in my unit, but not as good a job following up on
the recommendations from RCAs. I was able to implement the action items
right away in my unit after I participated in an RCA on patients'
falls, but other nurse managers didn't hear about the results from the
RCA for 2 or 3 months. The RCA teams develop really good ideas, but we
need follow-through to make sure everyone knows that this is what we're
going to do to change the system. Delays result from organizational
routing and financial constraints. Even when the recommendation is
signed, sometimes there's a delay getting the information down to the
nurse managers.
Physician 1: There should be an annual report of actions taken as a
result of reporting adverse events and close calls. For example, if
three units have developed a different way of labeling medication that
used to be labeled alike, then the rest of the staff should know about
it. [This was a reference to medication that looks alike and confuses
staff. One solution is for the pharmacy to buy the two medications from
different manufacturers so that the labels will be different.] It makes
people feel better to know the information they reported helped make
things better. I'd make sure that the information on improved medical
care gets reported back to the staff.
Administrative Official: The distribution of RCAs has been limited to
staff responsible for the action or system change, but in the future
the results will be distributed more broadly.
Physician 2: I haven't heard any results from the RCAs. A pamphlet on
the results would be a good idea.
Note: "Administrative Official" is a title we used in this report to
keep identity confidential.
[End of text box]
Workflow and Professional Training:
In addition, staff spoke to us frequently about workflow issues--how
safely handing off tasks between shifts and teams required trust but
could cause mistrust when the transition was not smooth or efficient.
VA clinicians clarified for us that mutual trust could be either gained
or lost between workers and units, depending on coordination. And they
drew conclusions about the importance of the quality and nature of
workflow to patient safety. Clinicians also elaborated on aspects of
the values they learned in training that did not facilitate a blame-
free workplace.
They indicated that shifting patient care between groups was an ongoing
challenge to patient safety. For analysis purposes, we found these
issues in continuity of care to be part of the larger problem of
workflow, because they entailed the coordination of tasks and
communication within and across teams. In the views of the clinicians
at the facilities we studied, if staff, teams, or units begin to feel
they cannot adequately communicate their patients' needs for care
because of workflow problems, then trust may be lost, in turn
diminishing patient safety.[Footnote 31] At one facility, where trust
and comfort were lower than at the others, clinicians told us that
workflow failures diminished trust and threatened patient safety. In
"Clinicians on Workflow Problems and Patients' Safety," some physicians
and nurses talk about these problems and how they tried to find
solutions to promote patients' safety.
Clinicians on Workflow Problems and Patients' Safety:
Nurse 1: Some units are less particular about paperwork and records
than others, so when we transfer patients, their information is
sometimes incomplete. Patients don't come back to my unit as quickly
from one unit as from other units, and sometimes their information is
not available.
Physician: Personnel tends to lose things, and this makes it hard to
recruit new staff.
Nurse 2: We often have difficulty getting the supplies we need. For
example, it's especially difficult to obtain blood on the night shift.
Nurse 3: At the change of a shift, I had to discharge one patient and
admit another. Since I couldn't do both at the same time, I chose to
admit but not to discharge. But my relief nurse expressed unhappiness
about the situation, suggesting that I had left my work for another
crew to do. I spoke with the relief nurse, and the problem of mistrust
was resolved when everyone understood the work context better. When
people communicate across shifts this way, they have a better
understanding of and appreciation for one another.
Nurse 4: I go to the ward before my shift starts to make sure the
patients' wounds have been properly dressed. I take dressings to
homebound patients when they weren't sent home with them. I cultivate
motivated individuals from the ward staff, letting them see the
procedures in the Dialysis Unit, and give them responsibility for those
patients when they're back on the ward and reward them. I stock snacks
because feeble elderly patients are sent to Dialysis without breakfast,
and then they're expected to get to breakfast after their dialysis
session and pay for their own meal. I see this situation as inherently
unsafe, so I supply them with free snacks.
[End of text box]
The professional values physicians and nurses learned in their formal
education or on the job can also be an obstacle to the Program, because
these values do not always foster a nonpunitive atmosphere. Some of the
values clinicians have been trained in run counter to the Program's
expectations for open reporting, as we show in "Clinicians'
Professional Values and the Patient Safety Program.“
Clinicians' Professional Values and the Patient Safety Program:
Nurse 1: There is much trust within the nursing profession. We have to
trust each other because of the critical nature of passing patients
from one shift to another.
Nurse 2: The only group I worry about is Clerical. Their work is
frontline and high-stress, but it's entry level, so they may have never
worked in a hospital before. We have to double-check their work because
there's no system in the clinic to verify orders, as there is in the
hospital.
Nurse 3: We trust those we work with. The exception is Housekeeping. We
have to continually call to complain about the cleanliness of the
clinic.
Nurse 4: Nurses have a value system in which we "eat our young," which
undercuts comfort in reporting errors. Traditionally, older nurses
taught younger ones their way of doing things, and the younger ones
were punished when they failed to do things that way. Now, we must
allow nurses to do things a new way without punishment.
Nurse 5: I keep hearing that we're looking to learn and not blame.
Nursing culture is a blaming culture, and [the Patient Safety Program]
is helping to stop this.
Nurse 6: The model in nursing is "a nun with a ruler.“
Physician 1: The culture is changing, but it's taking a while. I'm
impressed with administration here that tries to say, "How can we learn
from this?".
Physician 2: To promote the Program, you have to have a change to a no-
blame culture.
Physician 3: Clinicians have to stop blaming each other and learn from
their mistakes.
[End of text box]
VA clinicians explained that nurses see themselves as the patients'
first and last guard against harm during care. Nurses are expected to
be double-checking physicians' orders, medicines, and dressings and,
for example, preventing falls or suicide attempts. Generally speaking,
in their traditional role, nurses feel personally responsible for
patients' welfare and are designated to fulfill that role. They hold
fast to protocols as safety devices, follow rules, and double-check
work orders. Some spoke favorably of a bygone era when nurses could be
counted on to back up one another, while many others thought this
described their current work environment. In contrast, VA staff told us
that physicians have thought of themselves as taking more original and
independent actions but not as part of a multidisciplinary team. Their
actions, based on traditional professional values, would thus undercut
mutual trust. Physicians told us that patient safety would be improved
if they were better trained to work on teams.
Both nurses and physicians face many obstacles to improving patients'
safety in the increasingly complex and ever changing world of medicine.
VA clinicians take seriously their mission as caretakers of the
nation's veterans, many of whom are older and have multiple chronic
diseases, making these efforts to improve patient safety even more
challenging. Many told us that they feel ethically and morally bound as
frontline caretakers to keep their patients safe by reducing the number
of adverse events and close calls.
Improving Assessment of, Familiarity with, Participation in, and
Cultural Support for the Program:
Although VA conducted a cultural assessment survey in 2000 and plans to
resurvey VA staff in the near future, it has not measured staff
familiarity with, participation in, and cultural support for the
Program. For example, it did not ask about staff knowledge and
understanding of key concepts (close call reporting, RCAs, and VA's
confidential reporting system to NASA) or RCA participation. Although
the 2000 survey did describe some important attitudes about patient
safety, such as shame and punishment related to reporting adverse
events, it did not explicitly measure mutual trust among staff, a
central theme of VA clinicians in describing what affected patient
safety and a supportive culture. Finally, while NCPS staff asked each
facility to administer the survey to a random sample, many facilities
did not follow their directions. The VA survey may serve as a baseline
measure of national local trends, but it could not be used to identify
facility-level improvements or interventions.[Footnote 32]
Summary:
We found that three of the four facilities had a supportive culture
that allowed staff to trust one another and feel comfortable reporting
close calls and adverse events. At the fourth site, clinicians told us
their facility had an atmosphere of fear and blame that did not support
the Program. Content analysis revealed the most frequent themes were
effective leadership, good two-way communication, clinicians'
professional values, and workflow.
[End of section]
Chapter 4: Promoting Patient Safety:
Successful management actions at one facility had resulted in the most
complete adoption of safety practices under the Program at the time of
our study. These actions included (1) storytelling, a well-documented
oral tradition in medicine, to show changes in norms and values; (2)
teaching, coaching, and role modeling for open communication throughout
the hierarchy; and (3) offering rewards for participation in close call
reporting. Clinicians at that facility pointed to these practices,
which facilitated patient safety and their adoption of the Program's
concepts and activities. The three other facilities used some or few of
these practices; nonetheless, clinicians there proposed them as
potentially good ways to improve patient safety. While our work
reflects the clinicians' views at the four facilities we studied, these
findings correspond with other studies of organizations' attempts to
change culture.[Footnote 33]
Using Storytelling to Promote Culture Change:
VA leaders at some facilities we studied showed staff they support the
Program by telling stories. They used the stories to publicly
demonstrate a changed and open atmosphere for learning from adverse
events and close calls, for example. While leaders must still
distinguish episodes that warrant professional accountability, they
must fairly draw the line between system fixes and performance
issues.[Footnote 34] One way to do this is by repeating stories that
demonstrate that VA leaders encourage a culture that supports the
Program and an atmosphere of open reporting and learning from past
close calls and adverse events.
Leaders supported the Program by telling staff stories that
demonstrated a systems change to safeguard patients after a medical
adverse event was reported.[Footnote 35] Storytelling has a long
tradition in medicine as way of teaching newcomers about a group's
social norms.[Footnote 36] One leader shared with us the story he used
to kick off VA's Patient Safety Program. Each time he tells the story,
he confirms the importance of changing VA's culture and helps transform
the organization because staff remember it. Instead of dismissing an
employee who has reported not giving a patient the drug the patient was
supposed to receive, the leader judged the adverse event to be a
systems problem. In discussions with NCPS, the leader recognized that
this story was an opportunity to show his staff that the facility was
following the Program by taking a systems rather than a disciplinary
approach and to highlight that reporting close calls and adverse events
was critical in changing the patient care practice so that such
problems would not recur. "Leaders' Effective Promotion of Patient
Safety in Staff Meetings" contains another example of storytelling to
change communication practice.
Leaders' Effective Promotion of Patient Safety in Staff Meetings:
[The Administrative Official met with a unit leader and about 20
physicians and residents.]
Administrative Official: The Patient Safety Program includes close
calls as reportable incidents. [That is, VA is accepting staff reports
of close calls.] A culture change is needed at VA, brought about by
sharing a vision of what is valuable to us. We also want to show that
leadership endorses the Program.
[He walked the meeting through an aviation example that showed that the
first officer should have challenged the captain, raising parallels
with failure to question authority--or to "cross-check"--at this
facility. He asked the group how they challenged authority effectively.
Finally, he introduced RCAs as a new type of system analysis.
Physicians continued their discussion.]
Physician 1: Cross-checking is more effective if it's not hostile.
Physician 2: There are fewer errors in medical settings where there's a
stable team, but recently VA has been trying to do things more quickly
with fewer staff.
Physician 3: Communication is a problem on my unit, where we have 28
contract nurses.
Physician 4: Could it be bad if one unit reported a lot of close
calls?
Physician 5: [in a leadership position]: VA has 50 years of being
punitive. The Patient Safety Managers will be looking for patterns
across a large number of reports, not seeking to blame individuals.
Physician 6: Why can't the reporting simply be open and the names of
the reporters known?
[Several members of the meeting talked about the fear of punishment
that still existed.]
Physicians 7 and 8: Are the forms discoverable? Can they be subpoenaed?
Can the reports be anonymous?
[In a subsequent interview, leaders told about how the Program was
progressing.]
Leader 1: We must change doing what you're told without questioning
orders. We tell nurses that it's OK to challenge physicians in an
atmosphere of mutual respect. We're establishing it as a facility goal,
keeping it on the front burner and keeping it a priority.
Leader 2: Since leaders began visiting staff meetings to get the word
out on close call reporting, we've noticed a change--a significant
reduction in the fear of reporting close calls. Not all fear is gone,
but the close call program is a success.
Leader 3: Leadership raised safety consciousness with the close call
airplane accident lesson. If it had been handed to us as just another
memo, it might have been thrown away, but when leaders are there in
person to answer questions, then it raises people's awareness of
patient safety.
Physician 1: Leadership here went out and talked about patient safety.
Their support and emphasis and bringing their level of importance to it
made the Program happen.
[End of text box]
Deliberate Teaching, Coaching, and Role Modeling:
Staff at one facility told us that VA's leadership supported the
Program and the patient safety culture by teaching, coaching, and role
modeling patient safety concepts to their staff in more than a hundred
small meetings. VA's leaders had a three-part agenda in their initial
staff meetings. First, they taught a scenario in which two pilots
failed to communicate well enough to avoid a fatal crash. The first
officer did not cross-check and challenge an order from his captain to
descend in a wind shear, resulting in the plane's crashing and killing
37 people. Facility leaders depicted the strong parallels---including
the communication effects of unequal power relationships and
hierarchical decisionmaking discussed earlier---between the pilots'
communication to save the plane and clinicians' communications to save
the patient.
Second, they discussed the importance of communications in medical
care, coaching lower-level staff to speak up when they saw adverse
events and emphasized the importance of two-way communication. Finally,
they introduced a new close call reporting program at the facility and
modeled for staff that they supported this type of reporting in
introducing the new Program and its elements. "Leaders' Effective
Promotion of Patient Safety in Staff Meetings" presents a portion of
one such meeting and also interviews with VA staff when they discussed
how the staff meetings had raised their consciousness about patient
safety.
"Leaders' Effective Promotion" represents more than a hundred small
meetings conducted at one facility that successfully demonstrated that
patient safety was a priority for the organization. When top leaders
attended staff meetings, staff listened to their message. It may be no
coincidence that this facility had the highest rating for comfort in
reporting, according to the findings of our survey. Many staff at this
facility told us that because their top leaders spoke to them about the
Program, they concluded that the Program and its culture change were a
priority for their leaders. Midlevel staff also acknowledged progress
but admitted to some remaining fear.
Participants heard their leaders say that challenging authority--here
called "cross-checking"--was important for patient safety. They were
asked to compare their own communication patterns with the aviation
crew's communication in a similarly high-risk setting that depended on
teamwork. The administrative official at the medical facility meeting,
drawing an analogy between the aviation example and participants' work,
noted that an RCA had found that an adverse event could have been
prevented if authority had been challenged. His message to the
meeting's participants was that VA's leadership saw cross-checking as
acceptable and necessary.
Rewarding Close Call Reporting:
The same facility that held small meetings for staff developed a close
call reward system that reinforced the idea that reporting a close call
not only did not result in punishment but was actually rewarded. Staff
feared a negative atmosphere when the close call program was first
established, with staff telling on one another, but this did not occur.
The number of close calls at this facility was few before the reward
program began. In the first 6 months of the program, 240 close calls
were reported. While we were visiting the Patient Safety Managers, many
staff called them to report close calls; each staff member was given a
$4 cafeteria certificate.
Patient Safety Managers at this facility told us that they rewarded
reporting, no matter who reported or how trivial the report. The unit
with the month's best close call received a plate of cookies. The
Patient Safety Manager reported that a milestone had been reached when
a chief of surgery reported a close call--a first for surgery
leadership. "Rewarding Close Call Reporting" paraphrases leaders and
clinicians on the success of the close call program at their facility.
Rewarding Close Call Reporting:
Leader 1: With the close call program, the wards do not feel as
secretive. VA leadership thought the new close call program might cause
staff to turn on one another and begin to blame one another for
reporting close calls, but this has not happened.
Nurse 1: People are rewarded for reporting close calls and adverse
events--and not punished.
Nurse 2: I feel comfortable about reporting close calls and adverse
events. When management first introduced the close call program, we
thought everyone was going to tell on each other. If everyone starts to
find out things about you, you could lose your job, because it could be
on your record. You would have to ask yourself, "Is this something I
would really want to tell someone about?" We thought it would be like
"Big Brother Is Watching You." But that is not what it's like. I feel
comfortable reporting close calls and adverse events.
Administrative Official: To promote patient safety, we did a lot of
reward and recognition to let staff know that what they have done
[reporting close calls and adverse events] is important.
[End of text box]
Other facilities did not have as extensive a reward system. At one
facility, the Patient Safety Manager had recently given a certificate
to someone who had done a good job in describing an adverse event.
However, at another facility, the quality manager who supervised
Patient Safety Managers told us that she thought it improper to reward
staff for reporting: She did not want to reward people for almost
making a mistake. Clinicians in our interviews, however, pointed to the
need to develop reward programs around patient safety. For example, one
nurse said that if she were the director, she would call staff to thank
them for reporting close calls and adverse events and would develop a
reward system.
Summary:
We found that leaders used three management strategies at one facility
that promoted the Program: (1) storytelling; (2) teaching, coaching,
and role modeling open communication in staff meetings; and (3)
offering rewards for participation in close call reporting. These
strategies changed clinicians' attitudes and behavior, because they
believed that the Program is an organizational priority, and they acted
on this by reporting more close calls. An important part of the Program
is encouraging close calls to surface so that safeguards can be
established before patients are harmed.
[End of section]
Chapter 5: Conclusions and Recommendations:
Five years into VA's Program to improve the safety of patients' care at
its medical facilities, we found progress at certain facilities but
continuing barriers to the Program's adoption at others. Having
recognized the risks to patients that are inherent in medical care, VA
seeks with its Program to identify and fix system flaws before they can
harm patients. To successfully change its culture, VA acknowledges that
it is necessary to change staff attitudes, beliefs, and behavior from
those of fear of blame to open willingness to report close calls and
adverse events. The fear is rooted in, and reinforced by, many years of
professional training and experience in medical care settings. In the
four facilities in which we studied the Program's progress, we were
able to measure significant differences in clinicians' familiarity with
and participation in the Program and the levels of cultural support for
it.
We conclude that progress in patient safety could be facilitated if
VA's program efforts focused on facilities where familiarity with the
Program's major concepts is low--concepts such as close call reporting,
the NASA confidential reporting program, and RCAs--and on the
facilities where participation in RCAs and levels of cultural support
for the Program are low. VA may be able to use lessons learned by
focusing on clinicians' perspectives to prioritize future actions to
further the goal of patient safety.
VA should have tools available to determine which facilities face
barriers to adopting the Program and, therefore, need assistance in
stimulating culture change and promoting the Program. VA is to be
commended for conducting a cultural survey that showed staff attitudes
toward safety at the national level. However, since it was not a random
survey, it was not effective in discerning staff attitudes at the local
level. In addition, VA has not measured staff knowledge of the Program,
staff participation in RCAs, or whether facility staff have enough
mutual trust to support the Program. VA may be able to adapt measures
we have suggested, such as adding to its survey some of our questions
that focus on these issues, so as to identify facilities for specific
interventions and assess the Program's progress at the local and
national levels.
Measuring Clinicians' Familiarity with and Cultural Support for the
Program:
Clinicians' familiarity with the Program and opportunities to
participate in RCAs could be measured at each facility in order to
identify facilities that require specific interventions. Because low
familiarity or participation can hinder the success of the Program, VA
could attempt to measure and improve basic staff familiarity with the
Program's core concepts and ensure opportunities to participate in RCA
teams. Our study developed measures of familiarity with and
participation in the Program by analyzing responses from interviews of
a small random sample of clinicians, and these could be further
developed into useful measures in a larger study. These measures could
also be developed into goals to be achieved nationally and, more
importantly, locally for each facility.
According to the clinicians we interviewed, the supportive culture of
individual facilities plays a critical role in clinicians'
participation in the Program and warrants VA leadership's priority. In
one of the three facilities where staff had above average familiarity
with the Program, staff told us that fear prevented them from fully
participating in the Program. From the clinicians' vantage point, their
leaders need not accept given levels of mutual trust or comfort in
reporting close calls and adverse events; instead, once facilities are
identified as having low cultural support for the Program, that can be
a starting point for change. In our conversational interviews with
clinicians, they consistently pointed to specific workplace conditions
that fostered their mutual trust and comfort in reporting. Notably,
management can take actions to stimulate culture change by developing a
work environment that reinforces patient safety. Drawing from their own
experience, clinicians had views that were consistent with many studies
of culture change in organizations, indicating that leaders' actions
and open communication are important in the transformation sought under
the Program.
We were able to directly observe practices that have convinced
frontline workers that the Program is a priority for VA, that it is
worth their while to participate in it, and that by doing so medical
facilities are safer for patients. These practices included
leadership's demonstrating to staff that patient safety is an
organizational priority--for example, by coaching and by communicating
safety stories in face-to-face meetings with all staff--and that the
organization values reporting close calls because it rewards and does
not punish staff for reporting them.
Recommendations for Executive Action:
To better assess the adequacy of clinicians' familiarity with,
participation in, and cultural support for the Program, we recommend
that the Secretary of Veterans Affairs direct the Under Secretary for
Health to take the following three actions:
1. set goals for increasing staff:
* familiarity with the Program's major concepts (close call reporting,
confidential reporting program with NASA, root cause analysis),
* participation in root cause analysis teams, and:
* cultural support for the Program by measuring the extent to which
each facility has mutual trust and comfort in reporting close calls and
adverse events;
2. develop tools for measuring goals by facility; and:
develop interventions when goals have not been met.
Agency Comments and Our Evaluation:
We provided a draft of this report to VA for its review. The Secretary
of Veterans Affairs stated in a December 3, 2004, letter that the
department concurs with GAO's recommendations and will provide an
action plan to implement them. VA also commented that the report did
not address the question of whether VA's work in patient safety
improvement serves as a model for other healthcare organizations. GAO's
study was not designed to evaluate whether VA's program was a model,
compared with other programs, but was limited to how the program had
been implemented in four medical facilities. VA also provided several
technical comments that we incorporated as appropriate.
[End of section]
Appendix I: Content Analysis, Statistical Tests, and Intercoder
Reliability:
Content Analysis:
To analyze the data we collected, we used content analysis, a technique
that requires that the data be reduced, classified, and sorted. In
content analysis, analysts look for, and sometimes quantify, patterns
in the data. We conducted tests on clinicians' responses to our key
variables and found a number of significant differences. We also
conducted intercoder reliability tests--that is, we assessed the degree
to which coders agreed with one another. The tests showed that the
consistency among the coders was satisfactory.
Ethnography:
Ethnography is a social science method, embracing qualitative and
quantitative techniques, developed within cultural anthropology for
studying a wide variety of communities in natural settings. It allowed
us to study the Program in VA's medical facilities. Ethnography is
particularly suited to exploring unknown variables, such as studying
what in VA's culture at the four facilities affected the Program. In
our open-ended questions, we did not supply the respondents with any
answer choices. We allowed them to talk at length, and therefore the
interviews lasted anywhere from a half hour to an hour or more.
Ethnography is also useful for giving respondents the confidence to
talk about sensitive topics. We anticipated that clinicians would find
the study of VA's medical facility culture, including staff views of
close calls and adverse events, a sensitive subject. Therefore, we gave
full consideration to the format and context of the interviews.
Although ethnography is commonly associated with lengthy research aimed
at understanding remote cultures, it can also be used to inform the
design, implementation, and evaluation of public programs. Governments
have used ethnography to gain a better understanding of the
sociocultural life of groups whose beliefs and behavior are important
to federal programs. For example, the U.S. Census Bureau used
ethnographic techniques to understand impediments to participation in
the census among certain urban and rural groups that have long been
undercounted.[Footnote 37]
Data Collection:
We conducted fieldwork for approximately a week at each of two
facilities, for 3 weeks at a third, and for 25 days at the fourth.
Although ethnographers traditionally conduct fieldwork over a year or
more, we used a more recent rapid assessment process (RAP). RAP is an
intensive, team-based ethnographic inquiry using triangulation and
iterative data analysis and additional data collection to quickly
develop a preliminary understanding of a situation from the insider's
perspective.[Footnote 38]
We drew two samples, one judgmental and one random. To understand how
the Program was implemented at each medical facility, we conducted
approximately a hundred nonrandom interviews with facility leaders,
Patient Safety Managers, and a variety of facility employees at all
levels, from maintenance workers, security officers, nursing aides, and
technicians to department heads. This allowed us a detailed
understanding of how the Program was implemented at each facility.
To ensure that we represented clinicians' views at all four facilities,
we selected a random sample of 80, using computer-generated random
numbers from an employee roster of clinicians, yielding 10 physicians
and 10 nurses at each facility.[Footnote 39] While this provided us
with a representative sample of clinicians (physicians and nurses) from
each facility, the size of this sample was too small to provide a
statistical basis for generalizing from our survey results to the
entire facility or to all facilities. For both samples, we used a
similar semistructured questionnaire (see app. III). It consisted of
mostly open-ended questions and a few questions with yes-or-no
responses. At every interview, we asked staff for their ideas, and we
incorporated a number of their perspectives into this report.
A hallmark of ethnography is its observation of behavior, attitudes,
and values. Observation is conducted for a number of purposes. One is
to allow ethnographers to place the specific issue or program they are
studying in the context of the larger culture. Another, in our case,
was to allow some facility staff to feel more comfortable with us as we
interviewed them. Both purposes worked for us in this study.
Because we had observed meetings and RCA teams at work, we could better
understand respondents' answers. Respondents noted how comfortable they
were in talking to us and how different our conversational interviews
were from other interviews they had experienced in the past. We
observed staff in their daily activities. For example, we accompanied a
nurse while she administered medication using bar code technology that
scans the medication and the patient's wristband. We also observed
staff at numerous meetings, including RCA team meetings, patient safety
conferences, patient safety training sessions, staff meetings in which
patient safety was discussed, and daily leadership meetings.
Our methodology included collecting data from facility records. We
examined all close calls and adverse events reported for a 1-month
period and all RCA reports conducted at each facility, and we reviewed
administrative boards and rewards programs. We read minutes from
patient safety committees and other committees that addressed safety
issues.
Data Analysis:
Our data were mostly recorded, but some interviews were written,
depending on respondents' permission to record. Using AnnoTape,
qualitative data analysis software, we coded the interviews for both
qualitative and quantitative patterns, and we used the software to
capture paraphrases for our analysis.
We developed a prescriptive codebook to guide the coders in identifying
interviews and classifying text relevant to our variables. After
several codebook drafts, we agreed on common definitions and uses for
the codes. In the content analysis of our random sample data, we looked
for patterns, associations, and trends. AnnoTape allowed us to mark a
digital recording or transcribed text with our codes and then sort and
display all the marked audio or text bites by these codes. Because all
the coders operated from a common set of rules, we achieved a
satisfactory intercoder rater reliability score. AnnoTape also allowed
us to record prose summaries of the interviews, some of which
paraphrased what the clinicians said; the paraphrases we present in the
report reflect the range of views and perceptions of VA staff at the
four medical facilities. A rough gauge of the importance of their views
is discernible in the extent to which certain opinions or perceptions
are repeatedly expressed or endorsed.
Using the statistical package SAS, we analyzed the variables with two-
choice and three-choice answers and transferred them to an SAS file for
quantitative analysis. Among the quantifiable variables were five yes-
or-no questions asking about respondents' familiarity with key elements
of the Patient Safety Program. We created a new variable that reflected
a composite familiarity score for the Program, using the five questions
about familiarity with the key elements (the questions are listed in
the note to fig. 4). We also assessed respondents' levels of comfort in
reporting close calls and adverse events and mutual trust among staff
at each facility, based on each whole interview. We used these two
assessments, rated high, middle, or low to characterize cultural
support for the Patient Safety Program.
In quantifying verbal answers for display and comparison purposes, we
decided that the maximum individual familiarity, trust, and comfort
levels should be 10. Thus, in each key elements question, we let "yes"
equal 2 and "no" equal 0, ensuring that an individual who knew all of
the five elements would achieve a composite score of 10. Finally, we
averaged composite scores to get an average score for each facility. In
the trust and comfort summary judgments, we let "high" equal 10,
"medium" equal 5, and "low" equal 0. Rather then display these numbers,
we used a scale of high, medium, and low for 10, 5, and 0 and placed
the answers accordingly.
Significance Testing:
We were able to determine statistically significant differences in
clinicians' responses by facility and, unless otherwise noted, we
report only significant results.
First, we conducted a nonparametric statistical test, called Kruskal-
Wallis, on all possible comparisons in the subset of variables that we
report in our text.[Footnote 40] Four of these variables were central
to the report: comfort summary score, trust summary score, close call
score, and root cause score. In the Kruskal-Wallis test, each
observation is replaced with its rank relative to all observations in
the four samples. Tied observations are assigned the midrank of the
ranks of the tied observations. The sample rank mean is calculated for
each facility by dividing its rank sum by its sample size.
If the four sampled populations were actually identical, we would
expect our sample rank means to be about equal--that is, we would not
expect to find any large differences among the four medical facilities.
The Kruskal-Wallis test allows us to determine whether at least one of
the medical facilities differs significantly from at least one other
facility. This test showed that--for each of the comfort, trust and
close call variables--at least one of the medical facilities differed
significantly from at least one of the other medical facilities.
Next, we conducted a follow-up test to determine specifically which
pairs of medical facilities were significantly different from other
pairs on key variables. This follow-up test is a nonparametric multiple
comparison procedure called Dunn's test.[Footnote 41] Our using Dunn's
test meant testing for differences between six pairs of medical
facilities: A vs. B, A vs. C, A vs. D, B vs. C, B vs. D, and C vs. D.
Table 4 presents the results of Dunn's test, along with each facility's
sample rank mean and sample size. The pairs of facilities that are
statistically significantly different from one another are in the far
right column. Note that for the root cause characteristic, there are no
statistically significant findings from the multiple comparison
testing, which conforms to the results of the earlier Kruskal-Wallis
test on root cause.
Table 4: Nonparametric Multiple Comparison Results:
Characteristic: Comfort;
Facility A: 25.5 (20);
Facility B: 49.4 (20);
Facility C: 43.6 (19);
Facility D: 41.7 (20);
Statistically significant comparison[A]: A vs. B***, A vs. C***, A vs.
D**.
Characteristic: Trust;
Facility A: 28.8 (19);
Facility B: 44.4 (21);
Facility C: 46.3 (20);
Facility D: 41.7 (20);
Statistically significant comparison[A]: A vs. B*, A vs. C**.
Characteristic: Close call[B];
Facility A: 38.5 (20);
Facility B: 49.2 (20);
Facility C: 42.7 (20);
Facility D: 26.3 (18);
Statistically significant comparison[A]: B vs. D***, C vs. D**.
Characteristic: Root cause[C];
Facility A: 43.0 (20);
Facility B: 39.4 (21);
Facility C: 43.1 (20);
Facility D: 36.3 (19);
Statistically significant comparison[A]: None.
Source: GAO analysis.
Note: Numbers are sample rank means and, in parentheses, sample sizes.
[A] Significance levels 0.0250, 0.0167, and 0.0083 are indicated by
three, two, and one asterisks, respectively. These significance levels
were determined by dividing overall significance levels 0.15, 0.10, and
0.05, respectively, by 6, or the number of comparisons.
[B] A sum of scores on "Do you know what close call or near miss
reporting is?" and "Do you know what the Patient Safety Reporting
System to NASA is?"--a related subgroup of the knowledge questions.
[C] A sum of scores on "Do you know what an RCA is?" "Have you
participated in an RCA?" and "Do you know anyone who has participated
in an RCA?"--a related subgroup of the knowledge questions.
[End of table]
Intercoder Reliability:
Consistency among the three coders was satisfactory. We assessed
agreement among the coders for selected variables for interviews with
seven clinicians--that is, we assessed the extent to which they
consistently agreed that a response should be coded the same. To
measure their agreement, we used Krippendorff's alpha reliability
coefficient, which equals 1 when coders agree perfectly or 0 when
coders agree as if chance produced the results, indicating a lack of
reliability.[Footnote 42] Our Krippendorff's alpha values ranged from
0.636 to 1.000 for nine of the selected variables (see table 5).
Compared with Krippendorff's guidelines that alpha is at least 0.8 for
an acceptable level of agreement and ranges from 0.667 to 0.8 for a
tentative acceptance, we believe our overall our results are
satisfactory.
Table 5: Intercoder Reliability Assessment Results:
Variable: Q2 Facility location;
Krippendorff's alpha: 0.878.
Variable: Q5 Respondent set;
Krippendorff's alpha: 1.000.
Variable: Q8 Respondent title;
Krippendorff's alpha: 1.000.
Variable: Q17 Change;
Krippendorff's alpha: [A].
Variable: Q18 Promotes safety;
Krippendorff's alpha: [A,B].
Variable: Q19 Undercuts safety;
Krippendorff's alpha: [A,B].
Variable: Q20 Close call recognition;
Krippendorff's alpha: 0.796.
Variable: Q21 PSRS;
Krippendorff's alpha: 0.818.
Variable: Q23 RCA recognition;
Krippendorff's alpha: [A,B].
Variable: Q24 RCA participation;
Krippendorff's alpha: 0.808.
Variable: Q25 RCA knows participant;
Krippendorff's alpha: 0.636.
Variable: Summary comfort score;
Krippendorff's alpha: 0.757.
Variable: Summary trust score;
Krippendorff's alpha: 0.791.
Source: GAO analysis.
[A] For this question, we consider Krippendorff's alpha indeterminate:
(1) the coders did not disagree (there was no variation) or (2) there
was one disagreement among them but otherwise no variation.
[B] To calculate Krippendorff's alpha, we used a computer program in N.
Kang and others, "A SAS MACRO for Calculating Intercoder Agreement in
Content Analysis," Journal of Advertising 22:2 (1993): 17-28.
[End of table]
[End of section]
Appendix II: A Timeline of the Implementation of VA's Patient Safety
Program:
This timeline highlights the training programs and other events NCPS
completed between 1997 and 2004.
Year: 1997;
Event:
* VA announces a special focus on patient safety;
* VA drafts patient safety handbook[A];
* VA develops Patient Safety Event Registry[B].
Year: 1998;
Event:
* Patient Safety Awards Program begins[C];
* Expert Advisory Panel is convened to look at reporting systems.
Year: 1999;
Event:
* Four Patient Safety Centers of Inquiry are funded;
* NCPS is established and funded[D];
* VA informs Joint Commission on Accreditation of Healthcare
Organizations that it will go beyond JCAHO's sentinel event reporting
system to include close calls;
* VA pilots RCAs at six facilities;
* Institute of Medicine issues To Err Is Human.
Year: 2000;
Event:
* VA and NASA sign interagency agreement on the confidential Patient
Safety Reporting System;
* NCPS adverse event and close call reporting system established
throughout VA;
* NCPS trains clinical and quality improvement staff in patient safety
topics, including the RCA process;
* VA establishes Patient Safety Manager (hospital level) and Officer
(network level) positions.
Year: 2001;
Event:
* RCA training continues;
* Online and print newsletter Topics in Patient Safety begins
publication;
* RCA software is rolled out;
* Facilities and networks are given the performance measure of
completing RCAs in 45 days;
* Healthcare Failure Mode and Effect Analysis (HFMEA), a proactive risk
assessment tool is developed by VA and rolled out through multiple
videoconferences.
Year: 2002;
Event:
* Aggregate RCA implementation is phased in over the year[E];
* New hires are trained in RCAs and Patient Safety Officers and
Managers are given refresher training;
* The Veterans Health Administration's Patient Safety Improvement
Handbook, 3rd rev. ed. (VHA 1050.1), is officially adopted;
* Facilities are given a new performance measure, being required to
conduct proactive risk assessment, using HFMEA to review contingency
plans for failure of the electronic bar code medication administration
system;
* The American Hospital Association (AHA) sends Program tools developed
by VA to 7,000 hospitals[F];
* Rollout of confidential reporting to NASA is largely complete.
Year: 2003;
Event:
* Facility directors receive a day of training to reinforce what they
could do to improve the success of their patient safety programs;
* Facilities are given a performance measure for timely installation of
software patches to critical programs;
* VA begins to provide training, funded by the Department of Health and
Human Services, for state health departments and non-VA hospitals as
the "Patient Safety Improvement Corps, an AHRQ/VA Partnership".
Year: 2004;
Event:
* Facility managers, for example, Nurse Executives and Chiefs of Staff,
receive a day of patient safety training;
* VA plans a patient safety assessment to document the Program's
progress;
* Directors are given the performance measure of timely verification of
radiology reports.
Source: NCPS and GAO. We updated the timeline at www.patientsafety.gov
and revised it with input from NCPS.
[A] Revising VA's patient safety handbook was one of the first tasks
NCPS took on in 1999; it was finally published as Patient Safety
Improvement Handbook, 3rd rev. ed. (VHA 1050.1) and officially adopted
by VA in 2002. The handbook, now part of NCPS's training material, is
available at VA's Web site.
[B] VA's Safety Event Registry, developed in 1997, is an internal VA
program for collecting data on adverse events. VA reports certain
"sentinel events" to JCAHO.
[C] According to NCPS, the Patient Safety Awards Program, begun in
1998, is no longer active.
[D] In the report, we consider that the Patient Safety Program began in
1999, when NCPS was established.
[E] Regularly held aggregate RCAs examined close call and adverse event
reports that are grouped by commonly occurring events, such as falls.
[F] In 2002, AHA sent Patient Safety Program tools that VA had
developed to 7,000 hospitals. The tools were videotapes about the
Program and guides on how to conduct RCAs. AHA believed these tools
would help non-VA hospitals develop their own Programs on patient
safety.
[End of table]
From 1999 through 2004, NCPS has conducted training in the Patient
Safety Program. It was attended primarily by quality managers and
Patient Safety Officers and Managers. Typically, the training lasted 3
days and included an introduction to the new Patient Safety Improvement
Handbook and small group training in the RCA process. Trainees,
especially Patient Safety Managers, were expected to take the Program
back to their medical facilities, collect and transmit reported adverse
events and close calls to NCPS, and guide clinicians in the RCA teams.
We observed health fairs at several of the four facilities.
Beginning in 2003, NCPS convened medical facility directors and other
managers in 1-day sessions that introduced them to the systemic
approach to improving patient safety, including a blame-free approach
to adverse events in health care.
[End of section]
Appendix III: Semistructured Interview Questionnaire:
Interviewer, please fill out items 1-12.
1. Interview number:
2. Code name for VAMC:
3. Pseudonym:
4. File name:
5. From sample list:
6. Interviewer:
7. Person writing up interview:
8. Date:
9. Profession (circle or bold one) Nurse Doctor:
10. Title:
11. Unit:
12. Was informed consent signed? - yes -no:
Questions to Ask Respondent:
Background:
13. How many years have you had a license to practice in your specialty
as a:
doctor (years):
nurse (years):
14. How many years have you worked at VA?
15. How many years have you worked at this medical center?
16. What are the specialties of the people you work with on a regular
basis?
17. Tell me a little about what you do at work.
Reciprocity:
18. To what extent do you perceive there is trust or distrust:
(a) Within your profession at your VAMC? (nurses if nurse, doctors if
doctor, etc.):
(b) Within your unit or team?
(c) Between your profession and other departments? Please provide
examples.
What else?
Patient Safety:
19. In your time at VA, what changes have you seen with regard to
patient safety at this medical center? Please provide examples. What
else?
20. What do you find that supports an atmosphere that promotes patient
safety? Please provide examples. What else?
21. What undercuts patient safety? Please provide examples. What else?
Reporting:
22. Do you know what close call or near miss reporting is?
23. Do you know what the Patient Safety Reporting System to NASA is?
24. One of the goals of the Patient Safety Program is to create an
atmosphere in which VA staff feel comfortable reporting adverse events
and close calls without punishment or blame. To what extent do you
think this is happening at your medical center? Please provide
examples. What else? What more could be done? Root Cause Analysis:
25. Do you know what a root cause analysis (RCA) is? Explain.
26. Have you participated in an RCA? Please provide examples. Any
other? 27. Do you know anyone who has? Please provide examples. Anyone
else?
Wiidcard:
28. If you were in charge of the medical facility and you had all the
money and staff you needed, what would you do to bring about the
transformation to a patient safety culture?
Suggestions for Focus of Study:
29. What else should we be focusing on or asking about patient safety?
[End of section]
Appendix IV: Comments from the Department of Veterans Affairs:
THE SECRETARY OF VETERANS AFFAIRS:
WASHINGTON:
December 3, 2004:
Ms. Nancy Kingsbury:
Managing Director:
Applied Research and Methods:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Ms. Kingsbury:
The Department of Veterans Affairs (VA) has reviewed the Government
Accountability Office's (GAO) draft report, VA PATIENT SAFETY
INITIATIVE: A Cultural Perspective at Four Medical Facilities, (GAO-05-
83). VA concurs with GAO's recommendations and will provide an action
plan to implement the recommendations in our response to GAO's final
report.
VA notes your report demonstrates that the Department's efforts at
patient safety improvement have resulted in significant accomplishments
and identifies areas that merit special attention. Your data,
indicating 78 percent of the clinicians interviewed knew of the Root
Cause Analysis process, and 75 percent understood the concept of a
'close call" suggest the successful penetration of the patient safety
precepts that the Veterans Health Administration (VHA) has introduced.
More importantly, however, this suggests the active participation of
clinicians in patient safety improvement.
While VA is pleased with the positive tone of your report, GAO did not
address the question of whether VA's work in patient safety improvement
serves as a model for other health care organizations. This is a
significant question and was primary in GAO's proposal for this study
in 2002. VA believes its work does serve as a model, and GAO should
identify those aspects of the program that have led to the successes
noted throughout their report.
Thank you for the opportunity to review this draft report.
Sincerely yours,
Signed by:
Anthony J. Principi:
Enclosure:
DEPARTMENT OF VETERANS AFFAIRS (VA) COMMENTS TO GOVERNMENT
ACCOUNTABILITY OFFICE (GAO) DRAFT REPORT, VA PATIENT SAFETY INITIATIVE.
A Cultural Perspective at Four VA Medical Facilities (GAO-05-83):
To better assess the adequacy of clinicians' familiarity with,
participation in, and cultural support for the Initiative, GAO
recommends that the Secretary of Veterans Affairs direct the Under
Secretary for Health to take the following three actions:
1. Set goals for increasing staff:
* familiarity with the initiative's major concepts (close call
reporting, confidential reporting program with NASA, root cause
analysis):
* participation in root cause analysis teams.
* cultural support for the Initiative by measuring the extent to which
each facility has mutual trust and comfort in reporting close calls and
incidents.
2. Develop tools for measuring goals by facility.
3. Develop interventions when goals have not been met.
Concur - The Department of Veterans Affairs (VA) concurs with GAO's
findings and recommendation. A detailed action plan is being developed
and VA will provide the action plan to GAO as part of VA's response to
the final report.
VA believes the critique in GAO's report regarding mutual trust may
lead readers to believe that VA does not understand and address this
topic. VA believes that the topic is understood and addressed. For
example, the patient safety survey VA undertook in the year 2000
focused on many issues relevant to mutual trust. VA medical center
directors received the results of the 2000 survey for their respective
facilities, as well as national data, and the directors were able to
use this information in local patient safety improvement efforts. Many
of these questions were used in a new patient safety survey developed
by the Agency for Healthcare Research and Quality, the Department of
Defense and the American Hospital Association. VA will review the
updated survey that is planned for implementation in 2005 and consider
adding several questions to explicitly address the topic of mutual
trust.
[End of section]
Appendix V: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Nancy R. Kingsbury (202) 512-2700, kingsburyn@gao.gov;
Charity Goodman (202) 512-4317, goodmanc@gao.gov:
Staff Acknowledgments:
Additional staff who made major contributions to this report were
Barbara Chapman, Bradley Trainor, Penny Pickett, Neil Doherty, Jay
Smale, George Quinn and Kristine Braaten. Donna Heivilin, recently
retired from GAO, also played an important role in preparing this
report.
[End of section]
Glossary:
Center of Inquiry:
A research and development arm of NCPS's Patient Safety Program. The
centers concentrate on identifying and preventing avoidable, adverse
events, and each has a different focus.
Close Call:
An event or situation that could have resulted in harm to a patient
but, by chance or timely intervention, did not. It is also referred to
as a "near miss."
Frontline Staff:
Staff directly involved with patient care.
Adverse Event:
An incident directly associated with care or services provided within
the jurisdiction of a medical facility, outpatient clinic, or other
Veterans Health Administration facility. Adverse events may result from
acts of commission or omission.
Joint Commission on Accreditation of Healthcare Organizations:
JCAHCO is an accrediting organization for hospitals and other health
care organizations.
Medical Facility:
A VA hospital and its related nursing homes and outpatient clinics.
National Center for Patient Safety:
NCPS is the hub of VA's Patient Safety Program, where approximately 30
employees work, in Ann Arbor, Michigan. Other employees work in the
Center of Inquiry in White River Junction, Vermont, and in Washington,
D.C.
Patient Safety Reporting System:
PSRS, a confidential and voluntary reporting system in which VA staff
may report close calls and adverse events to a database at the National
Aeronautics and Space Administration.
Root Cause Analysis Team:
An interdisciplinary group that identifies the basic or contributing
causes of close calls and adverse events.
[End of section]
FOOTNOTES
[1] Certain management practices are essential in creating safety
within an organization and in the success of organizational change for
improving patient safety: (1) balancing the tension between production
efficiency and reliability (safety), (2) creating and sustaining trust
throughout the organization, (3) actively managing the process of
change, (4) involving workers in making decisions pertaining to work
design and work flow, and (5) using knowledge management practices to
establish the organization as a "learning organization." (See Ann Page,
ed., Keeping Patients Safe: Transforming the Work Environment of
Nurses, Washington, D.C.: National Academies Press, 2004, pp. 3-4.)
Throughout this report, we refer to the various patient safety
initiatives under the National Center for Patient Safety (NCPS) as the
Patient Safety Program, or the Program. The initiatives we studied
included adverse event and close call reporting, root cause analysis
(RCA), and the confidential reporting system to the National
Aeronautics and Space Administration (NASA).
[2] GAO, Patient Safety Programs Promising but Continued Progress
Requires Culture Change, GAO/T-HEHS-00-167 (Washington, D.C.: July 27,
2000), p. 3.
[3] See for example, Annick Carnino, "Management of Safety, Safety
Culture and Self Assessment," http://www.iaea.or.at/ns/nusafe/publish/
papers/mng_safe.htm, (Feb. 19/2002); Columbia Accident Investigation
Board, The CAIB Report, vol. 1 (Arlington, Va.: Aug. 26, 2003). http:/
/www.caib.us/ (Sept. 9, 2004) and Gaba, David "Structural and
Organizational Issues in Patient Safety: A Comparison of Health Care to
Other High-Hazard Industries," California Management Review 43:1 (Fall
2000): 83-102.). A review of research on influences on collaboration
also found that "mutual respect, understanding, and trust" appeared
more often than any other factor to be a positive influence (see Paul
Mattessich and others, Collaboration: What Makes It Work, 2nd ed. (St.
Paul, Minn.: Amherst H. Wilder Foundation, 2001)).
[4] Highly effective safety organizations share the following
characteristics: (1) acknowledgment of the high-risk, error-prone
nature of the organization's activities, (2) a blame-free environment
in which individuals can report close calls without punishment, (3) an
expectation of collaboration across ranks to seek solutions to
vulnerabilities, (4) the organization's willingness to direct resources
toward addressing safety concerns, (5) communication founded on mutual
trust, (6) shared perceptions of the importance of safety, and (7)
confidence in the efficacy of preventive measures. (See M. D. Cooper,
"Toward a Model of Safety Culture," Safety Science 36 (2000): 111-36,
and Lucian L. Leape and others, "Promoting Patient Safety by Preventing
Medical Error," JAMA 280:16 (Oct. 28, 1988): 1444-47.)
[5] Ethnography is research carried out in a natural setting--such as a
workplace--and using multiple types of data, both qualitative and
qualitative. The approach embraces diverse elements that influence
behavior. Most important, it recognizes that what people say, do, and
believe reflect a shared culture--a set of beliefs and values---that
can be discovered by systematic study of their behavior. Ethnography
produces a picture of social groups from their members' viewpoint. (See
Margaret D. LeCompte and Jean J. Schensul, Ethnographer's Toolkit, vol.
1, Designing and Conducting Ethnographic Research (Lanham, Md.: Rowman
& Littlefield, 1999).) Other ethnographers consider the multicultural
image of organizations as leading to a consideration of culture's
cohesive, as well as divisive, functions. In this case, culture is
defined as a learned way of coping with experience. Kathleen Gregory
notes "More researchers have emphasized the homogeneity of culture and
its cohesive functions." However, she also describes a multicultural
model that could be divisive in function among different occupational
or ethnic groups. See Kathleen Gregory, "Native-View Paradigms:
Multiple Cultures and Culture Conflicts in Organizations,"
Administrative Science Quarterly 28 (1983): 359-76.
[6] GAO, Organizational Culture: Techniques Companies Use to Perpetuate
or Change Beliefs and Values, GAO/NSIAD-92-105 (Washington, D.C.: Feb.
27, 1992); Weapons Acquisition: A Rare Opportunity for Lasting Change,
GAO/NSIAD-93-15 (Washington, D.C.: Dec. 1, 1992); Managing in the New
Millennium: Shaping a More Efficient and Effective Government for the
21st Century, GAO/T-OCG-00-9 (Washington, D.C.: Mar. 9, 2000); Results-
Oriented Cultures: Implementation Steps to Assist Mergers and
Organizational Transformations, GAO-03-669 (Washington, D.C.: July 2,
2003); and High-Performing Organizations: Metrics, Means, and
Mechanisms for Achieving High Performance in the 21st Century Public
Management Environment, GAO-04-343SP (Washington, D.C.: Feb. 13, 2004).
[7] One of the goals of the Center for Evaluation, Methods, and Issues
in GAO's Applied Research and Methods group is to find new tools for
evaluation; one purpose in conducting this study was to see if
ethnography was a practical tool for GAO to use in studying an
organization's culture. By statute, "[t]he Comptroller General shall
develop and recommend to Congress ways to evaluate a program or
activity the Government carries out under existing law." See 31 U.S.C.
§717(c) (2000).
[8] Regarding aspect no. 1, see James P. Spradley, The Ethnographic
Interview (New York: Holt, Rinehart and Winston, 1997).
[9] VA's survey was a nonrandom survey sent to 6,000 clinicians; it
provides a description of VA culture but not an adequate and reliable
measure for generalizing at the facility level. Although NCPS asked
each facility to use a random sample, NCPS staff acknowledged that in
many cases this was not done. Furthermore, although the survey
presented questions on cultural attitudes and beliefs, such as
attitudes about punishment and shame for reporting adverse events, it
did not address staff understanding of concepts such as close call
reporting, root cause analyses (RCAs), confidential reporting systems,
whether staff participated in RCA teams, or whether staff explicitly
had mutual trust.
[10] See James Beebe, Rapid Assessment Process (Lanham, Md.: Rowman &
Littlefield, 2001). Before we began fieldwork, we also visited each
facility and conducted numerous interviews for approximately 3 to 5
days in order to write our study protocol.
[11] Leape and others, "Promoting Patient Safety by Preventing Medical
Error," p. 1444.
[12] VA's health care system plays an important role in teaching
physicians and nurses. It has 193,000 full-time-equivalent employees.
The 158 medical facilities are organized into 21 regional networks.
[13] GAO/NSIAD-92-105.
[14] David M. Gaba, "Structural and Organizational Issues in Patient
Safety: A Comparison of Health Care to Other High-Hazard Industries,"
California Management Review 43 (2000): 83-102.
[15] For fiscal year 2004, information was collected through August 4.
[16] Efforts under NCPS that we did not study included prospective
analysis of potential problems (such as reviewing contingency plans for
failure of the electronic bar code medication administration system),
safety protocols focused on surgery, and a system of technical alerts
to warn clinicians of malfunctioning mechanical equipment.
[17] The Patient Safety Program does not replace VA's existing
accountability systems, which include VA internal review boards,
compromise or settlement of monetary claims, and referring possible
criminal cases to the Department of Justice. See 38 C.F.R. §§14.560,
14.561, 14.600 (2004). If an RCA team determines that a crime is
suspected or has been committed, it initiates the review process by
referring the matter to the facility director. Similarly, questions
involving quality of performance are handled outside the Program.
[18] All RCA material and findings are part of VA's medical quality-
assurance program. Records developed under the program are
confidential, privileged, and subject to limited disclosure. See 38
U.S.C. §5705 (2000).
[19] Only reported adverse events and close calls that meet certain
criteria of seriousness and frequency are examined in RCAs.
[20] John, Corrigan, and Donaldson, eds., To Err Is Human, p. 99.
[21] For more on NCPS and its implementation of the Program, see the
timeline in appendix II.
[22] Missing patients includes patients who have a pass to leave their
unit and have not returned on time, as well as patients who leave
without a pass.
[23] VA Office of Medical Inspector, VA Patient Safety Event Registry:
First Nineteen Months of Reported Cases Summary and Analysis
(Washington, D.C.: June 1997-Dec. 1998), p. 12.
[24] To measure how familiar the staff were with the Program's core
concepts, we calculated the average familiarity, grouped by facility,
by combining answers for the series of questions noted in figure 4.
More information about our methods is in appendix I; our questionnaire
is in appendix III.
[25] We studied the attitudes, beliefs, and behavior of clinicians
directly involved in patient care. Ethnographic studies of U.S.
hospital workers other than clinicians reveal their unique
perspectives. See, for example, Karen Brodkin Sacks and Dorothy Ramey,
My Troubles Are Going to Have Trouble with Me (Brunswick, N.J.: Rutgers
University Press, 1984), and Karen Brodkin Sacks, Caring by the Hour:
Women, Work, and Organizing at Duke Medical Center (Chicago: University
of Illinois Press, 1988).
[26] For our purposes, workflow refers to the coordination of tasks
within and across teams, and professional values refers to norms that
are learned from formal and informal training and that are reinforced
on the job.
[27] Cultural support is a composite measure of levels of mutual trust
and comfort in reporting close calls and adverse events for each of
four groups of clinicians.
[28] In chapter 2, we described a scale of low, medium, and high
familiarity with the Program that combined the answers to the following
questions: Do you know what a close call is? Do you know what the
Patient Safety Reporting System is? Do you know what an RCA is? Have
you participated in an RCA? Do you know anyone who has participated?
[29] Using content analysis, we grouped clinicians' responses to open-
ended questions in categories. We asked them a series of questions
about trust, such as "To what extent do you perceive there is trust or
distrust within your profession? Your team? And between your profession
and other departments?" To measure comfort in reporting, we asked, "One
of the goals of the Patient Safety Program is to create an atmosphere
in which VA staff felt comfortable reporting adverse events and close
calls without punishment or blame. To what extent do you think this is
happening at your medical facility?" Many clinicians returned to the
subject of trust and comfort in reporting adverse events and close
calls spontaneously in the interviews, as when they answered questions
like "What promotes patient safety?" and "What undercuts patient
safety?" (More detail on our methodology is in app. I; our questions
are in app. III.)
[30] Page, ed., Keeping Patients Safe, pp. 3-4.
[31] The supportive culture necessary for patient safety is hard to
achieve in a complex medical setting. According to the Institute of
Medicine, when hospital staff are not fearful of reporting and when
they have mutual trust, they cooperate better and are more successful
at integrating their work tasks within and across teams. However,
hospitals are complex social systems of numerous professions and work
groups, and the work often involves high-risk tasks, making intrateam
and interteam coordination difficult (see Page, ed., Keeping Patients
Safe, pp. 3-4). Charles L. Bosk notes distrust between clinicians in
different specialties, such as surgeons and radiologists or
anesthetists and internists (see Bosk, Forgive and Remember: Managing
Medical Error, Chicago: University of Chicago Press, 1979, p. 105)).
[32] VA told us that despite the sample not being random, the NCPS did
provide local results to facility directors in case the information was
useful.
[33] For example, Schein highlights practices that help leaders
transmit culture to, and embed it in, the organization and help staff
learn new practices from (1) how leaders react to critical incidents,
organizational crises, and deliberate role modeling, teaching, and
coaching and (2) criteria leaders use for allocating rewards and
status. See Edgar H. Schein, Organizational Culture and Leadership (San
Francisco, Calif.: Jossey-Bass, 1991).
[34] VA leaders told us that performance errors involve patterns of
behavior that require disciplining physicians and other staff. For
example, the same nurse giving out the wrong medicine three times in a
month becomes a performance issue.
[35] Storytelling can be a way to implement system change. See, for
example, Stephen Denning, The Springboard: How Storytelling Ignites
Action in Knowledge-Era Organizations (Boston, Mass.: Butterworth-
Heinemann, 2000); Ann T. Jordan, "Critical Incident Story Creation and
Culture Formation in a Self-Directed Work Team," Journal of
Organizational Change Management 9:5 (1996): 27-35; and
GAO/NSIAD-92-105.
[36] For more on storytelling as a tradition in medicine, see Bosk,
Forgive and Remember, pp. 103-10.
[37] GAO, Federal Programs: Ethnographic Studies Can Inform Agencies'
Actions, GAO-03-455 (Washington, D.C.: March 2003).
[38] See James Beebe, Rapid Assessment Process.
[39] At one site, we interviewed 11 physicians, so our random sample
actually consisted of 81 staff.
[40] Rank sum tests such as Kruskal-Wallis are designed for situations
in which the distributions of the populations that are the source of
data are unknown.
[41] Dunn's test is a multiple comparison procedure considered
appropriate for use following a Kruskal-Wallis test. See Wayne W.
Daniel, Applied Nonparametric Statistics (Boston: Houghton Mifflin,
1978), p. 212.
[42] The advantage of using Krippendorff's technique is, among others,
that it applies to any number of coders, any number of categories or
scale values, any level of measurement, incomplete or missing data, and
large and small sample sizes.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
