| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Perspective |
1 Both authors: Department of Radiology, Cincinnati Children's Hospital Medical Center, 3333 Burnet Ave, Cincinnati, OH 45229-3039.
Received August 27, 2004; accepted after revision September 27, 2004.
Address correspondence to L. F. Donnelly.
OBJECTIVE. The objective of this article is to describe the process of implementing a program for performance-based assessment of clinical faculty.
CONCLUSION. A performance-based assessment of clinical faculty program facilitated quality improvements in our department, improved communication concerning our reappointment process, and was well received by faculty. The presence of measures on a scorecard stresses what is important for clinical care and facilitates process improvement in these areas. Having practitioner-specific data compiled in an organized fashion helps meet the standards of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO).
Credentialing in medical institutions is the formal recognition of professional and technical competence [1]. Credentialing is based on the principle that it is not the right of every physician to practice medicine in a given institution but rather a privilege extended by that institution in accordance with applicable law [1]. The intent of credentialing is that an institution ensures that all the members of its health care team are competent and qualified to provide the services for which they have been granted privileges and that its governing board is responsible for the implementation of the credentialing process to guarantee the safety of its patients and the quality of care provided by its staff [1]. There are three major components of the process of credentialing: initial appointment, delineation of clinical privileges, and reappointment of medical staff [1, 2]. In this article, we focus on issues pertaining to reappointment.
In 2004, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) published new guidelines on the initial granting, renewal, and revision of privileges for practicing health care providers [1, 2]. These JCAHO guidelines state that hospital organizations must undertake a performance assessment of all medical staff to be used in the process of reappointment. The guidelines state that practitioner-specific data must be accumulated and used in the process of reappointment. Practitioner-specific data should be compared with aggregate data and should cover the areas of clinical judgment and technical skills, professionalism and communication, and continued education and clinical self-improvement [2].
Evaluation of performance by radiology faculty is extremely difficult. Many of the attributes that contribute to an excellent radiologist are difficult to define and even more difficult to measure. A number of difficulties in evaluating physician performance in general have been identified. "Competency" has been defined as a "complex set of behaviors built on the components of knowledge, skills, and attitudes" [3]. It has been suggested that ideal performance measures should be evidence-based and based on agreed-on standards, be reproducible, be attributable to the individual physician, be encountered in adequate numbers so that statistical evaluation is meaningful, and be feasible to collect [4]. For most radiology departments, identifying such parameters may be difficult. It is often difficult to identify reproducible, measurable, and available parameters that reflect the true nature of what it means to be a "competent" radiologist.
The purpose of our article is to describe a program for performance-based assessment of clinical radiology faculty for the purposes of reappointment. The intent is to create a program that fosters process improvement, meets JCAHO standards, and minimizes additional paperwork and data collection that are not already in process. This program is definitely a work in progress and is by no means perfect or universally applicable to all departments. We continually strive to improve the program. In addition, with rapidly changing technology, the targets or parameters that should be measured are constantly changing. We hope that the description of our program will serve as a template for performance-based review of faculty that others can use as a reference in building programs that work for their own departments.
Performance-Based Assessment of Clinical Faculty
The process for performance-based assessment of clinical faculty was implemented in a pediatric radiology department. However, we believe that despite the fact that our department is a pediatric radiology department, the same program could be easily implemented in other academic or general radiology departments. The department of radiology at our institution has 28 clinical pediatric radiology faculty, performs approximately 160,000 imaging examinations per year, annually trains seven pediatric radiology fellows and multiple radiology residents for accreditation with the Accreditation Council for Graduate Medical Education (ACGME), and has all radiology subspecialties represented including interventional radiology.
For process improvement and department monitoring, our department of radiology has established a departmental scorecard. The scorecard monitors how the department is performing on clinical services; education; research; professionalism, communication, and user satisfaction; finance and administration; and staffing. In each of these areas, there are multiple measures. For each measure, there is an established goal, our current measure, the interval at which the data for the measure are calculated, and the date at which the measure was last updated. The department scorecard consists of mean aggregate data for the department. As with other department scorecards [5], this scorecard measures the practice performance of the department in aggregate.
At the center of the program for performance-based assessment of clinical faculty is a similarly designed scorecard (Fig. 1) that sets expectations for performance. The department of radiology gathers information on all radiology faculty who participate in clinical care of patients. The data are compiled on the performance-based assessment of clinical faculty scorecard. These data are compiled annually. The scorecards and the databases from which the information is culled are stored in the department of radiology. The data compiled on the scorecards are used in the reappointment process of the medical staff at our institution. This process and the associated documents are considered part of the quality assurance activities of our institution and, as such, are not subject to legal discovery. All committees involved in the review of these documents and those preparing and submitting information claim all privileges afforded by the guidelines for quality assurance activities in our state.
|
The faculty of the department of radiology also undergo an annual review process in which the clinical, educational, research, and administration accomplishments are evaluated and the next year's goals are established. The faculty performance as it pertains to professionalism and effective communication is also reviewed. This information from the annual faculty review process is also considered during the reappointment process.
Three areas are evaluated on the performance-based assessment of clinical faculty scorecard: clinical judgment and technical skills, professionalism and communication, and education and clinical self-improvement. In each area, data are collected for multiple measures. For each measure, a goal is established, the current measurement is recorded, and the interval at which that data will be evaluated is established. The specific measures used are picked for a combination of reasons: importance to the quality of clinical care rendered, existing data currently available, and the accuracy of available data.
Clinical Judgment and Technical Skills
In the area of clinical judgment and technical skills, multiple measures
are recorded. One of the major measures in this area is culled from our
program for peer review in the evaluation of frequency of faculty errors.
Multiple potential systems for peer review are available, and these include
randomly assigned double interpretations, comparison between dictations and
surgical or autopsy reports, and use of commercially available products
[6].
We chose a system of peer review that takes advantage of the large number of interdisciplinary conferences and clinical team consultations of a tertiary hospital already in place. In accordance with our peer-review process [6], any time a radiology staff member becomes aware of a discrepancy in the formal interpretation of a study on reviewing that study in relationship to either a follow-up study, a prior study, clinical consultation with clinical colleagues, or a clinical case review conference, that case is logged into a peer-review database. In addition, all cases presented at interdisciplinary conferences of radiology and other services are reviewed and when discrepancies between radiology performance and clinical, surgical, or pathologic follow-up occur, these cases are entered into the system.
A process by which radiology faculty can add cases to a peer-review folder within our PACS has been established, and all such cases are reviewed on a monthly basis during a dedicated radiology conference. Faculty members are encouraged to place cases into the peer-review system so that there are an adequate number of cases to be analyzed and to ensure input from a wide variety of faculty. In practice, participation of faculty is wide-spread. Faculty often place their own mistakes ino the peer-review system.
For all cases entered into the peer-review database, the following data are recorded: faculty member involved in the case and the type of error made (perception, interpretation, communication, professionalism, or technical error). An error in perception is considered to be present when a finding was not recognized by the radiologist. An error in interpretation is considered to be present when the meaning assigned to an identified finding is inaccurate. An error in communication is considered to be present when the findings were identified and appropriately interpreted but that information did not reach the appropriate individuals. Professional errors will be discussed elsewhere. Technical errors are considered present when an imaging examination was not performed using the appropriate protocol or was not performed correctly.
The peer-review database is reviewed annually, and the number of appearances for each staff member is tallied and normalized for the number of imaging examinations interpreted by that particular faculty member. The percentage of cases requiring peer review is calculated for each faculty member. The mean percentage of cases requiring peer review is then calculated for the department along with the SD. In the event that a staff member has a value more that 2 SDs beyond the mean, this is reported to the radiologist in chief. In such a case, the type and significance of the errors are reviewed, and patterns of recurrent errors are identified and evaluated. Errors are considered significant if they resulted in or had potential to result in harm to the patient. If multiple significant errors or patterns of recurring errors are found, the radiologist in chief then notifies the staff radiologist and together the faculty and radiologist in chief determine whether any additional training or other interventions are necessary [6].
This peer-review program has potential limitations. The cases entered into the peer-review system that are not identified in interdisciplinary conferences are entered voluntarily by the radiologist, and this is subjective and could potentially lead to bias. In our department, the peer-review process is popular with the radiologists and the peer-review conference is perceived as one of the most valuable continuous learning tools we currently have. This may in part be due to the fact that the peer-review process was instituted before physician-specific data were being tracked. This peer-review process is described in more detail elsewhere [6].
Other measures for the area of clinical judgment and technical skills include the percentage of physician reports with a sign-off time of within 24 hr, whether pediatric advanced life-support certification is up-to-date, whether the staff member has participated in a presentation of or contributed to a publication on applications within pediatric radiology, and whether the staff member is eligible or certified with a Certification of Added Qualification in pediatric radiology from the American Board of Radiology. There are two additional measures for the radiologist involved in interventional radiology. This includes success rate for sonographically guided peripheral inserted central catheter (PICC) placement and procedural complication rate. For both of these parameters, as with peer review, the goal is to have no physicians with values that lie outside 2 SDs of the department mean.
Professionalism and Communication
The medical community and general press have recently focused on the
concept of professionalism for health care workers. Although professionalism
is a complex set of behaviors with multiple characteristics, there are, in our
opinion, three general categories that must be present in order for a health
care provider to be perceived as professional: clinical competence, effective
communication skills, and an ethical decision process with prioritization of
the patient's best interest. All three must be present for a health care
provider to be perceived as professional. For example, a physician who has
effective communication skills and sincerely has the patient's best interest
in place is not perceived as professional if the physician is clinically
incompetent. Likewise, a physician may be clinically competent and have the
patient's best interest as a priority, but if the physician is unable to
communicate or behave appropriately, the patient does not consider the
physician professional. Finally, a physician who is clinically competent and a
good communicator but talks his or her way out of performing an indicated
emergent procedure to attend a child's birthday party would not be considered
professional because he or she has not prioritized treatment in the patient's
best interest. For our performance-based scorecard, clinical competence is
evaluated in a separate area, as previously discussed.
The area of professionalism and communication primarily evaluates effective communication skills and physician behavior. Although it seems inherently obvious that effective communication skills are key to running a successful health care service, there is evidence that problems with communication is a major issue in health care. Approximately 30% of patient dissatisfaction is attributed to either problems in communication between patients or their families and health care professionals or perceived disrespectful behavior [7]. In pediatrics, approximately one third of parents report that their child's doctor or other health care professional communicates poorly in terms of listening carefully, showing respect, and explaining things well. Effective communication between patients and physicians has been shown to have a positive impact on patient satisfaction, health care outcomes, and medical costs [8]. In addition, poor communication and behavior have been shown to increase the chance of litigation in the event of an adverse effect [7].
Effective communication skills and professionalism are stressed as important and fundamental parts of the delivery of health care services in our radiology department. A booklet titled "Professionalism in the Radiology Department" is distributed to all faculty and incoming fellows and residents. The booklet focuses on the characteristics of a physician that are essential to patient and family-centered care such as being courteous, attentive, respectful, and enthusiastic and ensuring the patient's safety. For each of these characteristics, examples of behaviors to model and behaviors to avoid are described.
At an aggregate departmental level, patient and family surveys are used to monitor the professionalism of our staff. Families are asked questions concerning whether they were treated with dignity and respect by our front desk staff and technologists, hand-washing policies were followed, the staff introduced themselves and explained their role in their child's care, their privacy was respected, and the special needs of the child and family were addressed.
Concerning practitioner-specific data, two measures are followed to address professionalism and communication. We have instituted a complaint tracking system. Any complaints from parents, referring physicians, or hospital coworkers regarding radiology faculty behavior, professionalism, or communication are investigated. If the complaints are deemed legitimate, these are logged in the complaint tracking system. The goal set for the scorecard is that each radiology faculty member have no more than one such complaint per year.
In addition, each of our pediatric radiology fellows evaluates faculty professionalism and communication halfway through and at the end of each academic year. Each faculty member is graded on a scale of 1 (low) to 4 (high) on parameters such as using effective communication skills, being a role model for professionalism, interacting with referring physicians, exhibiting teamwork, and interacting with patients and families. A mean score for such parameters is calculated for each faculty member, and the goal is that each faculty member has a score of greater than 3.0 of the possible 4.0. We find that this method of using the fellow evaluations works in our department. However, it is subject to and based on the ability of the fellows to evaluate such behaviors. In our experience, the fellows actually perform higher in standards of professionalism than do the faculty (unpublished data), supporting the opinion that having fellows evaluate faculty behavior is a useful tool. In addition, this method is dependent on having a large number of fellows. Other measures may work better in other departments.
Education and Clinical Self-Improvement
Assessments of medical staff at most institutions rely heavily on
educational and clinical self-improvement measures. Continuing medical
education (CME) credits are evaluated by most state medical licensing programs
and are often included as part of recredentialing. In addition, at many
institutions, health care providers attend a multitude of educational programs
and perform self-assessment testing, often in the form of online test modules.
Therefore, a number of preexisting measures that can be used in this section
of the scorecard are typically available.
The concept of clinical self-improvement is also in line with the American Board of Medical Specialties concepts of lifelong learning and establishment of maintenance-of-certification testing [3, 4]. For our scorecard, we use the following measures: the faculty meets CME requirements for state medical licensing and has taken and passed our in-house JCAHO national safety standards training, a behavior and professionalism online test module, and a patient safety online test module. In addition, for those faculty involved in patient sedation, our institutional procedural sedation online module must have been taken and passed.
Summary
A performance-based assessment of clinical faculty program can easily be put together to accomplish several goals. The measures that are on the scorecard stress what is important for clinical care and facilitate process improvement in these areas. Having practitioner-specific data compiled in an organized fashion helps meet JCAHO standards. In many departments, such a program that relies primarily on preexisting data collected for other purposes can be put together.
We believe that institution of this program has led to quality improvements in our department, has improved communication concerning our reappointment process, and has been well received by our faculty. The process has highlighted many of the positive events that were already occurring. This program is a work in progress, and we are continuously striving to improve it. Many of the parameters used in our department may not work well in other departments. We hope that the description of our program may help serve as a template for others to develop their own programs for performance-based assessment of radiology faculty and to determine parameters to be measured in their departments.
References
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  J. B. Kruskal, C. S. Yam, J. Sosna, D. T. Hallett, Y. J. Milliman, and H. Y. Kressel Implementation of online radiology quality assurance reporting system for performance improvement: initial evaluation. Radiology, November 1, 2006; 241(2): 518 - 527. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. F. Donnelly and J. L. Strife Establishing a Program to Promote Professionalism and Effective Communication in Radiology Radiology, March 1, 2006; 238(3): 773 - 779. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. FitzGerald Performance-Based Assessment of Radiology Faculty Am. J. Roentgenol., January 1, 2006; 186(1): 265 - 265. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
