To comply with the ACGME NAS, the OUSMC Radiology Residency Program has developed and implemented a Clinical Competency Committee and Program Evaluation Committee, which will provide the structure needed to support the goals of the NAS.
Clinical Comptentency Committee (CCC)
- The program director must appoint the Clinical Competency Committee. (Core)
- At a minimum the Clinical Competency Committee must be composed of three members of the program faculty. (Core)
- The program director may appoint additional members of the Clinical Competency Committee.
- These additional members must be physician faculty members from the same program or other programs, or other health professionals who have extensive contact and experience with the program’s residents in patient care and other health care settings. (Core)
- Chief residents who have completed core residency programs in their specialty and are eligible for specialty board certification may be members of the Clinical Competency Committee. (Core)
- There must be a written description of the responsibilities of the Clinical Competency Committee. (Core)
- The Clinical Competency Committee should: review all resident evaluations semi-annually; (Core) ; prepare and ensure the reporting of Milestones evaluations of each resident semi-annually to ACGME; and, (Core) advise the program director regarding resident progress, including promotion, remediation, and dismissal. (Detail)
CLINICAL COMPETENCY COMMITTEE (CCC)
OSUMC Radiology Residency
Purpose of the CCC:
- Responsible for assessing each resident’s progress in acquiring the relevant reporting Milestones of the resident’s level of training.
- Responsible for determining residents’ progress on the educational milestones, must make recommendations regarding resident progress, including promotion, remediation, and dismissal.
- The CCC will use data from direct faculty observation, in service exams, case log progression, faculty, self and 360 evaluations and data from previous Milestone assessments to assess each resident’s progress and competency toward achieving the milestones.
- Recommendations will be part of an early warning system for residents who may not be progressing as expected.
- CCC decisions will be submitted for the Radiology Program Director who will make the ultimate decision as to what action to take regarding the resident. If an action is made contrary to the decision of the CCC, sufficient justification of a contrary action must be provided.
Committee Membership:
- Members including the Chair are appointed by the program director.
- Committee will consist of 3-8 faculty members per ACGME guidelines. At least 5 faculty members will be required to be part of the committee. Having a composition of 5 core faculty ensures that enough members will be present at each meeting to share in resident assessment.
- There will be no term limit to committee membership.
Current Committee members:
- Chair, Chris Vassiliou, DO
- Damon Brooks, DO; Hooby Yoon, DO; Jonathan Kirkland, DO; Donald von Borstel, DO
- Jeremy Fullingim, DO
Meetings:
- Meet semi-annually for milestone rankings. (Inaugural meeting is set for June 8th, 2016)
- Particular attention will be made to problem areas or perceived deficiencies.
- Meeting will be anywhere from 1 1/2 to 3 hours.
- Meeting will be held at DIA Headquarters.
- Attendance is mandatory. If a CCC member is to be absent, they are responsible for reviewing the material and submitting their input to the Radiology CCC chairman prior to the meeting.
Roles and Responsibilities of the Members:
Program Director:
- Does not serve as chair
- Takes meeting minutes
- Observer member of the committee
- Provides any information needed by committee members
- Document any necessary information to residents record
- Record recommendations on each resident by milestone
- Provides feedback from the CCC to the residents semi-annually at scheduled progress evaluation review. Feedback includes if necessary discussing strategies in implementation of steps toward improvement and remediation.
Program Coordinator:
- Schedules meeting and notifies attendees
- Aggregating data sources (both electronically and paper)
- Providing information to members before the meeting so they can complete required pre-work.
- Summarizing data, preparing "scorecards" or “snapshots”. Schedule meetings with residents to review CCC decisions, including Milestone status.
- With program director, submit Milestone information on each resident to the ACGME
CCC Chair:
- 3 year maximum consecutive term
- Responsible for calling the committee to order.
- Holding votes on all decisions.
- Reporting decisions to the Program Director if not present and other CCC members who are absent from the meetings.
- Ultimately, if adverse actions are required, the Chair of the CCC will assist the Program Director in conveying the decision of the CCC and help to explain the decision process to the Resident.
- Responsible for relaying any deficiencies of the educational curriculum to the Program Director for discussion at the next meeting of the Radiology Program Evaluation Committee (PEC)
- Chair will be available to meet the ACGME site visitor as needed.
- The CCC Chair is needed to provide oversight and organization to the committee. It also provides a single contact point that is available to approach with issues regarding the CCC. As CCC Chair, this member will have responsibility for the overall production of reports and implementation of action plans, in conjunction with the Program Director. As such, the Chair should be available to meet with members of the ACGME as needed.
Committee Faculty members:
- Mandatory meeting attendance
- Each member will be responsible for reviewing relevant assessment material on each resident prior to each meeting (preferably quarterly), and this material is available through New Innovations, MyPortolio (RSNA), and any submitted material from the Program Director or Coordinator.
- Collectively, during the meeting, members are expected to provide honest, thoughtful evaluations of each resident and participate in consensus decisions about the trainee’s competency level.
- Decisions on the trainee’s competency level must be based on multisource input, not personal input alone.
- Prepare and assure the reporting of milestones evaluations of each resident semi-annually to ACGME.
- Making recommendation to the Program Director for resident progress, including promotion, probation, remediation, dismissal and graduation.
- Identifying any perceived deficiencies in the program’s educational curriculum and relaying this information to the Radiology Program Evaluation Committee (PEC)
- Reaching a consensus agreement of milestones narrative meaning, determining how many assessments are needed for each milestone and determine the quality of dat available.
- Ensuring there is a current description of the CCC and its functions which are available to residents and faculty.
General committee information:
CCC members are responsible for reviewing all resident evaluations and other assessment materials prior to each meeting. Having reviewed the materials on all residents, each CCC member will have in mind where they would rank the resident on each Milestone, as well as an idea of each resident’s overall competency, when they arrive to each meeting. Discussion of the evaluation and assessment materials for each resident between the members of the CCC will result in a consensus agreement about where the resident should be ranked on the Milestone evaluation that will be submitted to the ACGME. The CCC members will also be responsible for recommending resident promotion, probation, remediation, dismissal and graduation to the Program Director, upon which, the Program Director will make the final decision. If an action is made contrary to the decision of the CCC, sufficient justification of a contrary action must be provided.
Throughout the review of evaluations, other assessment materials and the Milestone evaluation of each resident, the CCC members are responsible for identifying any potential or perceived deficiencies in the Residency program’s educational curriculum. If a deficiency is identified, the CCC chairman is responsible for relaying this information to the Program Director, for discussion at the next meeting of the Radiology Program Evaluation Committee (PEC).
Likewise, throughout the review of evaluation and assessment materials, each CCC member is responsible for identifying any weaknesses in the content or aggregate of evaluation tools. Any such deficiencies will be relayed to the CCC chairman for discussion with the Program Director.
If a member is unable to attend a CCC group meeting, then that member will provide their input to the CCC chairman one week prior to the CCC meeting so that the chairman will have the opportunity to review and report the findings to the committee.
If a deficiency is identified, the committee members are responsible for devising and implementing a corrective action plan. Members are expected to keep all results of resident evaluations confidential and shared only with fellow committee members.
Resident Enhancement and Corrective Action Plans:
CCC members will review individual resident performances all evaluations and assessment materials. If the resident is meeting competency or exceeding competency, recommendations can be made to provide the resident with a guide to further enhance his or her development. Areas for improvement or resident deficiencies will require a more detailed corrective action plan. If during the evaluation review process or at any other time, the CCC members identify a resident has critical deficiencies, or characteristics that may threaten the health and well-being of patients or the resident, immediate action can be taken to eliminate or minimize any harm. This can include an emergency meeting of the CCC to discuss the particular resident.
If a resident fails to progress through the milestones, or if a separate resident issue is identified, a formal action plan will occur as follows:
- The committee will determine an appropriate course of action to correct the issue
- The plan will be written out with specific recommendations and a timeline for the resident
to demonstrate progression. This will be kept with the Program Coordinator as part of the
resident’s file. - The CCC Chairman and Program Director will meet with the resident to review and enact the
action plan. - The plan will also be shared with the resident’s faculty advisor, with consideration to confidentiality issues
- The resident’s progression will again be reassessed at the quarterly CCC meeting. If the resident continues to show failure of progression in the same area, the CCC Chairman and Program Director will meet to determine the next step in corrective action.
Faculty Development:
- Committee members will be oriented to each assessment tool and its relationship to the reporting Milestones. The CCC will decide how many assessments are needed for any given Milestone, how to aggregate data across tools, and how to verify data quality.
- All program faculty will be trained regarding the reporting Milestones and their associated assessment tools to enhance rating consistency and accuracy. Training will include discussion of the Milestone levels and an agreement will be established on the meaning assigned to each tool’s rating anchors. The CCC’s approach to aggregating and interpreting assessment data from each resident will need to be discussed with and agreed on by all faculty members, in an ongoing fashion, to ensure consistency in evaluation decisions.
CCC process of continuous improvement:
- At each meeting, the CCC will have the opportunity to discuss revision of CCC guidelines and goals, including number and duration of meetings, reports generated, and implantation of action plans. Any changes to guidelines will need to undergo vote, a majority of members approving of changes.
- In the CCC minutes, an included section for questions to be addressed during annual program review. Questions include “Do all residents have the opportunity to achieve all levels of competency in a Milestone within the current program structure?”
- Documentation will be continued on how the Milestone consensus was reached and determine whether the appropriate assessment tools are available for all Milestones. Questions addressed will include: “Does the program gather sufficient data for each Milestone?”; “Any new assessment tools needing to be implemented?”
DIAGNOSTIC RADIOLOGY MILESTONE PROJECT
The Milestones are designed only for use in evaluation of resident physicians in the context of their participation in ACGME accredited residency or fellowship programs. The Milestones provide a framework for the assessment of the development of the resident physician in key dimensions of the elements of physician competency in a specialty or subspecialty. They neither represent the entirety of the dimensions of the six domains of physician competency, nor are they designed to be relevant in any other context.
MILESTONE REPORTING
The milestones are designed for programs to use in semi-annual review of resident performance and reporting to the ACGME. Milestones are knowledge, skills, attitudes, and other attributes for each of the ACGME competencies organized in a developmental framework from less to more advanced. They are descriptors and targets for resident performance as a resident moves from entry into diagnostic radiology residency through graduation. In the initial years of implementation, the Review Committee will examine milestone performance data for each program’s residents as one element in the Next Accreditation System (NAS) to determine whether residents overall are progressing.
For each reporting period, review and reporting will involve selecting the level of milestones that best describes each resident’s current performance level in relation to milestones. Milestones are arranged into numbered levels. Selection of a level implies that the resident substantially demonstrates the milestones in that level, as well as those in lower levels. A general interpretation of levels for diagnostic radiology is below:
Level 1: The resident demonstrates milestones expected of one who has had some education in diagnostic radiology.
Level 2: The resident is advancing and demonstrating additional milestones.
Level 3: The resident continues to advance and demonstrate additional milestones; the resident consistently demonstrates the majority of milestones targeted for residency.
Level 4: The resident has advanced so that he or she now substantially demonstrates the milestones targeted for residency. This level is designed as the graduation target.
Level 5: The resident has advanced beyond performance targets set for residency and is demonstrating “aspirational” goals, which might describe the performance of someone who has been in practice for several years. It is expected that only a few exceptional residents will reach this level.
Additional Notes
Level 4 is designed as the graduation target but does not represent a graduation requirement. Making decisions about readiness for graduation is the purview of the residency program director (see the following NAS FAQ for educational milestones on the ACGME’s NAS microsite for further discussion of this issue: “Can a resident graduate if he or she does not reach every milestone?”). Study of milestone performance data will be required before the ACGME and its partners will be able to determine whether Level 4 milestones and milestones in lower levels are in the appropriate level within the developmental framework, and whether milestone data are of sufficient quality to be used for high stakes decisions.
- Patient Care and Technical Skills
- Medical Knowledge
- Systems-based Practice
- Practice-based Learning and Improvement
- Professionalism
- Interpersonal and Communication Skills
Program Evaluation Committee (PEC)
The program director must appoint the Program Evaluation Committee (PEC). (Core)
Common Program Requirements NAS 12
The Program Evaluation Committee:
- must be composed of at least two program faculty members and should include at least one resident; (Core)
- must have a written description of its responsibilities; and, (Core)
- should participate actively in:
- planning, developing, implementing, and evaluating educational activities of the program; (Detail)
- reviewing and making recommendations for revision of competency-based curriculum goals and objectives; (Detail)
- addressing areas of non-compliance with ACGME standards; and, (Detail)
- reviewing the program annually using evaluations
- of faculty, residents, and others, as specified below. (Detail)
V.C.2. The program, through the PEC, must document formal, systematic evaluation of the curriculum at least annually, and is responsible for rendering a written, annual program evaluation. (Core)
The program must monitor and track each of the following areas:
- resident performance; (Core)
- faculty development; (Core)
- graduate performance, including performance of program graduates on the certification examination; (Core)
- program quality; and, (Core)
- Residents and faculty must have the opportunity to evaluate the program confidentially and in writing at least annually, and (Detail)
The program must use the results of residents’ and faculty members’ assessments of the program together with other program evaluation results to improve the program. (Detail)
progress on the previous year’s action plan(s). (Core)
The PEC must prepare a written plan of action to document initiatives to
Common Program Requirements NAS 13
improve performance in one or more of the areas listed in section V.C.2., as well as delineate how they will be measured and monitored. (Core)
The action plan should be reviewed and approved by the teaching faculty and documented in meeting minutes. (Detail)
Program Evaluation Committee
OSUMC Diagnostic Radiology Residency Program
The goal of this Program Evaluation Committee (PEC) is to oversee curriculum development and program evaluations for the OSUMC Diagnostic Radiology Residency Program.
The PEC of OSUMC Diagnostic Radiology Residency Program will meet semi-annually (December and June). The PEC will have at least three members, two program faculty and one trainee from the program. Faculty members may include physicians and non-physicians from the OSUMC Diagnostic Radiology Program. The PEC is composed of the following members:
1. Chair: Jeremy Fullingim, DO; Program Director
2. Chris Vassiliou, DO
3. Jonathan Kirkland, DO
4. Donald von Borstel, DO
5. Hooby Yoon, DO
6. John Walton, DO
7. Adam Foster, DO; PGY4 (R3)
The committee’s responsibilities are to:
* Plan, develop, implement, and evaluate educational activities of the program;
* Review and make recommendations for revision of competency-based curriculum goals and objectives;
* Address areas of non-compliance with ACGME standards;
* Review the program annually using evaluations of faculty, residents, or clinical fellows, and others;
* Document on behalf of the program, formal, systematic evaluation of the curriculum least annually and render a written Annual Program Evaluation (APE) to be included in the Annual Program Director Update and given to the GME office.
* Monitor and track each of the following:
- Resident performance
- Faculty development
- Graduate performance including performance on certifying examination;
- Program quality; and
- Progress in achieving goals set forth in previous year’s action plan.
* Review recommendations from the Clinical Competency Committee
The PEC will be provided with confidential resident and faculty evaluation data by the programs’s administrative staff in order to conduct their business.
The program director is ultimately responsible for the work of the PEC. The program director must assure that the annual action plan is reviewed and approved by the programs’s teaching faculty. The approval must be documented in meeting minutes. The program’s annual action plan and report on the program’s progress on initiatives from the previous year’s action plan must be sent to the GME office annually.