Pomphrey Consulting
  • Education
    • Courses & Webinars
    • Registration by Invoice
  • Trauma Registrar Staffing
  • Trauma Center Development
  • Resources
    • Contact
    • Calculators
    • Trauma Library
    • Blog
    • Careers
  • Our Story
    • Company Overview >
      • Our Mission
      • Our Experience
      • Our Story
    • What People Are Saying
    • Why Partner with Us?
    • CEO Bio

Blog Source

for Trauma Registry Knowledge

Data Point Inter-rater Reliability for Targeted Trauma Registrar Education Leads to Pristine Data

1/22/2021

 
Inter-rater reliability (IRR) audits are a must for every trauma center.  The American College of Surgeon (ACS) requires each trauma center to monitor data validity.  This is considered a criterion deficiency (CD 15-10).  They recommend to re-abstract 5 to 10 percent of your patient records. [i]  Data validation simply means checking the accuracy and quality of your trauma registry data.  You want to verify this source data before using it or exporting it to the National Trauma Data Bank (NTDB).  Once exported to NTDB it is used for benchmarking.  These risk -adjusted benchmarking reports are used to measure performance and outcomes for all trauma centers.  This to is a criterion deficiency if not done. (CD 15-5)[ii]

Consider for a moment the significance of these benchmarking reports and the reflection on your trauma centers performance.  The consequences of substandard data are immense.  Data quality or its overall utility rests solely on the trauma registrars’ instruction.  Therefore, a data point driven IRR is multifaceted for the trauma program.  Not only does it assure accurate benchmarking, but also accurate internal performance improvement, and research.
Trauma registrar education is a key component for every trauma program.  ACS recommends core education as they start and continuing education each year.  This is paramount to the overall success of a data driven program.  Data point IRR quickly identifies the specific area a trauma registrar is experiencing difficulty in verses the high-level chart-based IRR accuracy.
Typical IRR audits focus on this chart level accuracy.  In this approach the overall audit my yield an IRR of 96% for the chart on Patient A.  The issue with this approach on the surface this appears acceptable.  However, it does not identify or guide the reviewer to potential repetitive abstraction errors for key data elements.  Data point IRR is focuses on just that.
Pomphrey Consulting recommends data point driven IRR for optimal overall results. By utilizing this method, you are targeting the exact area of education needed.  As areas are identified, provide educational opportunities for the data element, and remedy this misinterpretation or misconception of the data elements definition, or specific data source.

How to use the data-point IRR strategy
First, identify data elements which are most important to your trauma program, it’s performance improvement initiatives and research.  Benchmarking data elements must also be factored in.  Several key considerations are:
  • Probability of Survival. (Includes: Age, Blunt verses Penetrating, initial vital signs (RTS) and all injuries (appropriate AIS coding for ISS calculations)
  • Risk Adjustment Elements (Pre-existing Conditions, Hospital Events, or prehospital management)
  • Key Benchmarking Elements (ICU days, Vent days, or Hospital days)
  • Verification Elements (Activations, Attending Presence)
 
Second, during the data validation audit, calculate and track IRR individually for these select data elements.  This is not a noteworthy extension of workload.  ACS again recommends 5 to 10 percent re-abstraction, but they do not specify which data elements, in part or in its entirety.  Our approach is simply a targeted approach for these key elements and tracking performance at this fundamental level. 
Third, utilizing these results clearly identify educational needs. We recommend considering any data-point IRR under 95% as an opportunity to improve. Education efforts can use specific cases to illustrate mistakes and clarify data definitions. In our experience, focusing on specific data points increases learner retention and increases registrar confidence.

Utilizing Results to Update Internal Data Dictionary
This data point driven exercise can also assist you in streamlining your internal data dictionary.  This exercise increasing data quality across the board, and thus optimizes consensus among all trauma registrars.  A prime example of this is “ED Admission Time”.  Guided by staff IRR results you reveal each trauma registrar is utilizing a different location within the electronic medical record (EMR).

Registrar #1 utilizes the ED Encounter Summary, Registrar #2 the ED Patient Care Timeline, and Registrar #3 the Pre-Hospital Arrival Time.  While these values may be the same, the likelihood is small.  While the times may be only minutes apart, this one data element affects the ED Dwell time calculation which may be an internal performance improvement initiative. Best practice is to substantiate each registrar utilize the same location.  With this goal in mind, the internal data dictionary would be updated to mirror the National Trauma Data Standard Data Dictionary with an informational hierarchy.

Data Source Hierarchy Guide
  1. ED Patient Care Timeline
  2. ED Encounter Summary
  3. Pre-Hospital Run Sheet
 
Data-point IRR can assist in identifying similar source issues and ensure your internal data dictionary is as precise as possible with clear hierarchy for each data element.  This may sound meticulous, but the advantage is a dramatic increase in the confidence of your trauma data internally, as well as with your benchmarking reports.

Benefits of this approach
Tracking data driven IRR permits a better overall view of trauma registrar performance and data accuracy. As the foundation of an educational strategy, it can lead to steady increases in overall IRR. This will increase data reliability and build your confidence in TQIP benchmarking reports. In addition, this strategy can allow trauma programs to use their registrar education budget more effectively.

Author

Michelle Pomphrey, Pomphrey Consulting President, MLT, RN, CSTR


 Do you want to strengthen and streamline your data validation processes?

Pomphrey Consulting uses proprietary software from Buck Data Validation Systems (BDVS) to automatically report data-point IRR. This system streamlines key aspects of data-point IRR, giving trauma program leaders powerful tools for improving data quality. BDVS can be customized to your program’s needs and internal data validation processes. To find out more, explore Data Management Services from Pomphrey Consulting.

[i] Resources for Optimal Care of the Injured Patient, Chapter 15, page 112

[ii] Resources for Optimal Care of the Injured Patient, Chapter 15, page 109

Comments are closed.

    Archives

    June 2021
    May 2021
    April 2021
    March 2021
    February 2021
    January 2021

    Categories

    All
    ACS
    TCAA
    Trauma Data
    Trauma Program Data
    Trauma Registrar

    RSS Feed

Pomphrey Consulting: assisting trauma hospitals in the US with data abstraction, data validation, program evaluation, state trauma registry development, and training.

Comprehensive Trauma Program

CONTACT

Phone
  •  540-448-2770
General Information
  • info@pomphreyconsulting.com
​Education Information
  • ​education@pomphreyconsulting.com

EDUCATION & TRAINING

SERVICES & RESOURCES

Courses and Webinars
Live Events and Conferences

Trauma Center Development
​Trauma Registrar Staffing
​FAQ
Registration by Invoice
Policies
​Accessibility
​Employee Log-In
© 2023 Pomphrey Consulting  |  All rights reserved  |  Website by Lingo Networks     
  • Education
    • Courses & Webinars
    • Registration by Invoice
  • Trauma Registrar Staffing
  • Trauma Center Development
  • Resources
    • Contact
    • Calculators
    • Trauma Library
    • Blog
    • Careers
  • Our Story
    • Company Overview >
      • Our Mission
      • Our Experience
      • Our Story
    • What People Are Saying
    • Why Partner with Us?
    • CEO Bio