Data Point Inter-rater Reliability for Targeted Trauma Registrar Education Leads to Pristine Data1/22/2021
Inter-rater reliability (IRR) audits are a must for every trauma center. The American College of Surgeon (ACS) requires each trauma center to monitor data validity. This is considered a criterion deficiency (CD 15-10). They recommend to re-abstract 5 to 10 percent of your patient records. [i] Data validation simply means checking the accuracy and quality of your trauma registry data. You want to verify this source data before using it or exporting it to the National Trauma Data Bank (NTDB). Once exported to NTDB it is used for benchmarking. These risk -adjusted benchmarking reports are used to measure performance and outcomes for all trauma centers. This to is a criterion deficiency if not done. (CD 15-5)[ii] Consider for a moment the significance of these benchmarking reports and the reflection on your trauma centers performance. The consequences of substandard data are immense. Data quality or its overall utility rests solely on the trauma registrars’ instruction. Therefore, a data point driven IRR is multifaceted for the trauma program. Not only does it assure accurate benchmarking, but also accurate internal performance improvement, and research. Trauma registrar education is a key component for every trauma program. ACS recommends core education as they start and continuing education each year. This is paramount to the overall success of a data driven program. Data point IRR quickly identifies the specific area a trauma registrar is experiencing difficulty in verses the high-level chart-based IRR accuracy. Typical IRR audits focus on this chart level accuracy. In this approach the overall audit my yield an IRR of 96% for the chart on Patient A. The issue with this approach on the surface this appears acceptable. However, it does not identify or guide the reviewer to potential repetitive abstraction errors for key data elements. Data point IRR is focuses on just that. Pomphrey Consulting recommends data point driven IRR for optimal overall results. By utilizing this method, you are targeting the exact area of education needed. As areas are identified, provide educational opportunities for the data element, and remedy this misinterpretation or misconception of the data elements definition, or specific data source. How to use the data-point IRR strategy First, identify data elements which are most important to your trauma program, it’s performance improvement initiatives and research. Benchmarking data elements must also be factored in. Several key considerations are:
Second, during the data validation audit, calculate and track IRR individually for these select data elements. This is not a noteworthy extension of workload. ACS again recommends 5 to 10 percent re-abstraction, but they do not specify which data elements, in part or in its entirety. Our approach is simply a targeted approach for these key elements and tracking performance at this fundamental level. Third, utilizing these results clearly identify educational needs. We recommend considering any data-point IRR under 95% as an opportunity to improve. Education efforts can use specific cases to illustrate mistakes and clarify data definitions. In our experience, focusing on specific data points increases learner retention and increases registrar confidence. Utilizing Results to Update Internal Data Dictionary This data point driven exercise can also assist you in streamlining your internal data dictionary. This exercise increasing data quality across the board, and thus optimizes consensus among all trauma registrars. A prime example of this is “ED Admission Time”. Guided by staff IRR results you reveal each trauma registrar is utilizing a different location within the electronic medical record (EMR). Registrar #1 utilizes the ED Encounter Summary, Registrar #2 the ED Patient Care Timeline, and Registrar #3 the Pre-Hospital Arrival Time. While these values may be the same, the likelihood is small. While the times may be only minutes apart, this one data element affects the ED Dwell time calculation which may be an internal performance improvement initiative. Best practice is to substantiate each registrar utilize the same location. With this goal in mind, the internal data dictionary would be updated to mirror the National Trauma Data Standard Data Dictionary with an informational hierarchy. Data Source Hierarchy Guide
Data-point IRR can assist in identifying similar source issues and ensure your internal data dictionary is as precise as possible with clear hierarchy for each data element. This may sound meticulous, but the advantage is a dramatic increase in the confidence of your trauma data internally, as well as with your benchmarking reports. Benefits of this approach Tracking data driven IRR permits a better overall view of trauma registrar performance and data accuracy. As the foundation of an educational strategy, it can lead to steady increases in overall IRR. This will increase data reliability and build your confidence in TQIP benchmarking reports. In addition, this strategy can allow trauma programs to use their registrar education budget more effectively. AuthorMichelle Pomphrey, Pomphrey Consulting President, MLT, RN, CSTR Do you want to strengthen and streamline your data validation processes?Pomphrey Consulting uses proprietary software from Buck Data Validation Systems (BDVS) to automatically report data-point IRR. This system streamlines key aspects of data-point IRR, giving trauma program leaders powerful tools for improving data quality. BDVS can be customized to your program’s needs and internal data validation processes. To find out more, explore Data Management Services from Pomphrey Consulting. Comments are closed.
|
Archives
June 2021
Categories
All
|
RSS Feed
Comprehensive Trauma Program
CONTACT |
|