1. Introduction

Industry standards are published for particular types of electromedical equipment by the International Electrotechnical Commission (IEC) and the International Organization for Standardization (ISO). These industry standards are updated on a regular basis by ISO/IEC workgroups. Work that is in progress by the Joint Workgroup 22 (JWG22) under the ISO/IEC 62D Electromedical Equipment Subcommittee will result in the publication of a new standard for ECG devices and systems with the designation of ISO/IEC 80601-2-86, which will be entitled “80601, Part 2-86: Particular requirements for the basic safety and essential performance of electrocardiographs, including diagnostic equipment, monitoring equipment, ambulatory equipment, electrodes, cables, and leadwires” ^[1]. JWG22 is a joint workgroup formed between the maintenance team that oversees the ECG particular standards and liaisons from other standard workgroups. This new standard is currently in draft form and constitutes a significant overhaul of current ECG equipment standards and, in effect, combines the three current ECG particular standards published by the IEC for diagnostic electrocardiographs ^[2], ECG patient monitors ^[3], and ambulatory ECG equipment ^[4]. The standard will additionally incorporate three ECG-related standards published by the Association for the Advancement of Medical Instrumentation (AAMI), which is the national standard development organization in the United States for health technology. The three additional AAMI standards will have safety and performance requirements for disposable electrodes (AAMI EC12 ^[5]), ECG cables and leadwires (AAMI EC53 ^[6]), and arrhythmia analysis performance reporting (AAMI EC57 ^[7]). Finally, 80601-2-86 will restore requirements that were omitted from a previously deprecated IEC diagnostic ECG particular standard that addressed the performance of computerized ECG analysis ^[8]. The enormous effort required for the development of the new 80601-2-86 standard represents a formidable task with far reaching implications, such that a comprehensive discussion of changes is beyond the scope of this paper. Therefore, the intention of this paper is narrowed in focus to give the reader a cursory level of understanding of the work in progress and a more detailed discussion about the impact it will have, specifically on performance requirements for computerized diagnostic ECG analysis algorithms, which is also commonly referred to by other terms, such as automated ECG interpretation or computerized ECG interpretation.

2. Impact of 801601-2-86 on Automated ECG Interpretation

When the new 80601-2-86 standard is introduced, ECG algorithm testing requirements, testing methods and data sets will be applied based on the intended use of the algorithm and not just the type of ECG device, which contains the algorithm. The requirements in the existing draft are structured with two different clauses, namely 201.12.4.1 Algorithm testing for Diagnostic 12 Lead and 201.12.4.2 requirements for testing computerized arrhythmia analysis algorithms ^[1]. Requirements in each of these clauses are applied to the specific types of ECG equipment for which they were originally defined in historical standards. In addition, these requirements are also applied to other computerized ECG analysis algorithms based on the intended use of the computerized ECG analysis output rather than the definition of the equipment itself.

There is some overlap between the outputs of these two types of ECG analysis algorithms, but they have different intended uses, and, therefore, the requirements, testing methods, and testing data sets are different for each of these two types of algorithms. The following discussion will focus on the impact of 80601-2-86 on performance testing for diagnostic 12 lead ECG analysis algorithms, which are also referred to by other descriptions, such as “automated ECG interpretation”. The statistical metrics, limitations of testing and underlying principles for automated ECG interpretation also apply to arrhythmia analysis algorithms as well but will not be discussed in this paper.

There are strong data to support the proposition that computerized ECG interpretation programs provide an important clinical adjunct to the physician that may even enhance physician overreading ^[9][10], but it is also clearly understood that the outputs of all computerized ECG interpretation algorithms have limitations ^[11] and require physician overread ^[12][13]. The historical requirements, testing methods and testing data sets have changed little over the years. Methods for measuring automated ECG interpretation have been consistently applied over the years by current ^[2] and past ^[8] industry standards. However, the data used for testing can heavily influence the measurement of accuracy and, at this time, there are no new additional databases that are appropriate for use as an industry standard, although some new efforts are ongoing ^[14]. Consequently, little progress has been made in improving the current quality of performance testing for algorithms in 80601-2-86. The work required to create better reference data sets for algorithm testing is particularly daunting and the improvements that can be made to the current performance testing are limited until better data sets are available for use within the context of an industry standard.

Developers will continue to improve the accuracy of automated ECG interpretation programs and individual manufacturers will continue to validate algorithm performance with private data sets. Furthermore, the emerging use of machine learning and artificial intelligence algorithms for ECG interpretation will add new complexities to the problem of understanding and characterizing algorithm safety and performance. This is also challenging regulatory agencies to expand their considerations for algorithm development and validation to address these new complexities ^[15]. Nevertheless, because of the ubiquitous presence of automated ECG interpretation software and because of the impact it can have on clinical diagnosis and decision making, it remains critical for the performance evaluation of algorithms to be a compulsory element of industry standards for ECG equipment.

3. Current Insights

At the time of preparing this paper, the first committee draft of 80601-2-86 had been published and circulated for comments by national standard organizations members of the IEC JWG22. The second committee draft is in preparation for circulation to the national committees for a second call for comments. The current state of 80601-2-86 combines several existing standards that apply to ECG equipment into a single standard that will include all ECG equipment within its scope and will also contain specific requirements for particular types of ECG equipment based on intended use claimed by the manufacturer. This will include requirements and conformance testing methods for computerized ECG analysis algorithms, which are defined in two broad categories, namely diagnostic 12 lead ECG interpretative algorithms and arrhythmia analysis algorithms. The quantification of performance and testing data sets have been in existence for decades. The goal of the new 80601-2-86 standard is to update the rationales and guidance contained in the informative annexes in such a way that it is more clearly understood how to apply the standard to the range of contemporary computerized ECG analysis algorithms based on the intended use of the ECG equipment in which they are used.

In particular, the requirements for measurement and analysis algorithms for diagnostic ECG interpretation restore some historical performance testing requirements and conformance testing methods that had been previously deprecated from current standards. Although the limitations of the conformance testing data sets have been well recognized and published, they still provide the only method of uniformly and consistently benchmarking algorithm performance. This is especially important because of the ubiquitous use of automated ECG interpretation by the clinical community and the important influence it can have on physician over reading.

Furthermore, the profound influence that automated ECG interpretation programs can have on physician ECG interpretation and clinical decision making has been well published by experts in electrocardiography and the importance of developing and evaluating the performance of these algorithms with scientific rigor is critical to ensuring that the appropriate use of computerized ECG interpretation programs is well understood and benefits patient care.

While the 80601-2-86 standard applies to the vast majority of ECG devices, the requirements for automated ECG analysis and interpretation are mostly relevant for traditional device types (e.g., rule-based analysis of resting 12-lead ECG and traditional Holter ECGs). However, the same concepts can be applied to novel technologies (e.g., machine learning/AI-based algorithms, non-standard lead technology/lead configuration) by using additional datasets relevant for the device’s intended use. Manufacturers should pay particular attention to factors that impact the quality and appearance of data sets for both algorithm development and testing, in particular, establishing appropriate sample sizes, patient population representation, and disease prevalence/representation to accurately reflect the clinical environment and intended use for which the algorithm is designed.

4. Summary

The introduction of 80601-2-86 is a significant overhaul of existing industry standards and will result in a single international standard that can be applied to all ECG equipment. The goals are to combine, update, and harmonize the safety and performance requirements from the multiple existing industry standards so that the new standard can be applied to all types of ECG and be appropriate for current ECG technology and clinical use.

Although the CSE and CTS test data sets have well known limitations, no other data sets have been accepted for inclusion in 80601-2-86. Manufacturers should continue to work together with clinical ECG experts to continue clinically meaningful improvements to computerized ECG analysis and should disclose the clinical validation of algorithm improvements to guide appropriate clinical use. More cooperation is needed between industry, clinical ECG experts and regulatory agencies to develop new data sets that can be made available for use by industry standards for algorithm performance evaluation.

This entry is adapted from the peer-reviewed paper 10.3390/hearts2030032