1. Please check and comment entries here.
Table of Contents

    Topic review

    Laboratory Demand Management Strategies

    (This entry belongs to Entry Collection "IT Support in the Healthcare Sector ")


    Inappropriate laboratory test selection in the form of overutilization as well as underutilization frequently occurs despite available guidelines. There is broad approval among laboratory specialists as well as clinicians that demand management strategies are useful tools to avoid this issue. Most of these tools, which may be adopted to local settings, are based on automated algorithms or other types of machine learning. We believe that artificial intelligence may help to further improve these available tools.

    1. Introduction

    Laboratory tests are fundamental for medical diagnosis, prognosis and treatment decisions [1] and are being ordered in rising numbers each year due to increased availability, mostly based on technological advances [2]. However, due to this fact that laboratory orders increase along with convenient availability, it seems that a certain amount of laboratory tests are ordered inappropriately [3][4]. On the one hand, inappropriate orders may present as overutilization, where tests with doubtful contribution to further patient management are ordered; on the other hand, there may be underutilization, when required tests are not being ordered [5]. In a systematic review, Zhi et al. [5] estimated an overall mean rate of overutilization of 20.6%. Subgroup analysis revealed a higher mean rate, around 44%, for inappropriate initial testing. However, single studies state that up to 70% of ordered tests may be of doubtful importance for patient management [6][7]. A workup of closed malpractice claims conducted by Gandhi as well as Kachalia et al. [8][9] revealed that failure to order the appropriate diagnostic or laboratory test contributed to missed or delayed diagnoses in 55% and 58% of cases in an ambulatory setting and the emergency department, respectively. Zhi et al. [5] state the overall mean rate of underutilization is 44.8%.
    Along with Sarkar et al. [10], who support the high proportions of errors in test selection by evaluating orders for coagulation disorders in real time, inappropriate ordering may be considered as a substantial threat to patient safety. Overutilization may lead to unnecessary follow-up investigations or treatments, increased workload and costs as well as patient anxiety, while underutilization may result in missed or delayed diagnoses [5][11][12]. Lack of knowledge, insecurity, pure habit, patient pressure or fear of lawsuits are possible causes for inappropriate testing [13][14][15]. The lack of knowledge is reflected by various studies, which observed inappropriate orders despite available guidelines or recommendations before the implementation of demand management (DM) tools [12][14][16][17][18].
    We summarize available DM strategies, which may be implemented into local settings to reduce inappropriate test utilization.

    2. Possible Strategies to Avoid Inappropriate Test Utilization

    DM tools may help to prevent overutilization and/or underutilization. Many studies combine several tools [14][17][19], which has been shown to have an additive effect on the overall outcome [20]. In addition, the collaboration of laboratory specialists and clinicians together with audits, feedbacks, reminders and multiple plan-do-study-act (PDSA) cycles will further improve efficiency in terms of a continuous improvement process [12][14][18][19]

    2.1 Alerts at the Stage of Order Entry

    Alerts appearing in the form of pop-up windows in the clinical physician order entry (CPOE) system may be designed to avoid various causes of overutilization [16][21] or to suggest an alternative test [22]. Minimum retesting intervals (MRIs), which may also be implemented in form of alerts at the stage of order entry, are discussed in section 2.3.

    2.2 Hold Back Orders in the Laboratory Information System (LIS)

    Informing the ordering provider through alerts at the stage of order entry would be the preferred solution; however, it may not always be possible to reject inappropriate orders in the CPOE system due to technical issues. In these cases, orders may be screened for appropriateness upon arrival in the LIS [4][23]. MRIs, which may also be considered as a subset of holding back orders, are discussed in the following section.

    2.3 Minimum Retesting Intervals

    MRIs are defined as “the minimum time before a test should be repeated, based on the properties of the test and the clinical situation in which it is used” [24]. Recommendations for MRIs are freely available, for example, from the collaboration of the Royal College of Pathologists, the Association for Clinical Biochemistry and Laboratory Medicine and the Institute of Biomedical Science [24]. MRIs may be implemented in the LIS, dependent on available technical possibilities [25][26]. One drawback of rejecting tests in the LIS is that unnecessary blood collections may be performed for cancelled tests. Therefore, it would be favorable if the requesting physician is at least alerted in the course of order entry [27][28]. Preferably the ordering physician is alerted at the stage of order entry  along with the choice to cancel the request or to continue with the order [21][29][30].

    Different outcomes are reported with regard to the reactions to the alert [21][29][30].

    2.4 Revision of Laboratory Ordering Forms and Profiles

    The position where tests are placed in the order entry system may affect the number of placed orders [3]. Furthermore, laboratory ordering profiles (LOPs), which are used to order a bundle of defined analytes with one click in the CPOE system, seem to be a source of overutilization; number of orders drops after removing tests from such LOPs [4][31]. One study describes the implementation of a panel for C-reactive protein (CRP) and erythrocyte sedimentation rate (ESR) testing [14], while others focus on LOPs for specific indications or diagnoses [12][32].

    2.5 Removal of Outdated Tests

    Apart from giving an alert for inappropriate orders, tests may also be entirely removed from the order entry system [4].  

    2.6 Display Costs

    Some studies evaluated the effect of displaying costs during the order entry process. Overall, the interventional impact was rated as “modest” by the authors [33][34][35]. Investigation of appropriateness of test selection was not part of the study designs. However, costs should never be the sole decision criterion for laboratory test ordering, reductions in expenditure may be also achieved by implementing DM strategies, which combat overutilization [14][16][17][19][21][25][27][28][29][30].

    2.7 Adding Tests

    Adding tests may be one attempt to prevent delayed or missed diagnoses [36].

    2.8 Reflex and Reflective Testing

    Another possibility of adding tests is through reflex or reflective testing. While reflex testing refers to the automated addition of tests according to a fixed algorithm within the LIS, reflective testing is the approach of adding tests and/or comments after the laboratory specialist has interpreted the results in synopsis with available clinical information [37]. For example, reflex testing may be suitable for the stepwise analysis of thyroid hormones, where thyroid-stimulating hormone (TSH) is the initial test, and subsequent analysis of free thyroid hormones should only be performed in the case of abnormal TSH results [18][19][38]. Reflex as well as reflective approaches may also be combined [39].

    In general, reflective testing is appreciated by physicians as well as patients and may have a significant positive outcome for patient management [40][41][42].

    2.9 Algorithms

    Algorithms are an advancement of reflex und reflective testing; several concatenated if/then queries are addressed, until a diagnostic decision is possible [43]. One practical example of such an algorithm is the PTT Advisor, a mobile application that helps to choose the appropriate follow-up tests in patients with a prolonged partial thromboplastin time (PTT) and normal prothrombin time [44]. However, an evaluation of apps with regard to the impact on test ordering would be meaningful [45]. Another way to implement diagnostic algorithms would be to program the according if/then cycles directly into the LIS [46].

    2.10 Education

    Educational interventions may be implemented in different ways. As sole method in the form of a workshop [15], as the first step of a two-stage process [19][31] or supplementary to IT-based solutions [12][14][17]. However, education as a sole method seems to be inferior compared to automated solutions [19][26][31]. IT-based solutions may also serve as a learning tool [21][27][45], although this could not be confirmed by others [26].

    3. Discussion and Conclusion

    There is broad approval that laboratory DM approaches are useful for appropriate test utilization, and several tools are already in use [47]. However, there are still a number of challenges. Due to different outcome criteria and settings, results may not be generalizable or comparable, which is why DM approaches have to be adapted to local settings. Therefore, harmonization strategies would be desirable, but currently they are not coordinated [48]
    Another challenge is that inappropriate orders remain [14]. One possibility for achieving appropriate test selection may be to conduct a health technology assessment prior to test implementation [49]
    However, these evidence-based assessments and further recommendations proposed for successful implementation, like the selection of quality indicators for monitoring and improvement as well as the ensuring of regular updates, are time-consuming [50]. We believe that artificial intelligence (AI) solutions are the next logical step, aiding in the development as well as improvement of DM strategies, as they could help to manage large data sets. Currently, few published articles deal with the issue of applying AI algorithms to laboratory test selection [51][52][53]. Machine learning (ML) models may also be used to identify prognostic factors [54]. The MRIs mentioned above are implemented as pre-defined alerts, and various alert ignorance rates are discussed in Section 2.3. Concerning this challenge, logistic regression models may be used to predict whether alerts will be accepted or overruled [55]. However, not all questions can be solved with AI. For example, using serum tumor markers alone for cancer screening may currently not be recommended even if data were retrospectively evaluated using various ML models [56].
    Furthermore, it has to be acknowledged that AI is only a tool of assistance [52]. A combination of computerized and physician-guided processes may be better than each one on their own [57]. Therefore, AI solutions may complement the recommended collaborations with clinicians for successful implementation [50]. Intensifying collaborations should be a feasible task, since a survey indicates that interest from both professions exists [47]. An advantage of complementary AI solutions would be that these systems, fed with unfiltered patient data, are capable of finding completely new diagnostic strategies that humans have not yet thought of - e.g. the prediction of the two-day mortality of thrombocytopenic patients on the basis of hematological tests only [58].
    In conclusion, the implementation of DM tools of laboratory specialists in collaboration with clinicians is increasing, and the incorporation of AI solutions is emerging in recent years. We believe that these solutions will help us to overcome technical barriers, a lack of harmonization and other challenges.

    The entry is from 10.3390/diagnostics11071141


    1. Whiting, P.; Toerien, M.; de Salis, I.; Sterne, J.A.; Dieppe, P.; Egger, M.; Fahey, T. A review identifies and classifies reasons for ordering diagnostic tests. J. Clin. Epidemiol. 2007, 60, 981–989.
    2. Fryer, A.A.; Hanna, F.W. Managing demand for pathology tests: Financial imperative or duty of care? Ann. Clin. Biochem 2009, 46, 435–437.
    3. Blumberg, G.; Kitai, E.; Vinker, S.; Golan-Cohen, A. Changing electronic formats is associated with changes in number of laboratory tests ordered. Am. J. Manag. Care 2019, 25, e179–e181.
    4. Mrazek, C.; Simundic, A.M.; Salinas, M.; von Meyer, A.; Cornes, M.; Bauçà, J.M.; Nybo, M.; Lippi, G.; Haschke-Becher, E.; Keppel, M.H.; et al. Inappropriate use of laboratory tests: How availability triggers demand—Examples across Europe. Clin. Chim. Acta 2020, 505, 100–107.
    5. Zhi, M.; Ding, E.L.; Theisen-Toupal, J.; Whelan, J.; Arnaout, R. The landscape of inappropriate laboratory testing: A 15-year meta-analysis. PLoS ONE 2013, 8, e78962.
    6. Cadamuro, J.; Gaksch, M.; Wiedemann, H.; Lippi, G.; von Meyer, A.; Pertersmann, A.; Auer, S.; Mrazek, C.; Kipman, U.; Felder, T.K.; et al. Are laboratory tests always needed? Frequency and causes of laboratory overuse in a hospital setting. Clin. Biochem. 2018, 54, 85–91.
    7. Miyakis, S.; Karamanof, G.; Liontos, M.; Mountokalakis, T.D. Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad. Med. J. 2006, 82, 823–829.
    8. Gandhi, T.K.; Kachalia, A.; Thomas, E.J.; Puopolo, A.L.; Yoon, C.; Brennan, T.A.; Studdert, D.M. Missed and delayed diagnoses in the ambulatory setting: A study of closed malpractice claims. Ann. Intern. Med. 2006, 145, 488–496.
    9. Kachalia, A.; Gandhi, T.K.; Puopolo, A.L.; Yoon, C.; Thomas, E.J.; Griffey, R.; Brennan, T.A.; Studdert, D.M. Missed and delayed diagnoses in the emergency department: A study of closed malpractice claims from 4 liability insurers. Ann. Emerg. Med. 2007, 49, 196–205.
    10. Sarkar, M.K.; Botz, C.M.; Laposata, M. An assessment of overutilization and underutilization of laboratory tests by expert physicians in the evaluation of patients for bleeding and thrombotic disorders in clinical context and in real time. Diagnosis 2017, 4, 21–26.
    11. Cornes, M. Case report of unexplained hypocalcaemia in a slightly haemolysed sample. Biochem. Med. 2017, 27, 426–429.
    12. Whiting, D.; Croker, R.; Watson, J.; Brogan, A.; Walker, A.J.; Lewis, T. Optimising laboratory monitoring of chronic conditions in primary care: A quality improvement framework. BMJ Open Qual. 2019, 8, e000349.
    13. Vrijsen, B.E.L.; Naaktgeboren, C.A.; Vos, L.M.; van Solinge, W.W.; Kaasjager, H.A.H.; Ten Berg, M.J. Inappropriate laboratory testing in internal medicine inpatients: Prevalence, causes and interventions. Ann. Med. Surg. 2020, 51, 48–53.
    14. Bartlett, K.J.; Vo, A.P.; Rueckert, J.; Wojewoda, C.; Steckel, E.H.; Stinnett-Donnelly, J.; Repp, A.B. Promoting appropriate utilisation of laboratory tests for inflammation at an academic medical centre. BMJ Open Qual. 2020, 9, e000788.
    15. Morgan, S.; Morgan, A.; Kerr, R.; Tapley, A.; Magin, P. Test ordering by GP trainees: Effects of an educational intervention on attitudes and intended practice. Can. Fam. Physician 2016, 62, 733–741.
    16. Juskewitch, J.E.; Norgan, A.P.; Johnson, R.D.; Trivedi, V.A.; Hanson, C.A.; Block, D.R. Impact of an electronic decision support rule on ESR/CRP co-ordering rates in a community health system and projected impact in the tertiary care setting and a commercially insured population. Clin. Biochem. 2019, 66, 13–20.
    17. Larochelle, M.R.; Knight, A.M.; Pantle, H.; Riedel, S.; Trost, J.C. Reducing excess cardiac biomarker testing at an academic medical center. J. Gen. Intern. Med. 2014, 29, 1468–1474.
    18. Taher, J.; Beriault, D.R.; Yip, D.; Tahir, S.; Hicks, L.K.; Gilmour, J.A. Reducing free thyroid hormone testing through multiple Plan-Do-Study-Act cycles. Clin. Biochem. 2020, 81, 41–46.
    19. Gilmour, J.A.; Weisman, A.; Orlov, S.; Goldberg, R.J.; Goldberg, A.; Baranek, H.; Mukerji, G. Promoting resource steward-ship: Reducing inappropriate free thyroid hormone testing. J. Eval. Clin. Pract. 2017, 23, 670–675
    20. Mostofian, F.; Ruban, C.; Simunovic, N.; Bhandari, M. Changing physician behavior: What works? Am. J. Manag. Care 2015, 21, 75–84.
    21. Lippi, G.; Brambilla, M.; Bonelli, P.; Aloe, R.; Balestrino, A.; Nardelli, A.; Ceda, G.P.; Fabi, M. Effectiveness of a computer-ized alert system based on re-testing intervals for limiting the inappropriateness of laboratory test requests. Clin. Biochem. 2015, 48, 1174–1176.
    22. Parkhurst, E.; Calonico, E.; Noh, G. Medical Decision Support to Reduce Unwarranted Methylene Tetrahydrofolate Reduc-tase (MTHFR) Genetic Testing. J. Med. Syst. 2020, 44, 152.
    23. Cadamuro, J.; Mrazek, C.; Wiedemann, H.; Felder, T.K.; Oberkofler, H.; Haschke-Becher, E.; Lippi, G. Effectiveness of a La-boratory Gate-Keeping Strategy to Overcome Inappropriate Test Utilization for the Diagnosis of Heparin-Induced Throm-bocytopenia. Semin. Thromb. Hemost. 2017, 43, 645–648.
    24. Lang, T.; Croal, B. National Minimum Retesting Intervals in Pathology: A Final Report Detailing Consensus Recommendations for Minimum Retesting Intervals for Use in Pathology; Document G147, Version 1; The Royal College of Pathologists, The Association for Clinical Biochemistry and Laboratory Medicine; The Institute of Biomedical Science: London, UK, 2015; pp.1–59.
    25. Salinas, M.; López-Garrigós, M.; Flores, E.; Blasco, A.; Leiva-Salinas, C. on behalf of the PRIMLAB working group. Successful implementations of automated minimum re-test intervals to overcome ferritin over-requesting in a Spanish hospital la-boratory. Clin. Chem. Lab. Med. 2020, 58, e287–e289.
    26. Mrazek, C.; Stechemesser, L.; Haschke-Becher, E.; Hölzl, B.; Paulweber, B.; Keppel, M.H.; Simundic, A.M.; Oberkofler, H.; Felder, T.K.; Cadamuro, J. Reducing the probability of falsely elevated HbA1c results in diabetic patients by applying automated and educative HbA1c re-testing intervals. Clin. Biochem. 2020, 80, 14–18.
    27. Waldron, J.L.; Ford, C.; Dobie, D.; Danks, G.; Humphrey, R.; Rolli, A.; Gama, R. An automated minimum retest interval rejection rule reduces repeat CRP workload and expenditure, and influences clinician-requesting behaviour. J. Clin. Pathol. 2014, 67, 731–733.
    28. Riley, J.D.; Stanley, G.; Wyllie, R.; Burt, H.L.; Horwitz, S.B.; Cooper, D.D.; Procop, G.W. An Electronic Strategy for Eliminating Unnecessary Duplicate Genetic Testing. Am. J. Clin. Pathol. 2020, 153, 328–332.
    29. Lapić, I.; Rogić, D.; Fuček, M.; Galović, R. Effectiveness of minimum retesting intervals in managing repetitive laboratory testing: Experience from a Croatian university hospital. Biochem. Med. 2019, 29, 030705.
    30. Moyer, A.M.; Saenger, A.K.; Willrich, M.; Donato, L.J.; Baumann, N.A.; Block, D.R.; Botz, C.M.; Khan, M.A.; Jaffe, A.S.; Han-son, C.A.; et al. Implementation of Clinical Decision Support Rules to Reduce Repeat Measurement of Serum Ionized Cal-cium, Serum Magnesium, and N-Terminal Pro-B-Type Natriuretic Peptide in Intensive Care Unit Inpatients. Clin. Chem. 2016, 62, 824–830.
    31. Keppel, M.H.; Kolbitsch, T.; Hoppe, U.C.; Auer, S.; Felder, T.K.; Oberkofler, H.; Mrazek, C.; Haschke-Becher, E.; Cadamuro, J. The clinically effective use of cardiac markers by restructuring laboratory profiles at Cardiology wards. Clin. Chem. Lab. Med. 2020, 58, 1565–1571.
    32. Delvaux, N.; Piessens, V.; Burghgraeve, T.; Mamouris, P.; Vaes, B.; Stichele, R.V.; Cloetens, H.; Thomas, J.; Ramaekers, D.; Sutter, A.; et al. Clinical decision support improves the appropriateness of laboratory test ordering in primary care without increasing diagnostic error: The ELMO cluster randomized trial. Implement. Sci. 2020, 15, 100.
    33. Horn, D.M.; Koplan, K.E.; Senese, M.D.; Orav, E.J.; Sequist, T.D. The impact of cost displays on primary care physician la-boratory test ordering. J. Gen. Intern. Med. 2014, 29, 708–714.
    34. Feldman, L.S.; Shihab, H.M.; Thiemann, D.; Yeh, H.C.; Ardolino, M.; Mandell, S.; Brotman, D.J. Impact of providing fee data on laboratory test ordering: A controlled clinical trial. JAMA Intern. Med. 2013, 173, 903–908, doi:10.1001/jamainternmed.2013.232.
    35. Silvestri, M.T.; Xu, X.; Long, T.; Bongiovanni, T.; Bernstein, S.L.; Chaudhry, S.I.; Silvestri, J.I.; Stolar, M.; Greene, E.J.; Dziura, J.D.; et al. Impact of Cost Display on Ordering Patterns for Hospital Laboratory and Imaging Services. J. Gen. Intern. Med. 2018, 33, 1268–1275.
    36. Salinas, M.; López-Garrigós, M.; Pomares, F.; Lugo, J.; Asencio, A.; López-Penabad, L.; Dominguez, J.R.; Leiva-Salinas, C. Serum calcium (S-Ca), the forgotten test: Preliminary results of an appropriateness strategy to detect primary hyperpara-thyroidism (pHPT). Bone 2013, 56, 73–76.
    37. Verboeket-van de Venne, W.P.; Aakre, K.M.; Watine, J.; Oosterhuis, W.P. Reflective testing: Adding value to laboratory testing. Clin. Chem. Lab. Med. 2012, 50, 1249–1252.
    38. Gill, J.; Barakauskas, V.E.; Thomas, D.; Rodriguez-Capote, K.; Higgins, T.; Zhang, D.; VanSpronsen, A.; Babenko, O.; Mar-tindale, R.; Estey, M.P. Evaluation of thyroid test utilization through analysis of population-level data. Clin. Chem. Lab. Med. 2017, 55, 1898–1906.
    39. Elnenaei, M.; Minney, D.; Clarke, D.B.; Kumar-Misir, A.; Imran, S.A. Reflex and reflective testing strategies for early detec-tion of pituitary dysfunction. Clin. Biochem. 2018, 54, 78–84.
    40. Oosterhuis, W.P.; Verboeket-van de Venne, W.P.H.G.; van Deursen, C.T.B.M.; Stoffers, H.E.J.H.; van Acker, B.A.C.; Bossuyt, P.M.M. Reflective testing—A randomized controlled trial in primary care patients.
    41. Darby, D.; Kelly, A.M. Reflective testing--what do our service users think? Ann. Clin. Biochem. 2006, 43, 361–368.
    42. Paterson, S.G.; Robson, J.E.; McMahon, M.J.; Baxter, G.; Murphy, M.J.; Paterson, J.R. Reflective testing: What do patients think? Ann. Clin. Biochem. 2006, 43, 369–371.
    43. Hoffmann, G.; Aufenanger, J.; Fodinger, M.; Cadamuro, J.; von Eckardstein, A.; Kaeslin-Meyer, M.; Hofmann, W. Benefits and limitations of laboratory diagnostic pathways. Diagnosis 2014, 1, 269–276.
    44. Savel, T.G.; Lee, B.A.; Ledbetter, G.; Brown, S.; LaValley, D.; Taylor, J.; Thompson, P. PTT Advisor: A CDC-supported initia-tive to develop a mobile clinical laboratory decision support application for the iOS platform. Online J. Public Health Inform. 2013, 5, 215.
    45. Meyer, A.N.D.; Thompson, P.J.; Khanna, A.; Desai, S.; Mathews, B.K.; Yousef, E.; Kusnoor, A.V.; Singh, H. Evaluating a mo-bile application for improving clinical laboratory test ordering and diagnosis. J. Am. Med. Inform. Assoc. 2018, 25, 841–847.
    46. Furundarena, J.R.; Uranga, A.; González, C.; Martínez, B.; Iriondo, J.; Ondarra, L.; Arambarri, A.; San Vicente, R.; Sarasqueta, C.; Lombardi, C.; et al. Initial study of anaemia profile for primary care centres with automated laboratory algorithms re-duces the demand for ferritin, iron, transferrin, vitamin B12 and folate tests. J. Clin. Pathol. 2020, Epub ahead of print.
    47. Ibarz, M.; Cadamuro, J.; Sumarac, Z.; Guimaraes, J.T.; Kovalevskaya, S.; Nybo, M.; Cornes, M.P.; Vermeersch, P.; Simundic, A.M.; Lippi, G. Clinicians’ and laboratory medicine specialists’ views on laboratory demand management: A survey in nine European countries. Diagnosis 2021, 8, 111–119.
    48. Ceriotti, F.; Barhanovic, N.G.; Kostovska, I.; Kotaska, K.; Perich Alsina, M.C. Harmonisation of the laboratory testing process: Need for a coordinated approach. Clin. Chem. Lab. Med. 2016, 54, e361–e363.
    49. Landaas, E.J.; Eckel, A.M.; Wright, J.L.; Baird, G.S.; Hansen, R.N.; Sullivan, S.D. Application of Health Technology Assessment (HTA) to Evaluate New Laboratory Tests in a Health System: A Case Study of Bladder Cancer Testing. Acad. Pathol. 2020, 7, 2374289520968225.
    50. Cadamuro, J.; Ibarz, M.; Cornes, M.; Nybo, M.; Haschke-Becher, E.; von Meyer, A.; Lippi, G.; Simundic, A.M. Managing inappropriate utilization of laboratory resources. Diagnosis 2019, 6, 5–13.
    51. Islam, M.M.; Yang, H.C.; Poly, T.N.; Li, Y.J. Development of an Artificial Intelligence-Based Automated Recommendation System for Clinical Laboratory Tests: Retrospective Analysis of the National Health Insurance Database. JMIR Med. Inform. 2020, 8, e24163.
    52. Islam, M.M.; Poly, T.N.; Yang, H.C.; Li, Y.C. Deep into Laboratory: An Artificial Intelligence Approach to Recommend Laboratory Tests. Diagnostics 2021, 11, 990.
    53. Xu, S.; Hom, J.; Balasubramanian, S.; Schroeder, L.F.; Najafi, N.; Roy, S.; Chen, J.H. Prevalence and Predictability of Low-Yield Inpatient Laboratory Diagnostic Tests. JAMA Netw. Open 2019, 2, e1910967.
    54. Tseng, Y.J.; Wang, H.Y.; Lin, T.W.; Lu, J.J.; Hsieh, C.H.; Liao, C.T. Development of a Machine Learning Model for Survival Risk Stratification of Patients with Advanced Oral Cancer. JAMA Netw. Open 2020, 3, e2011768.
    55. Baron, J.M.; Huang, R.; McEvoy, D.; Dighe, A.S. Use of machine learning to predict clinical decision support compliance, reduce alert burden, and evaluate duplicate laboratory test ordering alerts. JAMIA Open 2021, 4, ooab006.
    56. Wang, H.Y.; Hsieh, C.H.; Wen, C.N.; Wen, Y.H.; Chen, C.H.; Lu, J.J. Cancers Screening in an Asymptomatic Population by Using Multiple Tumour Markers. PLoS ONE 2016, 11, e0158285.
    57. Wang, D.; Khosla, A.; Gargeya, R.; Irshad, H.; Beck, A.H. Deep Learning for Identifying Metastatic Breast Cancer. arXiv 2016, arXiv:1606.05718.
    58. Lien, F.; Wang, H.Y.; Lu, J.J.; Wen, Y.H.; Chiueh, T.S. Predicting 2-Day Mortality of Thrombocytopenic Patients Based on Clinical Laboratory Data Using Machine Learning. Med. Care 2021, 59, 245–250.