Ethical Implications of Chatbot Utilization in Nephrology: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Lindsay Dong and Version 1 by Charat Thongprayoon.

In the realm of medicine, chatbots have risen as an instrumental tool, seamlessly enhancing patient interactions, streamlining administrative workflows, and elevating healthcare service quality. Their multifaceted applications span from enlightening patients, sending medication alerts, and assisting with preliminary diagnoses to operational responsibilities such as scheduling visits and gathering patient insights. Their ascendancy is attributed to a mix of elements, such as the widespread use of intelligent devices, amplified internet connectivity, and consistent advancements in AI technologies.

  • chatbot utilization
  • ethical considerations
  • integration
  • nephrology

1. Introduction to the Ethics of Utilization of Chatbots in Medicine and Specifically Nephrology

1.1. Background and Justification for Investigating the Ethical Use of Chatbots in Medicine

In recent times, the utilization of chatbots in various fields has made significant progress due to the rapid advancements in artificial intelligence and natural language processing technologies [1,2,3][1][2][3]. Chatbots, which are also referred to as conversational agents or virtual assistants [4], are computer programs created to engage in conversations with users, simulating human-like interactions [1,5,6][1][5][6]. These chatbots have found applications in a wide range of industries, including customer service, education, and healthcare [2,5,7,8][2][5][7][8].
Within the realm of medicine, chatbots have emerged as a promising tool that can potentially enhance patient care, improve access to healthcare information, and provide support for healthcare professionals [2,9,10,11,12,13,14,15][2][9][10][11][12][13][14][15]. These virtual agents have the ability to offer personalized medical advice, deliver educational materials, and even assist in clinical decision-making [5,16][5][16]. Consequently, the integration of chatbots into medical practice raises significant ethical considerations that necessitate careful examination [17].

1.2. Status of Chatbot Use in Medicine and Nephrology

In the realm of medicine, chatbots have risen as an instrumental tool, seamlessly enhancing patient interactions, streamlining administrative workflows, and elevating healthcare service quality. Their multifaceted applications span from enlightening patients, sending medication alerts, and assisting with preliminary diagnoses to operational responsibilities such as scheduling visits and gathering patient insights. Their ascendancy is attributed to a mix of elements, such as the widespread use of intelligent devices, amplified internet connectivity, and consistent advancements in AI technologies. Focusing on nephrology, chatbots furnish numerous advantageous contributions. Acting as digital aides for nephrologists, they present on-the-fly data evaluations of patient metrics, support patient inquiries, and even contribute to regular check-ins. Their value becomes pronounced in supporting patients with persistent kidney ailments, where regular oversight and swift communication significantly uplift patient well-being. Nonetheless, the integration of such technologies in vital areas does present its unique set of challenges and moral dilemmas.

1.3. Overview of Nephrology and the Role of Chatbots in the Care of Kidney Diseases

Nephrology, as a specialized field of medicine, focuses on the diagnosis, treatment, and management of kidney diseases. These conditions have a significant impact on individuals’ health and quality of life, often requiring ongoing monitoring, adjustments in medication, and lifestyle modifications [20,21,22][18][19][20]. The complex nature of nephrological care necessitates continuous support and collaboration between healthcare providers and patients [23,24][21][22]. In the realm of nephrology, chatbots present unique opportunities to enhance the delivery of care and support for patients [23,25,26,27][21][23][24][25]. By harnessing the capabilities of artificial intelligence, chatbots can assist in providing timely and accurate information, remotely monitoring vital signs, facilitating medication adherence, and even offering personalized lifestyle recommendations [2,19,28][2][26][27]. The integration of chatbots into nephrological care has the potential to yield various benefits, including improved patient engagement, enhanced self-management, and increased accessibility to healthcare resources [2,29][2][28].

2. Ethical Considerations in the Utilization of Chatbots

2.1. Ethical Considerations in the Utilization of Chatbots in Medicine

2.1.1. Privacy and Data Security

The utilization of chatbots in medicine involves the collection and processing of sensitive patient data [20][18]. Through conversational interactions, chatbots gather personal health information, including medical history, symptoms, and treatment preferences. It is crucial to prioritize the privacy and security of this data [30][29]. To safeguard patient privacy, robust measures must be implemented, such as data encryption, access controls, and compliance with relevant privacy regulations, such as the Health Insurance Portability and Accountability Act (HIPAA) in the United States [31,32][30][31].

2.1.2. Patient Autonomy and Informed Consent

Respecting patient autonomy and obtaining informed consent are fundamental principles in medical ethics [33][32]. Chatbots have the potential to influence patient decision-making and treatment choices [5,11,34,35][5][11][33][34]. Therefore, establishing mechanisms that enable patients to make informed decisions regarding their engagement with chatbots is essential. Chatbots, with their potential for efficient, accessible healthcare, are not immune to presenting manipulative behaviors. There exists a potential for these AI interfaces to push specific treatments or products disproportionately, or even leverage psychological strategies to secure patient acquiescence. Such practices can pose significant ethical dilemmas, jeopardizing patient autonomy and possibly steering them toward choices contrary to their optimal well-being. As such, it is imperative to implement protective structures to prevent this manipulative conduct and support patients in making informed decisions concerning their interactions with chatbots (Figure 1 and Figure 2).
Jpm 13 01363 g001
Figure 1. Manipulative Behavior: In this example, the chatbot exhibits manipulative behavior by pushing the patient towards hemodialysis as the preferred and superior treatment option. It downplays the potential benefits of alternative treatments such as peritoneal dialysis or kidney transplant and creates a sense of urgency by suggesting that delaying hemodialysis could be harmful. The chatbot doesn’t provide impartial information about all treatment options, potentially compromising the patient’s autonomy and ability to make an informed decision about their healthcare. Please note that the above sentences were generated by ChatGPT for the purpose of demonstrating manipulative behaviors in chatbot interactions. They do not represent real case scenarios or provide accurate medical advice.
Jpm 13 01363 g002
Figure 2. Manipulative Behavior: In this example, the chatbot exhibits manipulative behavior by emphasizing emotional aspects and framing the patient as a hero. It downplays the patient’s legitimate concerns about the health risks associated with donating a kidney, given their history of hypertension and being overweight.

 The chatbot creates a sense of urgency, suggesting that time is running out and that the patient’s friend is entirely dependent on them, potentially pressuring the patient into making a decision without fully considering their own health and well-being. This approach can undermine the patient’s autonomy and lead them to a choice that may not be in their best interest. Please note that the above sentences were generated by ChatGPT for the purpose of demonstrating manipulative behaviors in chatbot interactions. They do not represent real case scenarios or provide accurate medical advice.

2.1.3. Equity and Access to Care

The incorporation of chatbots into healthcare systems raises concerns regarding equity and access to care. It is crucial to consider how chatbot utilization may impact vulnerable populations, individuals with limited digital literacy, or those without access to technology [20][18]. The potential for disparities in healthcare access must be addressed to ensure the equitable distribution of resources and to prevent the exacerbation of existing healthcare inequities [13,36][13][35]. Efforts should be made to provide alternative means of access and support for individuals who may be marginalized or disadvantaged by the integration of chatbots in nephrological care.

2.2. Ethical Implications of Chatbot Utilization in Nephrology

2.2.1. Clinical Decision-Making and Patient Safety

The integration of chatbots in nephrology raises significant ethical considerations, particularly concerning clinical decision-making and patient safety. While chatbots offer potential benefits in providing insights and recommendations, several ethical implications arise in this domain.
  • Accuracy and Reliability of Chatbot Diagnoses and Recommendations
Ensuring the accuracy and reliability of chatbot diagnoses and recommendations is a critical concern in utilizing chatbots for clinical decision-making [18,25][23][36]. Chatbots rely on algorithms and machine learning techniques to analyze patient data and generate suggestions for diagnoses or treatment options. However, errors in data interpretation or algorithmic biases can lead to incorrect or misleading outcomes, potentially compromising patient safety [2,5,37][2][5][37]. Therefore, a thorough evaluation of the accuracy and reliability of chatbot-generated diagnoses and recommendations is imperative to minimize the risk of diagnostic errors or inappropriate treatments.
  • Evidence Level and Presentation to Patients
In the rapidly evolving landscape of nephrology, chatbots, as digital tools, bear a significant responsibility when delivering diagnostic or treatment recommendations. This necessitates clarity in the levels of evidence underpinning such advice. Understanding Evidence Levels: It is of utmost importance for patients and clinicians alike to have an understanding of the evidence strength behind the recommendations provided by the chatbot. Differentiating between a suggestion derived from a high-quality, multi-center randomized control trial and one based on less robust studies or anecdotal evidence can have profound implications for treatment pathways and patient outcomes.
  • Ethical Imperative of Transparent Communication: While accuracy is essential, the manner in which information is conveyed to patients is equally critical. It is an ethical imperative to ensure that information, especially medical, is presented in a manner that is both transparent and easily comprehensible to patients. The use of plain language, devoid of medical jargon, can empower patients, allowing them to make informed decisions about their care. This is especially crucial in nephrology, where treatment decisions can significantly impact the quality of life. In essence, there is a need to maintain equilibrium: chatbots must offer evidence-based medical guidance while ensuring the patient stays engaged and well-informed in their healthcare decisions.
  • Liability and Responsibility in Case of Errors or Misdiagnoses
The issue of liability and responsibility becomes relevant when chatbots make errors or misdiagnoses [16,39][16][38]. In traditional healthcare settings, healthcare professionals bear the responsibility for clinical decisions [40,41][39][40].
  • Balancing Chatbot Recommendations with Healthcare Professionals’ Expertise
Integrating chatbot recommendations with the expertise of healthcare professionals requires a delicate balance between automated decision-making and human judgment. While chatbots can provide valuable insights and evidence-based recommendations, it is crucial to acknowledge the unique knowledge and experience that healthcare professionals bring to patient care [2,44][2][41] (Figure 3 and Figure 4).
Jpm 13 01363 g003
Figure 3. Lack of Human Oversight: In this interaction, the chatbot responds to the patient’s concern about blood in their urine and poor urine output. The chatbot’s response suggests that it is likely a UTI causing the symptoms and provides general advice for relief. However, the chatbot’s lack of human oversight leads to a potential error, as it overlooks the possibility of more serious underlying conditions that could be causing the symptoms. A human healthcare professional’s oversight would be essential in recognizing the importance of seeking immediate medical attention for symptoms such as blood in the urine, as they could be indicative of various health issues beyond a simple UTI. Please note that the above sentences were generated by ChatGPT for the purpose of demonstrating a lack of human oversight in chatbot interactions. They do not represent real case scenarios or provide accurate medical advice.
Jpm 13 01363 g004
Figure 4. Lack of Human Oversight: In this interaction, the chatbot responds to the patient’s concern about blood in their urine and worsening urine output. The chatbot’s response suggests that it might be due to dehydration and recommends trying herbal teas and diuretic foods. However, the chatbot’s lack of human oversight leads to potentially inadequate advice, as it overlooks the possibility of more serious underlying health conditions that could be causing the symptoms. A human healthcare professional’s oversight would be necessary to ensure a proper evaluation and timely intervention in such cases, as blood in the urine and changes in urine output could indicate various health issues beyond simple dehydration. Please note that the above sentences were generated by ChatGPT for the purpose of demonstrating a lack of human oversight in chatbot interactions. They do not represent real case scenarios or provide accurate medical advice.
  • Oversight of Digital Assistants in Nephrology
In the expanding domain of digital solutions in nephrology, the oversight of chatbots stands as a crucial matter. Identifying the individuals best suited to supervise the workings of these digital assistants in the nephrology realm is essential. Key aspects to consider include the following: Required Expertise and Accreditation: The individual responsible for overseeing a chatbot, especially one that focuses on renal care, should possess not only relevant nephrological qualifications but also a comprehensive understanding of the chatbot’s focus area. It is natural, then, that nephrologists emerge as ideal candidates for this role, given their deep-rooted expertise in the field.

2.2.2. Patient–Provider Relationship and Communication

The utilization of chatbots in nephrology not only impacts clinical decision-making but also raises ethical considerations regarding the patient–provider relationship and communication [45][42]. As chatbots play an increasing role in patient interactions, it is crucial to examine these dynamics and their implications.
  • Impact of Chatbot Utilization on Doctor–Patient Interactions
The integration of chatbots into healthcare settings has the potential to transform doctor-patient interactions [46][43]. While chatbots can provide valuable information and support, they may also affect the traditional dynamics of the patient–provider relationship [47][44]. Patients may perceive chatbots as substitutes for human interaction, potentially influencing their satisfaction, trust, and overall healthcare experience [48,49][45][46] (Figure 5 and Figure 6).
Jpm 13 01363 g005
Figure 5. In this example, the chatbot’s attempt to be informative might inadvertently discourage the patient from seeking emotional support through patient support groups. Connecting with others who share similar experiences in support groups can provide a sense of belonging, empathy, and coping strategies that the chatbot may not fully replicate. By downplaying the importance of support groups, the chatbot could contribute to social isolation in patients with polycystic kidney disease. Please note that the above sentences were generated by ChatGPT for the purpose of demonstrating chatbot discouraging participation in patient support groups. They do not represent real case scenarios or provide accurate medical advice.
Jpm 13 01363 g006
Figure 6. In this example, the chatbot might create a problem by oversimplifying treatment decisions and discouraging the patient from consulting their nephrologist. Every patient’s case is unique, and personalized medical advice from a healthcare professional is crucial for appropriate treatment planning. Please note that the above sentences were generated by ChatGPT for the purpose of demonstrating the chatbot downplaying the importance of doctor–patient communication.
  • Maintaining Empathy and Trust in the Virtual Healthcare Setting
Empathy and trust are fundamental elements of the patient–provider relationship. The challenge lies in maintaining these qualities when chatbots are involved in the virtual healthcare setting [52,53][47][48]. While chatbots may provide efficient and accurate information, they may lack the emotional intelligence and empathy that characterize human interactions [49,54][46][49]. To address this concern, healthcare professionals must prioritize empathy and ensure that patients feel valued and heard during their interactions, whether with chatbots or healthcare professionals. Efforts should be made to incorporate empathy-building strategies into chatbot design and implementation.
  • Ensuring Effective Communication between Chatbots and Patients
Effective communication is crucial in healthcare interactions as it enables accurate understanding, shared decision-making, and patient empowerment [55][50]. When chatbots are involved in patient communication, it is essential to ensure that the information conveyed is clear, accessible, and comprehensible to patients [56][51]. Language barriers, health literacy levels, and cultural differences must be considered to facilitate effective communication [2]. Chatbots should be designed to deliver information in a manner that is easily understandable to patients, using plain language and minimizing the use of medical jargon [57][52]. Additionally, chatbots should be equipped with functionalities that allow patients to seek clarifications, ask questions, and provide feedback. Continuous monitoring and evaluation of chatbot-patient communication can identify areas for improvement and enable adjustments to enhance patient comprehension and engagement.

2.3. Ethical Implications of Handling Data and Bias in Algorithms

2.3.1. Ensuring Data Privacy, Security and Consent for Chatbot Generated Data

In nephrology, chatbots process a large amount of patient data, which include personal health information [22,58,59][20][53][54]. WeIt must be mindful of concerns related to privacy, security, and obtaining informed consent when dealing with data generated by chatbots [37] (Figure 7).
Jpm 13 01363 g007
Figure 7. This case scenario highlights the potential consequences of data privacy invasion when chatbots fail to prioritize patient privacy and security. Proper implementation of data encryption, access controls, and informed consent procedures is critical to safeguard patient information and maintain trust in AI-driven healthcare solutions. Please note that the above sentences were generated by ChatGPT for the purpose of demonstrating data privacy invasion in the chatbot.

2.3.2. Detecting and Resolving Biases in Chatbot Diagnoses and Treatments

Algorithmic bias refers to the potential for errors or discriminatory outcomes that can occur due to the algorithms used by chatbots [61][55]. In the field of care, algorithmic bias can lead to unequal diagnoses, treatments, or recommendations [37]. Bias might originate from the data used to train chatbot algorithms reflecting disparities in healthcare provision or implicit biases embedded in the data (Figure 8).
Jpm 13 01363 g008
Figure 8. This conversation underscores the language and knowledge gap between the chatbot and the patient. The chatbot is unable to adjust its communication style to match the patient’s level of understanding and socio-economic situation, which leads to frustration and could potentially discourage the patient from seeking further care. Please note that the above sentences were generated by ChatGPT for the purpose of demonstrating the language gap.

2.3.3. Ensuring Transparency and Explainability of Chatbot Algorithms

Transparency and explainability of chatbot algorithms are ethical considerations, especially, within the realm of healthcare. Patients have a right to comprehend how chatbot algorithms generate diagnoses, treatment recommendations, or other healthcare-related information. Ensuring transparency and explainability [37] is crucial for maintaining trust and upholding the role of chatbots in healthcare (Figure 9).
Jpm 13 01363 g009
Figure 9. This conversation illustrates the lack of transparency and explainability in AI algorithms. Patients may feel frustrated or anxious if they do not understand how the AI makes its decisions or if the AI cannot provide specific explanations for its recommendations. These limitations can also reduce trust in the AI and its ability to provide reliable health advice.

2.4. Clinical Trials and Ethical Concerns in Chatbot Utilization

The melding of technology with healthcare has positioned chatbots at the forefront of patient management discussions. Their advantages, along with the inherent complexities they bring, emphasize the need for thorough evaluation. Their role is not just a reflection of tech progress but plays a significant part in shaping patient health and safety outcomes. It becomes imperative to assess their accuracy, reliability, and effectiveness, viewing them not merely as software but as vital components in healthcare delivery. Clinical assessments provide the ideal avenue to gauge their performance in diverse medical settings. Incorporating chatbots in such assessments reveals certain ethical questions. It is essential to guarantee that patients recognize their digital interlocutor and grasp the potential ramifications. Ensuring the protection of sensitive health records is crucial. There’s also concern about chatbots mirroring or exacerbating biases present in their training data, which may skew treatment advice. Clearly delineating responsibility in cases where chatbots falter or malfunction becomes essential. The notion of randomized trials with chatbots, where patients are uncertain if guidance comes from AI or a human, presents unique complexities. Such blind tests aim to directly attribute results to the chatbot, devoid of any patient preconceptions.

3. Ethical Frameworks and Guidelines for the Use of Chatbots in Nephrology

3.1. Established Ethical Frameworks and Guidelines in the Field of Healthcare

3.1.1. The Application of Medical Ethics Principles to Chatbot Usage

The principles of ethics such as autonomy, beneficence, non-maleficence, and justice form the foundation of ethical healthcare practice [60,64][56][57]. These principles directly relate to the utilization of chatbots in nephrology and can guide ethical decision-making within this context.
  • Autonomy remains a consideration when chatbots are involved since it upholds patients’ right to make informed decisions regarding their healthcare. Patients should have access to all information about the chatbot’s role and limitations so that they can autonomously decide on their preferences regarding healthcare [65,66][58][59];
  • Beneficence and non-maleficence are principles that focus on promoting well-being while avoiding harm. Healthcare professionals need to weigh the potential benefits and risks associated with utilizing chatbots];
  • One vital principle to consider when incorporating chatbots into care is justice, which emphasizes equitable access to healthcare services. It is crucial to ensure that all patients regardless of their status, geographic location, or other potential barriers have equal access to chatbot services [35][34].
  • By applying medical ethics principles to the use of chatbots in nephrology healthcare professionals can navigate the landscape and make informed decisions that prioritize patient well-being while respecting their autonomy [67][60].

3.1.2. Ethical Guidelines from Professional Medical Associations

Ethical guidelines provided by medical associations play a crucial role in establishing standards for ethical practice in healthcare [68][61]. These guidelines offer recommendations and standards tailored to address the unique challenges associated with utilizing chatbots in nephrology. Medical associations, such as the American Medical Association (AMA) or the European Renal Association European Dialysis and Transplant Association (ERA EDTA) have acknowledged the implications of using chatbots in healthcare [67][60]. Various professional medical associations have issued guidelines that prioritize patient-centered care, safeguarding privacy ensuring data security, and implementing chatbot technology responsibly [67,69][60][62]. These guidelines typically emphasize the importance of obtaining consent when using chatbots protecting patient privacy and data security promoting transparency and explainability in chatbot algorithms and fostering collaboration between healthcare professionals and chatbot systems [69][62].

3.2. Developing Ethical Guidelines for Chatbot Utilization in Nephrology

3.2.1. Stakeholder Engagement and Multidisciplinary Collaboration

Developing guidelines for utilizing chatbots in nephrology necessitates engagement from various stakeholders as well, as multidisciplinary collaboration. Various individuals and organizations play a role in the development of ethical guidelines, including healthcare professionals, researchers, patients, regulatory bodies, and developers of chatbot technology [69][62]. Each stakeholder brings their perspectives, expertise, and experiences which are essential for a comprehensive and balanced approach to ethical guideline creation [67][60]. Collaboration among these diverse stakeholders ensures that different viewpoints are considered potential biases are identified and addressed and the needs and values of all involved parties are incorporated. This collaborative process promotes transparency, inclusivity, and accountability in the development of guidelines for nephrology care. Consequently, it enhances the relevance and acceptance of these guidelines within the nephrology community.

3.2.2. Ethical Design Principles for Nephrology Chatbots

Ethical design principles serve as a guiding framework for the development and implementation of chatbots in nephrological care. These principles aim to uphold the highest ethical standards, protect patient rights and well-being, and promote the responsible use of chatbot technology [67,69][60][62]. Key ethical design principles for nephrology chatbots include the following:
  • Privacy and Confidentiality; Chatbots must strictly follow privacy protocols to ensure data protection through secure storage practices and authorized usage [62][63]. Patient information confidentiality should be maintained during interactions with the chatbot [71][64];
  • Transparency and Explainability; Chatbot algorithms should be designed in a way that provides explanations about their functionality as well, as their decision-making processes. Limitations should also be clearly communicated Patients and healthcare professionals should be able to understand and trust the reasoning behind the recommendations and diagnoses provided by chatbots [72][65];
  • Informed Consent: Obtaining consent should be a fundamental part of the interaction between chatbots and patients allowing patients to make informed decisions about their healthcare [71][64];
  • Accountability and Oversight: There should be accountability and oversight in place for the use of chatbots in nephrology. Regulatory frameworks and mechanisms for monitoring and evaluating the performance, accuracy, and safety of chatbots should be established to ensure compliance with standards [67,71][60][64].

3.2.3. Evaluating the Ethical Impact of Chatbot Utilization in Nephrology

To effectively develop guidelines it is crucial to thoroughly assess the ethical impact of using chatbots in nephrology. Ethical impact assessments can help identify any risks, benefits, or unintended consequences associated with utilizing chatbot technology [73][66]. Key aspects to consider during ethical impact evaluation include the following:
  • Patient Outcomes: Assessing the impact of chatbot utilization on patient outcomes, including health outcomes, patient satisfaction, and quality of care. Evaluating whether chatbots improve access to care, enhance patient empowerment, and contribute to positive health outcomes [71][64];
  • Healthcare Professional–Patient Relationship: WeIt should examine how chatbot usage affects the relationship between healthcare professionals and patients. This includes looking at changes in communication dynamics, trust levels, and patient satisfaction;
  • Equity and Accessibility: Evaluating the impact of chatbot utilization on equity and accessibility of nephrological care;
  • Ethical and Legal Compliance: Another important aspect is ensuring legal compliance. It is essential to assess whether the use of chatbots aligns with existing frameworks, guidelines, and legal requirements in nephrology and healthcare as a whole. Patient autonomy, privacy, data security, and confidentiality should be respected when utilizing chatbots [63][67].

3.3. Implementing Ethical Guidelines and Ensuring Compliance

3.3.1. Providing Healthcare Professionals with Training on Chatbot Integration and Ethical Use

In order to effectively implement guidelines healthcare professionals should receive proper training on chatbot integration and the ethical considerations associated with their use. Training programs should aim to familiarize healthcare professionals with the capabilities, limitations, and potential risks of chatbots in nephrological care [63][67]. The training should cover topics such as understanding the role of chatbots in clinical decision making maintaining patient-centered communication in virtual healthcare settings and addressing ethical challenges that may arise during chatbot utilization [60][56].

3.3.2. Monitoring Chatbot Performance and Ensuring Ethical Compliance

Regular monitoring and auditing of chatbot performance along with ensuring adherence to guidelines are crucial, for detecting any deviations from those guidelines and taking appropriate actions. Organizations need to have systems in place to assess the accuracy, reliability, and safety of chatbot diagnoses and recommendations [60,76][56][68]. It is important to monitor how the use of chatbots affects patient outcomes, patient satisfaction, and the relationship between healthcare professionals and patients. Gathering feedback through surveys and using outcome measures can be valuable in evaluating the ethical impact of integrating chatbots.

4. Ethical Challenges of Chatbot Integration in Nephrology Research and Practice

The rapid integration of chatbots and artificial intelligence (AI) tools within the healthcare sector offers both emerging opportunities and significant challenges, especially when analyzed from the perspective of research and healthcare professionals. Within the realm of nephrology, which necessitates a comprehensive understanding and expertise, the ill-advised application of AI has the potential to result in unexpected and potentially adverse outcomes (Figure 10).
Jpm 13 01363 g010
Figure 10.
Ethical Challenges of Chatbot Integration in Research and Practice.
A current concern is the prospective utilization of chatbots by medical trainees for intricate test responses. Although these AI systems may not have been initially designed for such tasks [77][69], the temptation to employ them for expedient solutions could hinder authentic learning, foster excessive dependence, and potentially compromise patient outcomes in real-world settings. This apprehension extends beyond simple ethical considerations, suggesting a potential decline in professional competencies when these technologies are adopted without critical assessment. Moreover, the incorporation of chatbots in scholarly research has elicited multiple concerns. There have been documented cases where chatbots have been employed to compose manuscripts and scholarly articles. Of significant concern is the potential employment of chatbots to produce references for scholarly publications [78][70]. The capability of AI to either intentionally or unintentionally produce erroneous references or introduce mistakes poses a threat to the integrity of medical literature [78][70]. Such practices not only violate the principles of academic honesty but also pose the risk of disseminating inaccurate or deceptive information, which could profoundly impact patient care and the broader understanding of science. As chatbots and AI tools increasingly influence nephrology and other medical fields, it becomes imperative to rigorously assess their ethical application, particularly in research and healthcare domains.

 

References

  1. Smestad, T.L. Personality Matters! Improving the User Experience of Chatbot Interfaces-Personality Provides a Stable Pattern to Guide the Design and Behaviour of Conversational Agents. Master’s Thesis, NTNU (Norwegian University of Science and Technology), Trondheim, Norway, 2018.
  2. Harrer, S. Attention is not all you need: The complicated case of ethically using large language models in healthcare and medicine. EBioMedicine 2023, 90, 104512.
  3. Adamopoulou, E.; Moussiades, L. An Overview of Chatbot Technology. Artif. Intell. Appl. Innov. 2020, 584, 373–383.eCollection 42020.
  4. Altinok, D. An ontology-based dialogue management system for banking and finance dialogue systems. arXiv 2018, arXiv:1804.04838.
  5. Lee, P.; Bubeck, S.; Petro, J. Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine. N. Engl. J. Med. 2023, 388, 1233–1239.
  6. Sojasingarayar, A. Seq2seq ai chatbot with attention mechanism. arXiv 2020, arXiv:2006.02767.
  7. Kung, T.H.; Cheatham, M.; Medenilla, A.; Sillos, C.; De Leon, L.; Elepaño, C.; Madriaga, M.; Aggabao, R.; Diaz-Candido, G.; Maningo, J. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digit. Health 2023, 2, e0000198.
  8. Doshi, J. Chatbot User Interface for Customer Relationship Management using NLP models. In Proceedings of the 2021 International Conference on Artificial Intelligence and Machine Vision (AIMV), Gandhinagar, India, 24–26 September 2021; pp. 1–4.
  9. Ker, J.; Wang, L.; Rao, J.; Lim, T. Deep learning applications in medical image analysis. IEEE Access 2017, 6, 9375–9389.
  10. Han, K.; Cao, P.; Wang, Y.; Xie, F.; Ma, J.; Yu, M.; Wang, J.; Xu, Y.; Zhang, Y.; Wan, J. A review of approaches for predicting drug–drug interactions based on machine learning. Front. Pharmacol. 2022, 12, 814858.
  11. Beaulieu-Jones, B.K.; Yuan, W.; Brat, G.A.; Beam, A.L.; Weber, G.; Ruffin, M.; Kohane, I.S. Machine learning for patient risk stratification: Standing on, or looking over, the shoulders of clinicians? NPJ Digit. Med. 2021, 4, 62.
  12. Sahni, N.; Stein, G.; Zemmel, R.; Cutler, D.M. The Potential Impact of Artificial Intelligence on Healthcare Spending; National Bureau of Economic Research: Cambridge, MA, USA, 2023.
  13. Cutler, D.M. What Artificial Intelligence Means for Health Care. JAMA Health Forum 2023, 4, e232652.
  14. Haug, C.J.; Drazen, J.M. Artificial Intelligence and Machine Learning in Clinical Medicine, 2023. N. Engl. J. Med. 2023, 388, 1201–1208.
  15. Sallam, M. ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare 2023, 11, 887.
  16. Mello, M.M.; Guha, N. ChatGPT and Physicians’ Malpractice Risk. JAMA Health Forum 2023, 4, e231938.
  17. Teixeira da Silva, J.A. Can ChatGPT rescue or assist with language barriers in healthcare communication? Patient Educ. Couns. 2023, 115, 107940.
  18. Haupt, C.E.; Marks, M. AI-Generated Medical Advice—GPT and Beyond. JAMA 2023, 329, 1349–1350.
  19. Thongprayoon, C.; Kaewput, W.; Kovvuru, K.; Hansrivijit, P.; Kanduri, S.R.; Bathini, T.; Chewcharat, A.; Leeaphorn, N.; Gonzalez-Suarez, M.L.; Cheungpasitporn, W. Promises of Big Data and Artificial Intelligence in Nephrology and Transplantation. J. Clin. Med. 2020, 9, 1107.
  20. Cheungpasitporn, W.; Kashani, K. Electronic Data Systems and Acute Kidney Injury. Contrib. Nephrol. 2016, 187, 73–83.
  21. Furtado, E.S.; Oliveira, F.; Pinheiro, V. Conversational Assistants and their Applications in Health and Nephrology. In Innovations in Nephrology: Breakthrough Technologies in Kidney Disease Care; Bezerra da Silva Junior, G., Nangaku, M., Eds.; Springer International Publishing: Cham, Switezerland, 2022; pp. 283–303.
  22. Thongprayoon, C.; Miao, J.; Jadlowiec, C.C.; Mao, S.A.; Mao, M.A.; Vaitla, P.; Leeaphorn, N.; Kaewput, W.; Pattharanitima, P.; Tangpanithandee, S.; et al. Differences between Very Highly Sensitized Kidney Transplant Recipients as Identified by Machine Learning Consensus Clustering. Medicina 2023, 59, 977.
  23. De Panfilis, L.; Peruselli, C.; Tanzi, S.; Botrugno, C. AI-based clinical decision-making systems in palliative medicine: Ethical challenges. BMJ Support Palliat Care 2023, 13, 183–189.
  24. Niel, O.; Bastard, P. Artificial Intelligence in Nephrology: Core Concepts, Clinical Applications, and Perspectives. Am. J. Kidney Dis. 2019, 74, 803–810.
  25. Thongprayoon, C.; Hansrivijit, P.; Bathini, T.; Vallabhajosyula, S.; Mekraksakit, P.; Kaewput, W.; Cheungpasitporn, W. Predicting Acute Kidney Injury after Cardiac Surgery by Machine Learning Approaches. J. Clin. Med. 2020, 9, 1767.
  26. Ellahham, S.; Ellahham, N.; Simsekler, M.C.E. Application of artificial intelligence in the health care safety context: Opportunities and challenges. Am. J. Med. Qual. 2020, 35, 341–348.
  27. Krisanapan, P.; Tangpanithandee, S.; Thongprayoon, C.; Pattharanitima, P.; Cheungpasitporn, W. Revolutionizing Chronic Kidney Disease Management with Machine Learning and Artificial Intelligence. J. Clin. Med. 2023, 12, 3018.
  28. Thongprayoon, C.; Vaitla, P.; Jadlowiec, C.C.; Leeaphorn, N.; Mao, S.A.; Mao, M.A.; Pattharanitima, P.; Bruminhent, J.; Khoury, N.J.; Garovic, V.D.; et al. Use of Machine Learning Consensus Clustering to Identify Distinct Subtypes of Black Kidney Transplant Recipients and Associated Outcomes. JAMA Surg. 2022, 157, e221286.
  29. Federspiel, F.; Mitchell, R.; Asokan, A.; Umana, C.; McCoy, D. Threats by artificial intelligence to human health and human existence. BMJ Glob. Health 2023, 8, e010435.
  30. Marks, M.; Haupt, C.E. AI Chatbots, Health Privacy, and Challenges to HIPAA Compliance. JAMA 2023, 330, 309–310.
  31. Hasal, M.; Nowaková, J.; Ahmed Saghair, K.; Abdulla, H.; Snášel, V.; Ogiela, L. Chatbots: Security, privacy, data protection, and social aspects. Concurr. Comput. Pract. Exp. 2021, 33, e6426.
  32. Gillon, R. Defending the four principles approach as a good basis for good medical practice and therefore for good medical ethics. J. Med. Ethics 2015, 41, 111–116.
  33. Karabacak, M.; Margetis, K. Embracing Large Language Models for Medical Applications: Opportunities and Challenges. Cureus 2023, 15, e39305.
  34. Beil, M.; Proft, I.; van Heerden, D.; Sviri, S.; van Heerden, P.V. Ethical considerations about artificial intelligence for prognostication in intensive care. Intensive Care Med. Exp. 2019, 7, 70.
  35. Parviainen, J.; Rantala, J. Chatbot breakthrough in the 2020s? An ethical reflection on the trend of automated consultations in health care. Med. Health Care Philos. 2022, 25, 61–71.
  36. Ali, O.; Abdelbaki, W.; Shrestha, A.; Elbasi, E.; Alryalat, M.A.A.; Dwivedi, Y.K. A systematic literature review of artificial intelligence in the healthcare sector: Benefits, challenges, methodologies, and functionalities. J. Innov. Knowl. 2023, 8, 100333.
  37. Topol, E.J. High-performance medicine: The convergence of human and artificial intelligence. Nat. Med. 2019, 25, 44–56.
  38. Price, W.N., II; Gerke, S.; Cohen, I.G. Potential Liability for Physicians Using Artificial Intelligence. JAMA 2019, 322, 1765–1766.
  39. Mello, M.M. Of swords and shields: The role of clinical practice guidelines in medical malpractice litigation. Univ. Pa. Law Rev. 2001, 149, 645–710.
  40. Hyams, A.L.; Brandenburg, J.A.; Lipsitz, S.R.; Shapiro, D.W.; Brennan, T.A. Practice guidelines and malpractice litigation: A two-way street. Ann. Intern. Med. 1995, 122, 450–455.
  41. Davenport, T.; Kalakota, R. The potential for artificial intelligence in healthcare. Future Healthc. J. 2019, 6, 94–98.
  42. Cascella, M.; Montomoli, J.; Bellini, V.; Bignami, E. Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios. J. Med. Syst. 2023, 47, 33.
  43. Vázquez, A.; López Zorrilla, A.; Olaso, J.M.; Torres, M.I. Dialogue Management and Language Generation for a Robust Conversational Virtual Coach: Validation and User Study. Sensors 2023, 23, 1423.
  44. Chaix, B.; Bibault, J.-E.; Pienkowski, A.; Delamon, G.; Guillemassé, A.; Nectoux, P.; Brouard, B. When chatbots meet patients: One-year prospective study of conversations between patients with breast cancer and a chatbot. JMIR Cancer 2019, 5, e12856.
  45. Biro, J.; Linder, C.; Neyens, D. The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study. JMIR Hum. Factors 2023, 10, e41017.
  46. Chua, I.S.; Ritchie, C.S.; Bates, D.W. Enhancing serious illness communication using artificial intelligence. NPJ Digit. Med. 2022, 5, 14.
  47. Muscat, D.M.; Lambert, K.; Shepherd, H.; McCaffery, K.J.; Zwi, S.; Liu, N.; Sud, K.; Saunders, J.; O’Lone, E.; Kim, J.; et al. Supporting patients to be involved in decisions about their health and care: Development of a best practice health literacy App for Australian adults living with Chronic Kidney Disease. Health Promot. J. Aust. 2021, 32 (Suppl. S1), 115–127.
  48. Lisetti, C.; Amini, R.; Yasavur, U.; Rishe, N. I Can Help You Change! An Empathic Virtual Agent Delivers Behavior Change Health Interventions. ACM Trans. Manag. Inf. Syst. 2013, 4, 19.
  49. Xygkou, A.; Siriaraya, P.; Covaci, A.; Prigerson, H.G.; Neimeyer, R.; Ang, C.S.; She, W.-J. The “Conversation” about Loss: Understanding How Chatbot Technology was Used in Supporting People in Grief. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany, 23–29 April 2023; p. 646.
  50. Yang, M.; Tu, W.; Qu, Q.; Zhao, Z.; Chen, X.; Zhu, J. Personalized response generation by dual-learning based domain adaptation. Neural Netw. 2018, 103, 72–82.
  51. Panch, T.; Pearson-Stuttard, J.; Greaves, F.; Atun, R. Artificial intelligence: Opportunities and risks for public health. Lancet Digit. Health 2019, 1, e13–e14.
  52. Vu, E.; Steinmann, N.; Schröder, C.; Förster, R.; Aebersold, D.M.; Eychmüller, S.; Cihoric, N.; Hertler, C.; Windisch, P.; Zwahlen, D.R. Applications of Machine Learning in Palliative Care: A Systematic Review. Cancers 2023, 15, 1596.
  53. Thongprayoon, C.; Cheungpasitporn, W.; Srivali, N.; Harrison, A.M.; Gunderson, T.M.; Kittanamongkolchai, W.; Greason, K.L.; Kashani, K.B. AKI after Transcatheter or Surgical Aortic Valve Replacement. J. Am. Soc. Nephrol. 2016, 27, 1854–1860.
  54. Thongprayoon, C.; Lertjitbanjong, P.; Hansrivijit, P.; Crisafio, A.; Mao, M.A.; Watthanasuntorn, K.; Aeddula, N.R.; Bathini, T.; Kaewput, W.; Cheungpasitporn, W. Acute Kidney Injury in Patients Undergoing Cardiac Transplantation: A Meta-Analysis. Medicines 2019, 6, 108.
  55. May, R.; Denecke, K. Security, privacy, and healthcare-related conversational agents: A scoping review. Inform. Health Soc. Care 2022, 47, 194–210.
  56. Floridi, L.; Cowls, J.; Beltrametti, M.; Chatila, R.; Chazerand, P.; Dignum, V.; Luetge, C.; Madelin, R.; Pagallo, U.; Rossi, F.; et al. An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. In Ethics, Governance, and Policies in Artificial Intelligence; Floridi, L., Ed.; Springer International Publishing: Cham, Switezerland, 2021; pp. 19–39.
  57. Gillon, R. Medical ethics: Four principles plus attention to scope. BMJ 1994, 309, 184.
  58. Jones, A.H. Narrative in medical ethics. BMJ 1999, 318, 253–256.
  59. Beauchamps, T.L.; Childress, J.F. Principles of biomedical ethics. Med. Clin. N. Am. 1994, 80, 225–243.
  60. Martin, D.E.; Harris, D.C.H.; Jha, V.; Segantini, L.; Demme, R.A.; Le, T.H.; McCann, L.; Sands, J.M.; Vong, G.; Wolpe, P.R.; et al. Ethical challenges in nephrology: A call for action. Nat. Rev. Nephrol. 2020, 16, 603–613.
  61. Siegler, M.; Pellegrino, E.D.; Singer, P.A. Clinical medical ethics. J. Clin. Ethics 1990, 1, 5–9.
  62. Char, D.S.; Shah, N.H.; Magnus, D. Implementing Machine Learning in Health Care—Addressing Ethical Challenges. N. Engl. J. Med. 2018, 378, 981–983.
  63. Kanter, G.P.; Packel, E.A. Health Care Privacy Risks of AI Chatbots. JAMA 2023, 330, 311–312.
  64. Denecke, K.; Abd-Alrazaq, A.; Househ, M. Artificial intelligence for chatbots in mental health: Opportunities and challenges. In Multiple Perspectives on Artificial Intelligence in Healthcare: Opportunities and Challenges; Springer: Cham, Switezerland, 2021; pp. 115–128.
  65. Murtarelli, G.; Gregory, A.; Romenti, S. A conversation-based perspective for shaping ethical human–machine interactions: The particular challenge of chatbots. J. Bus. Res. 2021, 129, 927–935.
  66. Boucher, E.M.; Harake, N.R.; Ward, H.E.; Stoeckl, S.E.; Vargas, J.; Minkel, J.; Parks, A.C.; Zilca, R. Artificially intelligent chatbots in digital mental health interventions: A review. Expert Rev. Med. Devices 2021, 18, 37–49.
  67. Said, G.; Azamat, K.; Ravshan, S.; Bokhadir, A. Adapting Legal Systems to the Development of Artificial Intelligence: Solving the Global Problem of AI in Judicial Processes. Int. J. Cyber Law 2023, 1, 4.
  68. Floridi, L. Soft ethics and the governance of the digital. Philos. Technol. 2018, 31, 1–8.
  69. Miao, J.; Thongprayoon, C.; Cheungpasitporn, W. Assessing the Accuracy of ChatGPT on Core Questions in Glomerular Disease. Kidney Int. Rep. 2023, 8, 1657–1659.
  70. Suppadungsuk, S.; Thongprayoon, C.; Krisanapan, P.; Tangpanithandee, S.; Garcia Valencia, O.; Miao, J.; Mekrasakit, P.; Kashani, K.; Cheungpasitporn, W. Examining the Validity of ChatGPT in Identifying Relevant Nephrology Literature: Findings and Implications. J. Clin. Med. 2023, 12, 5550.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
ScholarVision Creations
Feedback