Integrating Telemedicine Training into the Graduate Medical Education Curriculum

The primary objective of this study was to examine current emergency medicine resident knowledge of telemedicine, expose residents to standardized telemedicine patients virtually, and analyze the effectiveness of telemedicine training on completing a successful encounter. emergency medicine residents first underwent a virtual standardized telemedicine encounter using the ZoomTM telecommunications application without prior training in telemedicine. Standardized patients were then queried on resident success during this untrained encounter using a survey with aspects of a successful telemedicine encounter. The following session involved a lecture by a telemedicine physician. After this intervention, the residents underwent a repeat virtual encounter, with standardized patients responding to the same questions on the fundamental aspects of a successful interview. Residents also underwent a post-survey on their experiences with the training. Statistically significant differences were noted on aspects of the encounter related to obtaining informed consent (0% vs. 61%, p = 0.00012), asking about privacy in the patient’s environment (6% vs. 87%, p <0.00001), verifying name and/or date of birth (29% vs. 94%, p =0.00014), explaining the telehealth process and expectations to the patient (0% vs. 94%, p <0.00001). No statistical significance was seen with: resident introducing themselves (94% vs. 100%, p =.31732), asking focused questions about medical condition (100% vs. 100% p = 1), closing the encounter by explaining care plan (94% vs. 94%, p = 1). Overall, emergency medicine residents had significant improvement on aspects of an encounter with a standardized patient that were unique to telemedicine after undergoing training from an expert in the field. Similar training could easily be utilized across the country in emergency medicine residencies. This training could prove to be essential in the future because of the global health crisis of the COVID-19 pandemic, but also for rural emergency patient care and remote educational programs.


INTRODUCTION
With the dawn of the COVID-19 pandemic and the need for enhanced social distancing measures, the use of telemedicine has grown significantly in the past year.The Centers for Medicare and Medicaid Services (CMS) made policy changes during the pandemic to allow expanded access to telehealth services.These regulatory changes included improving provider payment for telehealth visits, enabling patients to access telehealth services from home, and allowing providers to offer telehealth consultations to out-of-state patients. 1With broadened access to telemedicine during the pandemic, the Center for Disease Control and Prevention (CDC) reported a 154% increase in telehealth visits during the last week of March 2020 compared to the same week in March 2019. 2 As the application of telemedicine becomes more common and relevant, it is essential to educate trainees on how to deliver care through this platform.
While medical schools have started to integrate telemedicine training into their curricula, 3 there are few reports of telemedicine training in graduate medical educa-tion programs.A recent study found that only 1 out of 104 specialties (Child and Adolescent Psychiatry) listed telehealth in the competency-based milestones by the Accreditation Council for Graduate Medical Education (ACGME). 4n 2016, the American Medical Association (AMA) adopted a policy to encourage undergraduate and graduate medical education to incorporate telemedicine training as a core competency. 5Formal training in telemedicine is recommended because the virtual patient-physician interaction requires a different skill set compared to the traditional in-person interaction. 6,7Competencies that are specific to telemedicine encounters include, but are not limited to, the virtual patient-physician relationship, telehealth consent, HIPAA considerations, digital communication and "webside" etiquette, the virtual examination, and familiarity with video-conferencing platforms.
Tele-emergency medicine's importance is at the forefront in the rural areas of the United States, where limitations both in facilities and physicians can lead to delays in treatment for many throughout the country.A study conducted in 2014 revealed improved care coordination and shorter time to care for emergencies in rural areas when an emergency telemedicine physician was incorporated. 8espite this, and a growing number of emergency departments across the country adopting telemedicine services for psychiatry and neurology, 9,10  In this study, we describe an innovative telesimulation program for emergency medicine residents that utilized standardized patients to simulate telehealth visits.The primary objective of this study was to examine current emergency medicine resident knowledge of telemedicine and analyze the effectiveness of the telemedicine training on completing a successful encounter.We also aimed to analyze the residents' attitudes towards telemedicine training as a whole and how likely this workshop would have an impact on their future careers.

METHODS
The study was conducted at a 3-year EM residency program that matches 7 residents per year.The primary teaching site is a community level II trauma center with an ED volume above 75,000 patients per year.Seventeen emergency medicine residents first underwent a virtual session using standardized patients to complete a simulated telemedicine encounter without any prior training on telemedicine.The Zoom TM telecommunications application was used to simulate teleconferencing with the patient.The scenarios can be found in the appendix.Each encounter was a maximum of ten minutes.After each encounter, the standardized patients evaluated each resident with regards to competencies in telemedicine.One week after the initial training session, sixteen of those seventeen residents completed asynchronous learning with assigned readings on telemedicine and attended a virtual lecture given by an associate professor of medicine at the University of Central Florida who is also an experienced telemedicine physician.The lec-ture focused on the fundamentals of a successful telemedicine encounter, including but not limited to: obtaining telehealth consent, virtual physical examination, ensuring privacy through telemedicine.After this intervention, the sixteen residents underwent another virtual telemedicine encounter with standardized patients.Each resident rotated through four different scenarios.Each encounter was a maximum of ten minutes, followed by a ten minute debrief focusing on the fundamental elements of a telemedicine patient encounter.The standardized patients again rated the residents' ability to complete a successful telemedicine interview.The assessment questions were curated by the director of simulation and the experienced telemedicine physician (Table 1).The responses were scored as follows: "0" for "Not Done", "1" for "Done Partially", and "2" for "Done Completely".Residents also completed a post-workshop survey on their experiences with the training (Table 2).These responses were scored as "Strongly Disagree", "Somewhat Disagree", "Neither Agree nor Disagree", "Somewhat Agree" and "Strongly Agree".

RESULTS
Standardized patients evaluated 17 emergency residents before telemedicine training, and 16 of those 17 residents after telemedicine training with a 13 question survey focused on aspects of a successful telemedicine interview.Standardized patients scored these questions at "0" for "Not Done", "1" for "Done Partially", and "2" for "Done Completely".The average scores of each question seen in Table 1 are shown in Table 3, as well as the left-tailed Z score and p-value for each question.
Statistically significant increases were noted on the following scoring categories: verifies my name and location, obtains telehealth informed consent, asks about my privacy in my environment, speaks clearly and ensure that I can hear him/her, explains the telehealth examination process and sets expectations, centers him/herself in the center of the screen, and looks directly and the camera and explains if he/she needs to look away.A statistically significant decrease was seen only in: asks if I understand the plan and have any other questions.
Aspects of the encounter that did not have statistically significant differences on pre-and post-test surveys included: introducing him/herself, asks focused questions regarding my condition, pauses to allow me to answer questions, explains his/her assessment of my condition (ie.differentials, diagnosis), and closes the encounter by explaining the care plan (ie.prescriptions, follow-ups).
Fourteen residents responded to a survey after the posttraining sessions, with 92.8% of respondents stating that they "strongly agree" that the telemedicine training was helpful to their education.With regards to the question of "I understand how to do a virtual physical exam", only 28.6% of respondents stated "strongly agree".The remainder of the resident survey can be found in table 2.  Q12.Centers him/herself to be in the center of the screen.
Q13. Looks directly at the camera and explains if he/she needs to look away.

Question Response
I found this telemedicine workshop enjoyable.
I found this telemedicine workshop useful for my practice.
Extremely useful-71% Moderately useful-14% Very useful 14% The use of standardized patients contributed to the realism of the telemedicine simulation.Somewhat agree-7% Agree-21% Strongly agree-71% The use of a virtual platform (Zoom) contributed to the realism of the telemedicine simulation.Somewhat agree-14% Strongly agree-86%

Scenario: Anxiety
Summary Adult patient will complain of feeling anxious after losing his/her job.He/she has as history of anxiety and depression, but lost insurance and cannot get see his/her usual to get refills.The patient has also been quarantining at home due to COVID-19 and feels more anxious after watching the news and reading articles on social media.If asked questions about depression/suicidality, the patient will report feeling anxious and depressed, but deny suicidal ideations.The patient will ask for a refill on Xanax.The learner may or may not realize that controlled substances cannot be prescribed for telemedicine encounters, which will be addressed in the debrief.The protocols for patients when there is a safety concern (ie.suicidal risk, kidnapping, etc) will also be discussed during the debrief.

Virtual Exam
• Minimal physical examination

Summary
The patient is middle-aged with a history of asthma and complaints of difficulty breathing.The patient can speak English with some difficulty (English is his/her second language, Spanish his/her first language).If the learner believes that an interpreter is needed, he/she can suggest using a trained interpreter through a company or check the availability of a trained health care professional.If this is done, the facilitator (faculty) in the Zoom room will state he/ she is a trained health care professional who can speak Spanish and help translate if needed.If the learner does not ask for a trained health care professional, the patient will suggest using her son if needed.The facilitator will say he is the son of the patient and can translate if necessary.The patient will ultimately have a diagnosis of asthma exacerbation / bronchitis with wheezing and coughing.The patient will not be in any respiratory distress, but will have a faint wheeze (described by patient as an audible "whistle sound" while breathing).The pulse and respiratory rate should be examined by the physician with help from the patient and will be normal.There will be no retractions, no belly breathing, and normal work of breathing.The patient should be asked to ambulate around the room to check for dyspnea on exertion (he/she will be breathing normally after walking around).The patient does not need to be transported to the hospital, but should be given prescriptions for inhaler/nebs and steroids and follow-up with a primary care physician to should be planned.

Virtual Exam
• General: speaking full sentences, no respiratory distress Scenario: Rash Summary Patient will complain of a rash on the right inner thigh.The rash started 3 days ago and has increased in size.It initially started out looking like bug bite and was pruritic.The rash is now described as red, warm, slightly raised, not pruritic (mildly tender), but no drainage.No associated fever, chills.No known allergies.The patient will be using a video camera that cannot be moved from the desk, so he/she will have difficulty showing the rash.Given the location of the rash and the inability to move the patient's camera, the patient will attempt to show the rash and will ask the physician for help.The physician will be expected to help answer technical questions and help troubleshoot any technical difficulties.Ultimately, the patient will not be able to show the rash, so instead will show photos of the rash from the patient's cell phone.The diagnosis will be a simple cellulitis from an insect wound, requiring antibiotics.The patient should not be transferred to a hospital.

Virtual Exam
• Non-toxic, well-appearing • Unable to examine the lower extremity • Unable to visualize the rash on the video • Assessment of rash will be photos on patient's personal device that is held up to the video camera (one photo from 3 days ago that looks like insect bite; one photo taken during telemedicine rash that looks like cellulitis with no streaking or abscess) him/herself.Q2.Verifies my identity and location.Q3.Obtains telehealth informed consent.Q4.Asks about my privacy in my environment.Q5.Speaks clearly and ensures that I can hear him/her.Q6.Asks focused questions regarding my condition.Q7.Pauses to allow me to answer questions.Q8.Explains the telehealth examination process and sets expectations.Q9.Explains his/her assessment of my condition (ie.differentials, diagnosis).Q10.Closes the encounter by explaining the care plan (ie.prescriptions, follow-ups).Q11.Asks if I understand the plan and have any other questions.
emergency resident exposure to telemedicine training is limited.Few studies have been conducted on this training.

Table 1 . Standardized Patient Assessment of Resident Performance
Integrating Telemedicine Training into the Graduate Medical Education Curriculum

Table 3 . Average Pre-and Post-Scores for Standardized Patient Assessment of Residents
Residents uniformly felt the training was helpful to their education after completing the training.Participants did feel less confident with the ability to do a physical exam during the encounter, which could be ameliorated with more practice in this environment.Given that a large portion of residency programs across the country are utilizing teleconferencing for weekly didactic sessions, similar training could easily be utilized across the country in emergency medicine residencies.This training could prove to be essential in the future because of the global health crisis of the COVID-19 pandemic, but also has other emergency applications including rural emergency medicine, rapid triaging of patients and virtual medical education.SummaryTeenage patient will complain of right ankle pain since last night.He/she was playing volleyball last night and twisted his/her ankle.Patient took ibuprofen and applied ice night, which helped the pain.Patient called for telemedicine evaluation this morning because ankle still hurts, particularly with walking.The learner should recognize this is a minor and obtain consent to evaluate the patient.If learner asks to speak to parent/guardian, the facilitator (faculty) in the Zoom room will say he/she is the parent and gives consent.If the learner does not verify the patient's age and does not recognize the need for consent, the facilitator will discuss this in the debrief.Ultimately, the patient will not meet criteria for an x-ray (able to walk, no deformity, no point/bony tenderness).The learner should advise ace wrap, ice, and NSAIDs.
ETHICS APPROVAL AND CONSENT TO PARTICIPATEThis study was conducted in accordance with the Declaration of Helsinki and approved by the University of Central Florida Institutional Review Board (Study number 00001861).The requirement for written informed consent was waived as the obtained data was de-identified.Submitted: January 01, 2024 EST, Accepted: January 01, 2024 EST This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CCBY-4.0).View this license's legal deed at http://creativecommons.org/licenses/by/4.0 and legal code at http://creativecommons.org/licenses/by/4.0/legalcodefor more information.o Example of Consent (share screen of an example) • Did the resident realize the patient was a minor?If yes, did they obtain consent from the parent?(Facilitator = parent) o If the resident did not obtain parental consent, mention that for all minors, it is similar to in-person assessments.Parent/Guardian must give consent to evaluate and treat a minor.• Did the resident perform a virtual examination?o Check for swelling/deformity o Ask patient to palpate for any point tenderness o Assess gait / weight-bearing ability • Review how a telemedicine exam is performed: o Ask patient/family/caregiver to assist with exam o If there are limitations in completing an exam, refer patient to in-person assessment o Use of home devices (smart watch, blood pressure cuff, glucometer) • Did the resident use of clinical decision rules to help determine if patient requires further workup?o Discuss the use of clinical decision rules to help make decisions on work up since resources are limited on a virtual platform • Did the resident close the encounter by discussing a plan and asking if there are any other questions/concerns?o Ideal Plan: rest, ice, ace wrap, elevate; see PCP/ortho if pain does not improve within 1 week or if pain/swelling worsens Integrating Telemedicine Training into the Graduate Medical Education Curriculum Academic Medicine & Surgery Is it ok for them to be there?Document who else is present.•Did the resident explore suicidality?(Given history of depression and recent worsening anxiety) o If so, the patient will NOT be suicidal o If not, please discuss that it is important to check for safety • Discuss next steps if there are any safety concerns o Example: suicidal, homicidal, kidnapping, human trafficking, child abuse o Telemedicine organizations have protocols in place to help in these emergencies o Provider should remain on the videoconference with the patient; should continue to engage the patient o Provider should simultaneously text/email/message the company to deploy additional help to patient's Integrating Telemedicine Training into the Graduate Medical Education CurriculumAcademic Medicine & Surgery • Respiratory: normal work of breathing, no retractions, no belly breathing • Lower extremities: no edema, no unilateral leg swelling, no calf tenderness • Vitals: instruct pulse check (wrist/neck or iPhone health app or home BP monitor or home pulse Ox), check respiratory rate by having patient put hand on chest and counting breaths (chest/hand rise) • Assess for dyspnea on exertion: ask patient to walk around the room and repeat respiratory exam to look for work of breathing, distress, respiratory rate • Patient will complain of intermittently hearing "faint whistling sound" with breathing (wheezing), which will not be audible during virtual exam The patient can walk around the room to look for dyspnea on exertion o The patient can show neck muscles / abdomen to look for retractions o If the encounter is at a facility (ie Nursing home), the provider can ask a health care professional to listen to breath sounds with a stethoscope o Technology exists such as eStethoscopes (transmits breaths sounds to provider) • Did the resident acknowledge how to troubleshoot the potential language barrier?o Options for interpretive services include 3 rd party through the telemedicine company (ie Cyracom); health care professional who can 3-way call patient and provider; if no other options then can use family member o Document who was used for interpretation • Did the resident close the encounter by discussing a plan and asking if there are any other questions/concerns?o Ideal Plan: Prescribe inhaler/nebs/steroids; however it is also ok to recommend further evaluation by PCP or urgent care or ER given age and SOB • GU exam (if concerned about Fournier's) will be deferred by patient Critical Actions • Help troubleshoot position of patient/camera • Suggest other ways to examine the rash (ie.photos) • Assess patient's comfort with examining inner thigh (ie.sensitive area?) Debriefing Points • Please give general feedback on the following (brief): o Did the resident introduce him/herself?o Did the resident verify patient's identity?§ Name/DOB/Location o Did the resident verify the environment was private?• Did the resident center him/herself in the screen?• Did the resident look directly at the camera and explain if he/she needed to look away?• Did the resident help the patient troubleshoot technical difficulties?o Can suggest that the resident moves the camera (patient will not be able to); can suggest taking photos of the rash (patient will ultimately do this) o The resident should recognize when technology is a barrier to getting through an encounter.If this happens, provider should refer patient to in-person assessment • Did the resident ask if the rash is in a sensitive location?o Discuss that examination of sensitive areas requires consent/agreement from both patient and physician o Discuss the intimate exam (video on/off, use of towels, consent by both parties) • Did the resident close the encounter by discussing a plan and asking if there are any other questions/concerns?o Ideal Plan: Prescribe PO antibiotics via telemedicine Integrating Telemedicine Training into the Graduate Medical Education Curriculum Academic Medicine & Surgery