Introduction: Metaverse is the augmented virtual world formed by convergence of virtual and physical space. Users interact within this created world, meeting each other virtually, immersing themselves in performing virtual activities, which subsequently could lead to real experiences. Conventionally, the healthcare “industry” is conservative in deploying future ready technology. Aims and Objectives: This overview discusses the untapped potential of metaverse applications in healthcare from a clinician's perspective. Bereft of technical jargon, the article points out the advantages, disadvantages, limitations, and challenges in actual deployment of the metaverse in clinical practice in the real world. The exponential transformation occurring in this area is highlighted. The highly technical literature is simplified for easier comprehension. Findings: Clinical applications, use of the metaverse in training, education, and augmenting telehealth consultations, in an immersive milieu, is discussed. Direct “in-person” interaction with digital products and solutions will be a new experience for a healthcare provider and the beneficiary. The role of digital twins is illustrated. Consultation process and various clinical applications in the metaverse are outlined. Technology-enabled futuristic training and education is discussed. Conclusion: Demonstrating significant improvement in healthcare outcomes using the metaverse will be difficult to prove. This alone will ultimately lead to the development of a business model, insurance reimbursement and behavioral modification necessary for accepting and using, a hitherto unused method in patient care.
Keywords: Augmented reality and healthcare, blockchain and healthcare, metaverse and healthcare, virtual reality and healthcare
| Introduction|| |
The Greek word Meta (μετα) implies “with, among, after, beyond, that is transcending reality,” as in the term metaphysics. Augmented reality (AR)/virtual reality (VR), extended reality (XVR), mixed reality (MR), AR and XVR, internet of medical things, Web 3.0, I Cloud, Enhanced Data for Global Evolution, quantum and spatial computing, robotics, artificial intelligence (AI), 5G, and blockchain when deployed individually or collectively for one or more applications result in the metaverse. Metaverse is the interface where users are immersed in a make-believe world with visual, auditory, and even sensory inputs. Patients with a physical symptomatic ailment transacting in a virtual immersive space may initially appear outlandish. Users will need to pay a fee to access a portal in the healthcare metaverse. Participants would take their avatars, cryptocurrencies, and other information from one organization's metaverse to another. Adoption will require time and trust. The US Food and Drug Administration currently reviews VR/AR devices through its medical extended reality program (MERP). MERP would be the forerunner to set regulations for using the metaverse in healthcare. Protecting virtual patients from bad care, predatory schemes, false claims, and fraud is critical. Individuals will “experience” a reality without physical involvement. This could be an escape strategy avoiding facing real issues in real time. There are many ways in which the metaverse can transform and improve healthcare. Metaverse can be enlarged with Web 3 standards such as blockchain to facilitate ownership, online payments, and traceability. Legality of advocating treatments virtually needs to be addressed. Forestalling potential challenges could lead to realistic expectations. A bibliometric analysis of research trends in the Health Metaverse revealed 34,000 papers in 22 years from the Web of Science database.
| History|| |
The term “metaverse” was first used in a science fiction novel Snow Crash by Neal Stephenson. In 1999, The Matrix released in theaters featured characters unaware that they were part of a simulated reality world. In 2003, several VR video games were released. From 2011, novels appeared sporadically featuring the metaverse. In 2021, Facebook was renamed as META.
| Consultation Process|| |
One's digital identity will select specific virtual healthcare services. The digital twin/avatar will meet the virtual doctor. This 1:1 “photocopy” will have all health data generated from all diverse sources – electronic medical records, measurements from wearable devices, administrative data, consumer behavior, lifestyle, etc., Initially, a doctor visible in three-dimension (3D), having access to all personalized and easily accessible data, will interact with the beneficiary. Eventually, a self-generated 3D image of internal organs could be displayed with home devices. Appropriate hardware (computer, VR goggles, broadband) and software are necessary. AI algorithms without full data may commit errors. AI algorithms may not see the “full picture” like humans. A doctor visible in 3D will interact with the Avatar. “Examining a patient” may be replaced by “examining the data.” For less complex disorders, an algorithm knowing millions of similar clinical problems perhaps could be better than an actual doctor, with limited experience in a small community. [Figure 1] shows the author in a Simulation Laboratory in a virtual world. [Table 1] illustrates various clinical applications of the metaverse.
| Uses of Metaverse in Healthcare|| |
Data from millions of different avatars can be used by health organizations in the Metaverse platform. Cost-effective, outcome improvement may be possible using AI in a simulated environment, simultaneously obtaining insights about different treatments. Digital modeling will enable tracking and tracing healthcare facilities, equipment, and supplies in near-real-time, matching supply and demand. Virtual models are being made of human organs, facilities, and supply chains. In two decades, digital twins will be commonplace. Health data from in utero, to even cryofreezing of organs after clinical death, will be perpetually stored in the cloud. Wearable devices will converge, storing terra-bytes of individual information.
Immersive learning outside the Operation theatre (OT) is valuable. Head-mounted displays (HMDs) will contribute to surgical accuracy and education. MR is a continuum between the real world and the virtual world. Telemonitoring, or remote surgical guidance, is another area where AR and HMD show much promise. Surgeon's interactions with computer systems will be enhanced with 3D microscopy, endoscopy, advanced neuroimaging, and surgical robotics. With improved conceptual understanding of complex anatomy and enhancing visuospatial skills, the learning curve is reducing. Immersive touch provides a high-performance haptic and AR system combining human tracking and 3D visualization. Simulated surgical procedures can be practiced on a hologram on a virtual patient.,, Safety and security of patients' confidential information in the metaverse are crucial. Existing HIPAA guidelines dealing with telehealth and mobile device integration need to be upgraded. Patient compliance, adherence, and acceptance would also be crucial. Health metaverse may become a training test bed for surgical robots. Gamification is at present limited to health management and fitness applications. AR technology can introduce virtual coaches and exercise methods. Payment, insurance, data governance, privacy, and security must be adapted. Patients and doctors can meet in a 3D clinic with better teleconsultation user experience. High-tech hardware such as glasses, gloves, sensors, and other wearables will remotely read vital signs.
The Asian Thoracic Surgery Education Group organized an Outreach program using Extended Reality (XR) technology platform. 200 thoracic surgeons from different continents accessed the virtual environment wearing HMD units. Observing the surgery, in a virtual environment, discussions occurred in real time. Moving the mouse, every corner of the Operation room (OR) was visible. 3D headset ensured “presence” in the OR. Events such as these have led to formation of The Metaverse Doctors Association linking healthcare and the tech industry.
| Technology|| |
The futuristic programming model will include a new Hospital OS digital platform, electronic clothing, and data storage chips. In this living and open developing platform, millions of users will seamlessly move from one world to another using their own avatars. Using a headset connecting various digital environments, one can enter the virtual metaverse. High-end headsets and 5G can trick the human eye into seeing in 3D as animations move around in a virtual world. AR provides real-time guidance in the field of view, through integration with surgical navigation systems and data fusion from multiple imaging sources. With interoperability being crucial, assets such as avatars, 3D models, mixed reality, spatial settings, and data must work across all platforms and networks. VR/AR goggles, hand controllers, headgear, and even a full-body haptic suit to interact with healthcare providers in a virtual space may be necessary. Johns Hopkins neurosurgeons performed the first-ever AR-assisted surgery to insert six screws in a patient's spine. The Augmedics headsets enabled each team to see the patients' internal anatomies, including bones and tissue, which one doctor compared “to having a GPS for surgery.”, Microsoft Mesh, a “holoportation” and MR platform powered by Azure cloud services, makes digital connections life-like. This enables new ways to remotely teach, learn, and perform tasks virtually. Azure lets differently located individuals join 3D holographic experiences on various devices. Students wearing VR headsets practice lifesaving cardiac procedures for various simulated cardiac diseases on life-like mannequins. Microsoft's HoloLens technology has potential to provide medical care remotely. With Carnegie Mellon, meta AI is working on ReSkin, an open-source touch-sensing synthetic skin for robotic hands. ReSkin helps robots learn high-frequency tactile sensing, and DIGIT and ReSkin combined with TACTO facilitate tactile perception research. These gloves will locate the human hand in a VR space and their relation to virtual objects. The sense of pressure, texture, vibration are then simulated. Companies experimenting with AR and VR are partnering with hospitals, such as ThirdEye's AR glasses for first responders. 3D displays will have better ergonomics during extended operations, a Zoom-like service for the OT with in-built privacy and compliance. Incremental advances made a big difference. Brain–machine interface, wearables, and implants translate neuronal information into commands capable of controlling external software or hardware, such as a computer or robotic arm.
| Digital Twin|| |
A digital twin is a highly complex virtual model of an exact counterpart (the twin) of a physical object. The “object” could be a car, a building, a bridge, a jet engine, or a human. Connected with sensors on the physical asset, collected data are mapped onto the virtual model. In 1960, radiologists used phantoms to replicate reaction of human tissue to radiation. The expression “digital twin” appeared in 1994 in medical imaging. The digital twin of a patient is created by transferring the patient's genomic and clinical data and real-time changes in body parameters to the digital environment. Digital twin is the virtual twin of an individual - a 1:1 copy of all health data from antenatal to clinical death. All health information collected from wearable devices, Electronic medical record (EMR), etc., will converge into a universal metaverse ecosystem. Digital twin is the basis of the metaverse. When one falls ill, the “virtual profile” would be computationally treated. 25% of healthcare executives report using digital twins and 66% believe that investment in digital twins will increase. Philips and Siemens have developed digital twins of the human heart, simulating cardiac catheter interventions. Dassault Systemes has created a specialized digital heart model. Sim and Size platform helps neurosurgeons treat aneurysms with personalized simulations. IBM offers digital twin technology. Avatar Medical creates patient avatars to help surgeons visualize medical images. Digital twin-based Indian start-ups are deploying the “Whole Body Digital Twin” assisting individuals prevent and reverse chronic metabolic diseases.
| Virtual Reality|| |
VR is an immersive, completely artificial, computer-simulated image and environment, with real-time interaction stimulating all senses. Though in its infancy, VR technologies are being introduced in healthcare. AR- and VR-enabled experience will improve healthcare approaches, resulting in better outcomes. Medical applications of VR originally patient-centric are now moving toward clinician-as-user applications. VR/AR is used for designing operating rooms, including visualization of staff, equipment and surgical procedures, and patient education and treatment. In 2012, Pensieri and Pennacchini identified 12,000 publications on VR applications in healthcare. VR training with HMD provides an immersive and interactive learning environment. VR-based simulator training and assessment approaches are being used in spinal procedures. Providing more intensive, repetitive, and engaging training virtually has several advantages. These include (1) performing tasks with various difficulty levels for rehabilitation, (2) augmented real-time feedback, (3) immersive and engaging experiences, (4) safe simulation of real-world activities of daily living. In MR, there is a combination of virtual and physical components. In AR, the virtual component is superimposed onto the physical reality. XR is being used in cardiac surgery, 3D cardiac models promoting cell biology concepts, and multiplexed proteomics images. XR platforms are also used for biomedical education, medical training, surgical guidance, and molecular data visualization. This enhances comprehensibility of complex biological systems.
| Education|| |
Social communication, creating, sharing, and providing new immersive experiences in education is now part of the metaverse. Metaverse-based health education enables use of infinite space and data. 3D printing and AR/VR increase students' attentional span, facilitate higher learning enriching active learning experience, and create an interactive world of 3D images. AR images are superimposed in the real world, while VR produces simulated images in a nonexistent world. Cost and extensive training requirements need to be addressed. 3D printing technology used for robotic surgery training in urology may substitute for live animals and cadaver training. VR with AR simulators can train a wide range of psychomotor and basic robotic skills. This is in addition to dissection, retraction, cutting, and suturing skills. Trainees and instructors can track progress and individualize feedback using remote cloud-based access to simulation performance metrics. Scalability for mass production would depend on having a fully digital curriculum. Accuvein's skin-projected vein map helps one learn intravenous injections and phlebotomies. Continuous personal development will be the buzz words. Students will virtually enter different learning pods relevant to their courses. Virtual lecture rooms will allow students to listen to visiting professors and technology entrepreneurs giving masterclasses from anywhere. Bioscientists will conduct live experiments in AR/VR laboratories. Real-time ethical hacking and defense exercises on real-life infrastructures can be simulated by coders and cyber security students. AR/MR tools, used in teaching anatomy, will strengthen teaching quality-overcoming constraints of limited dissection specimens.
| Research|| |
David Baker's team deciphered the crystal structure of a monomeric retroviral protease using a protein folding game. The protein structure for a prospective acquired immunodeficiency syndrome treatment was obtained by 60,000 participants in 10 days, using a digital laboratory, enabling people to fold protein amino acid chains. Human-level touch is being replicated in robots using advances in tactile sensing ecosystem. Hardware, simulators, libraries, benchmarks, and data sets are necessary to build a touch sensitive AI system. DIGIT is a robotic compact, high-resolution tactile fingertip sensor for in-hand manipulation. A simulator like TACTO is prototyping, debugging, and benchmarking new advancements in robotics. Robotics contributes to rehabilitation, assistive surgery, elderly care, and prosthetics. Practical applications, acceptance, ease of handling, cost, and user acceptance are critical.
| Blockchain|| |
Blockchain facilitates ownership, online payments and traceability. An internet of things (IoT) sensor-based blockchain framework could track and trace drugs, monitor temperature, and counterfeit drug as they pass through the entire supply chain. Security challenges would include working with resource-constrained IoT devices and blockchain scalability to handle IoT sensor-based information. Blockchain could solve identity, data security, record management, and identify fraud. Blockchain can make administrative, care delivery, and payment processes transparent. Auditability, immutability, transparency, and encryption may allow more efficient, effective, secure, and transparent credential management processes, resulting in user and patient autonomy over their data.
| Economic Impact of the Health Metaverse|| |
When Zuckerberg rechristened Facebook (the world's most populated “country”) into META and diverted $10 billion into the metaverse division, he was getting future ready. AR in the global healthcare market was $1.06 billion in 2020, $1.42 billion in 2021, and expected to be $4.15 billion by 2025. Investment of $93 million was made in 2020 and $198 million in 2021 for supporting US startups using VR or AR for healthcare and wellness. Healthcare payments may be through cryptocurrency. Fortune Business Insights Report “Tele rehabilitation Market Size, Share and COVID-19 Impact Analysis” reports a market value of USD 3.32 billion in 2019. With a Compound annual growth rate (CAGR) of 13.4%, it is expected to be USD 9.13 billion by 2027.
| Problems and Challenges|| |
The conservative healthcare “industry” requires unequivocal evidence that investment in the metaverse is cost-effective, appropriate, need based and will improve healthcare outcomes. Interoperability, portability, stakeholder customization, human factors (cyber-attacks, distrust, skills, resistance), legislation, and regulation need to be addressed. Products and solutions constituting Digital Health gives patients and providers the ability to view, share, exchange, create, and interact with digital content. For the Health Metaverse to take off, profound digital transformation in process, workflow, practice, and delivery methods is essential. The metaverse needs to share data across institutional, systematic, and national lines. It is also prone to cyber-attacks risk VR organizations need to assess. Privacy and compliance issues are also a challenge.
Limitations of the Metaverse
The COVID-19 pandemic has promoted innovative development of digital health. Exploiting full potential requires massive infrastructure from uninterrupted 5G to high-tech hardware (glasses, sensors, and other wearables) and reimbursement from insurance providers. A new revenue generating, self-sustaining, business model is necessary. A study in 2021 suggested that 40% of the world have no access to the Internet. This itself would result in a class distinction of “haves” and “have nots.”
Virtual medical cities are planned for Singapore and Indonesia. Singapore-based Meta Health Ltd will have a metaverse through partnership with Aimedis a medical technology company. Hospitals and accredited medical service providers can open virtual clinics and provide services within specific platforms by leasing agreements with nonfungible tokens. The metaverse has to unify data, patient history, diagnostics, and provider coordination, leading to promise of data-driven outcomes.
| Conclusion|| |
The metaverse, though in its infancy, has promise for healthcare innovation and improvement. Having a place where people can face reality without necessarily experiencing the reality is intriguing as is watching the implementation evolve. Metaverse in healthcare could become a major innovation.
Ms Lakshmi rendered secretarial assistance.
Conflicts of interest
There are no conflicts of interest.
Informed consent/institutional ethical committee approval
Single author - Conception of the idea, drafting the article, critical revision of the article and final approval of the version to be published.
| References|| |
McKnight RR, Pean CA, Buck JS, Hwang JS, Hsu JR, Pierrie SN. Virtual reality and augmented reality-translating surgical training into surgical technique. Curr Rev Musculoskelet Med 2020;13:663-74.
Bernardo A. Virtual reality and simulation in neurosurgical training. World Neurosurg 2017;106:1015-29.
Koo H. Training in lung cancer surgery through the metaverse, including extended reality, in the smart operating room of Seoul National University Bundang Hospital, Korea. J Educ Eval Health Prof 2021;18:33.
Aulisio MC, Han DY, Glueck AC. Virtual reality gaming as a neurorehabilitation tool for brain injuries in adults: A systematic review. Brain Inj 2020;34:1322-30.
Renaudin CP, Barbier B, Roriz R, Revel D, Amiel M. Coronary arteries: New design for three-dimensional arterial phantoms. Radiology 1994;190:579-82.
Grieves MW. Product lifecycle management: The new paradigm for enterprises. Int J Prod Dev 2005;2:71-84.
Desselle MR, Brown RA, James AR, Midwinter MJ, Powell SK, Woodruff MA. Augmented and virtual reality in surgery. J Comput Inf Sci Eng 2020;22:18-26.
Sutherland J, Belec J, Sheikh A, Chepelev L, Althobaity W, Chow BJ, et al.
Applying modern virtual and augmented reality technologies to medical images and models. J Digit Imaging 2019;32:38-53.
Pensieri C, Pennacchini M. Virtual reality in medicine. In: Sivan Y, editor. Handbook on 3D3C. Progress in IS. Cham: Springer; 2016. p. 353-401.
Coyne L, Merritt TA, Parmentier BL, Sharpton RA, Takemoto JK. The past, present, and future of virtual reality in pharmacy education. Am J Pharm Educ 2019;83:7456.
Pfandler M, Lazarovici M, Stefan P, Wucherer P, Weigl M. Virtual reality-based simulators for spine surgery: A systematic review. Spine J 2017;17:1352-63.
Kim WS, Cho S, Ku J, Kim Y, Lee K, Hwang HJ, et al.
Clinical application of virtual reality for upper limb motor rehabilitation in stroke: Review of technologies and clinical evidence. J Clin Med 2020;9:3369.
Yuk FJ, Maragkos GA, Sato K, Steinberger J. Current innovation in virtual and augmented reality in spine surgery. Ann Transl Med 2021;9:94.
Venkatesan M, Mohan H, Ryan JR, Schürch CM, Nolan GP, Frakes DH, et al.
Virtual and augmented reality for biomedical applications. Cell Rep Med 2021;2:100348.
Kye B, Han N, Kim E, Park Y, Jo S. Educational applications of metaverse: Possibilities and limitations. J Educ Eval Health Prof 2021;18:32.
Bui I, Bhattacharya A, Wong SH, Singh HR, Agarwal A. Role of three-dimensional visualization modalities in medical education. Front Pediatr 2021;9:760363.
Costello DM, Huntington I, Burke G, Farrugia B, O'Connor AJ, Costello AJ, et al
. A review of simulation training and new 3D computer-generated synthetic organs for robotic surgery education. J Robot Surg 2022;16:749-63.
Romand M, Dugas D, Gaudet-Blavignac C, Rochat J, Lovis C. Mixed and augmented reality tools in the medical anatomy curriculum. Stud Health Technol Inform 2020;270:322-6.
Khatib F, DiMaio F; Foldit Contenders Group, Foldit Void Crushers Group; Cooper S, Kazmierczyk M, et al.
Crystal structure of a monomeric retroviral protease solved by protein folding game players. Nat Struct Mol Biol 2011;18:1175-7.
Pradhan B, Bharti D, Chakravarty S, Ray SS, Voinova VV, Bonartsev AP, et al.
Internet of things and robotics in transforming current-day healthcare services. J Healthc Eng 2021;2021:9999504.
Singh R, Dwivedi AD, Srivastava G. Internet of things based blockchain for temperature monitoring and counterfeit pharmaceutical prevention. Sensors (Basel) 2020;20:3951.
Apollo Telemedicine Networking Foundation, No. 21, Greams Road, Chennai - 600 006, Tamil Nadu
Source of Support: None, Conflict of Interest: None