DTSonics: Sonified Digital Twin of the Human Body
Digital twins, a concept originating in engineering and used for representing physical objects, have garnered increasing attention in healthcare. Health digital twins can model the human body by integrating patients’ biomolecular, lifestyle, and environmental profiles, enabling real-time simulation of disease trajectories and prediction of health outcomes. Yet, health digital twins remain an emerging technology that is not yet widely understood or adopted. In this paper, we aim to make the concept of a digital twin more accessible by developing an interactive, sonified version that models aspects of the human body.
We developed DTSonics, a prototype of a sonified digital twin, by mapping simulated data from wearable sensors and controlling sound and visuals using Max/MSP and TouchDesigner. In this simulation, health data signals were mapped to musical parameters such as pitch, rhythm, and timbre, generating dynamic soundscapes that reflect the participant’s current physiological state. This enables the real-time modelling of health data and allows participants to interact with their own personalised, health-based digital twin.
Traversing the intersection of music, biology, and data science, this study aims to highlight the emerging role of health digital twins and provide audiences with unique insights into the complexity and variability of the human body. Future work could focus on incorporating additional clinical parameters and integrating with biofeedback sensors and other relevant hardware.
Jia Wei Lee
National University of Singapore
Jia Wei Lee is a PhD Candidate at the National University of Singapore Yong Loo Lin School of Medicine, supported by the NUS President’s Graduate Fellowship (ISEP). Her research focuses on computational genomics, precision medicine, and AI-driven digital twins. She is passionate about interdisciplinary collaboration and science communication—to go beyond scientific research with the goal of informing, educating and inspiring.
Dr Jia Yi Lee
Jia Yi Lee is a Singaporean composer who uses sound as a medium of understanding or perceiving the world. Her music explores color, space, and embodiment, linking them to larger issues surrounding culture, community, and the environment. In her curiosity-driven work, she challenges traditional notions of sound, creating imaginative, colorful sound worlds that captivate listeners. Her work ranges across instrumental music, electronic media, and interdisciplinary collaborations. She is currently Assistant Professor of Music at Davidson College, and co-founder of the New Uncertainty Collective.
Davidson College