Journal Club for Apr 2022: On-Body Mechano-Acoustics

On-Body Mechano-Acoustics

Xiaoyue Ni
Assistant Professor
Department of Mechanical Engineering and Materials Science
Department of Biostatistics and Bioinformatics
Duke University
ni.pratt.duke.edu

1. Introduction

Telemedicine has become essential during the COVID-19 pandemic but monitoring human-body signals comfortably and accurately in ambulant settings remains challenging. For this purpose, existing technologies aim to interface available sensors or sensing components to the body in a wearable form but usually encounter a fundamental mechanical problem lying in the differences between electronic materials and biomaterials. While traditional electronics are planar, rigid, and static, biological tissues are soft, curved, and continuously evolving. The mismatch in their mechanical properties usually causes huge strains and unstable interfaces under deformation. Integrating stiff electronic materials with elastic biomaterials is tricky.

The emerging flexible electronics exploits novel structures, materials, and forms to create electronics that can stretch, compress, twist, and deform into complex, curvilinear shapes while maintaining functionalities, and hence can be intimately integrated onto the human skin [ ] The conformal, seamless contact enables an advanced form of device to noninvasively detect human signals at a level that conventional devices with rigid form factor cannot achieve. For example, the low thermal impedance enabled by the close contact allows blood flow sensing using a thermal actuator and surrounding thermal sensors to measure the temperature gradient due to the stream [ ]. The large area electrodes laminated and wrapped around the elbow records local muscle activation events and provide high-resolution control signals for prosthetic limbs [ ]. The sealed interface between microfluidic channels and skin enables a wearable form of sweat collection and perspiration analysis for novel biomarkers related to metabolism [ ]. The recent progress in creating highly stretchable systems that incorporate assemblies of sensing elements demonstrates multifunctional operation for clinical-grade physiological monitoring in a wireless mode [ ].

Figure 1: Epidermal electronics enables advanced form of thermal, electrical, chemical, light sensing, paving a way for wireless, cost-effective, clinical-grade physiological monitoring. (images reproduced based on Refs. [ ])

Among all the signals (thermal, electrical, chemical, optical), mechano-acoustic (MA) signals are hugely underexplored. In meantime, they are interesting signals to study in-depth for mainly two reasons: 1) the body is almost transparent to mechanical or acoustic waves.2) the MA signals accompany almost any bioactivities – there is profound health information to obtain from MA signals, ranging from the quasi-static mechanics to high-frequency acoustics. Commercial devices are available to digitize some of the signals. For example, inertial measurement units capture body orientation or gaits [ ]. Strain gauges or force transducers built in a form of straps worn around the belly or the chest measure the respiration patterns or efforts [ ]. Actigraphy watch tracks locomotion or physical activities. A digital stethoscope records body sounds. The microphone records speech and vocals. These devices are all in a rigid, planar form with bulky construction. Their mass density and moduli are much higher than human skins, causing large acoustic impedance mismatch, attenuating most MA signals at the skin-air interface. These conventional devices have multiple limitations in detection bandwidth, mounting locations, and continuous monitoring.

2. Mechano-acoustic sensing

Recent advance in epidermal electronics embraces a soft, skin-like device architecture that incorporates a MEMS accelerometer to capture a full spectrum of MA signals [ ]. The device, when mounted at the suprasternal notch, records rich, multiparametric information. The suprasternal notch is a special anatomical position at which cardiac, respiratory, and digestive systems cross and co-locate (Fig. 2a). The skin vibrations at this soft notch position are a superposition of MA signals arising from a broad range of bioactivities associated with the various physiological systems.

Figure 2: Mechano-acoustic sensor enabled by heterogeneous integration of hard and soft materials. (images reproduced based on Refs. [ ])

The mechanics of the skin around the neck is very complicated. The device needs extra compliance to perform functionalities under large bending, stretching, and twisting deformation. In the meantime, to support the wireless operation and manufacturing options, the sensing system uses commercial components such as Bluetooth chips, lithium batteries, wireless charging circuits, peripheral electronics, and printed circuit boards (Fig. 2a). A key challenge lies in the materials and mechanical engineering to create an ultra-soft device to accommodate the conventionally rigid electrical architecture.

Heterogeneous hard-soft integration stands out as one solution. As shown by the previous work [ ], a network of serpentine-shaped interconnects of polyimide, with its modulus to be ~2 GPa, embedded in silicone elastomers with a modulus of ~2 kPa, exhibits J-shape stress-strain behavior that mimics the skin mechanics (Fig. 2b). In the initial loading phase where bending deformation dominates the constituent filaments, the effective modulus of the elastic sheet is approximately few kPa. Fig. 2c shows that, when employing the same serpentine structures to create multilayer, conductive interconnects to join islands of functional components, the resulting device consists of hard material contents but exhibits soft, skin-like mechanical behavior [ ]. This underlying principle shares similarity to the creation of mechanical metamaterials with tunable overall mechanics regardless of the intrinsic properties of constituent materials.

Several other mechanical and material considerations contribute to further enhancing the flexibility and stretchability of the MA sensor. For example, pre-compressed serpentine beam and optimized serpentine geometries boost elastic stretchability. A layer of soft silicone gel at the base of the system but within the encapsulation provides a decent degree of mechanical isolation of the device from the skin, reducing the stress accumulation at the location of the islands during the motions or deformations of the body. The elastomeric membranes serve as the encapsulation layers and provide waterproof function. An air pocket design not only reduces the mass density of the device but also the tensile, bending, and twisting stiffness by 2 ~ 3 factors. The resulting soft device can undertake 43% stretch, and 90° of twist but still function. The circuit design incorporates state-of-art SoCs (systems on chips) that are compatible with user interfaces of portable devices such as cell phones.

Figure 3: Representative MA signals from the suprasternal notch. (Images adapted from Refs. [ ])

Fig. 3a shows an example of three-axis acceleration data taken from the suprasternal notch of a healthy subject, where the z-axis is normal to the skin and takes the largest amplitude of vibrational signals. Over a 60-s interval, the subject engages in various activities that include sitting quietly, talking, drinking water, changing body orientation, walking, and jumping. At rest, the pulsed heartbeat signals and sinusoidal respiratory cycles (due to chest wall motions) show up clearly even in the unprocessed, raw data. Holding breath yields a plateau around 10 s. Speaking and swallowing produce acoustic vibrations. Leaning back and forth leads to the switch in all-axes data because of the change of projection of gravity in the device frame in reference to the canonical frame. Physical activities are associated with excessive accelerations. The signals exhibit different characteristics in the time and frequency domain, shown by the time series and the corresponding spectrogram analysis on representative events (Fig. 3b). Signal processing techniques exploiting the unique features can parse out quantitative, multiparametric physiological information from a single stream of data. The device enables continuous monitoring of vital signs including heart rate (HR), respiration rate (RR), and physical activities (PA), as well as unconventional biomarkers such as talking time and swallow counts, in an ambulant environment.

3. Automated, multiparametric monitoring                                                            

Not like the traditional multimodal health monitoring systems that are usually restrained in clinical or lab environments, nor like the existing wearables that focus on specialized monitoring modes on few, key biomarkers such as vital signs. MA sensors, taking the form factor as a soft patch and capturing multiparametric information from a single device, overcome the tradeoff between functionality and ease-of-use, enabling remote, cost-effective healthcare in home settings. Such capability is highly desired during the COVID-19 pandemic.

Figure 4: The health monitoring system incorporating an MA sensor for automated, multiparametric monitoring of respiratory biomarkers and vital signs in clinical and home settings for COVID-19 patients (Images adapted from Ref. [ ]).

An automated cloud-based data recording and analysis platform is built around the wireless MA device to continuously record vital signs and unconventional biomarkers, tailored for COVID- 19 patients, frontline health care workers, and others at high risk (Fig. 4a). The time-frequency analysis, followed by a convolutional neural network (CNN), outputs the detection of respiratory and vocal events, with coughs being of specific interest. Exiting microphone-based methods to monitor coughs usually encounter issues with background sounds and/or environmental noises. Due to the match of mechanical impedance between the device and the skin and the mismatch between the device and the air, the MA sensor captures skin vibrations directly and is insensitive to the air-borne acoustics, enabling subject-specific counting of each class of respiratory/vocal events. The robust, continuous recording generates reliable statistics for the occurring frequency of coughs, providing a signature of disease. In addition, the measured amplitude of the skin vibrations exhibits a high correlation with the intensity of plosive vocal or respiratory events, which has the potential to generate metrics of infectiousness, as activities like coughing, talking, and laughing can yield aerosols/droplets that contribute to virus transmission [ ]. Unlike microphone measurement of loudness which can be unreliable because of the undetermined separation between the device and the subject, epidermal sensing records high-fidelity timing and intensity information of sounds, serving as reliable means to quantify one aspect associated with risks of spreading the disease.

A Cloud-infrastructure and a user interface are implemented to support the automated wireless data transmission, storage, and analysis with a minimum need for manual operation. Once turned on and interfaced with the SN, the device can record MA data continuously for ~72 hours.  The signal processing and machine learning algorithms operating on the server deliver semi-real-time analysis output to a graphical dashboard for feedback to health workers and/or patients. Fig. 2b shows the example results for the detected coughing and talking frequency and intensity (color-coded) in 5-min windows, as well as the vital signs including HR, RR with their corresponding amplitude information (related to cardiac output and respiration effort) from remote, continuous 48-h monitoring of a COVID-19 patient.

4. Summary and Outlook

Recent progress in epidermal electronics and wearable devices provides new opportunities for precise, non-invasive, long-term recording of body mechanics. The soft, skin-like wireless device incorporating a single MEMS accelerometer captures a full spectrum of mechano-acoustic (MA) signals, from subtle vibration of the skin to precise kinematics of the body. A single accelerometer sensor captures body orientation/gaits (~0 Hz, ~10-1 Hz), respiration (~10-1 Hz), locomotion (~100 Hz), organ sounds (~101 Hz, ~102 Hz), and vocal sounds (~102 Hz). The related algorithm demonstrates the potential ability to measure cardiac activities, respiration pattern, actigraphy, body orientation, speech, swallowing, coughing, snoring, and sleep stages all at once, shedding light on the exploration of other unconventional biomarkers based on MA sensing.

A natural extension of the investigation is to explore high-dimensional MA sensing networks using arrays or distributed MA sensors. Example possibilities include using spatially distanced accelerometers to measure elastic waves. Provided a mechanical actuation source, the device using multiple accelerometers can leverage the spectral analysis of surface waves to derive the mechanical properties of the propagation media, such as those used in seismology for non-destructive testing of the layering profile of geotechnical materials [ ]. The results will enable wearable elastography, with a continuous monitoring capability for precise, robust, depth-sensitive tissue stiffness mapping.

We invite all researchers active in this field or who have a general interest in wearable electronics to share their perspectives. Introducing their recent progress related to this subject is also highly welcomed. We are looking forward to a fruitful discussion.

Reference