Qualcomm : Porting Sonde’s Cloud API to Snapdragon processors to harness voice as a new vital sign

Over 100 muscles are used to create voice. The nerves that innervate these muscles come from the brain as the concepts and words are formulated there. The air that powers the voice is generated from our lungs. The energy that is needed to power these systems comes from the heart.

Sonde Health - a Qualcomm Platform Solutions Ecosystem Program member - can capture health states from these body systems with a simple 6-30 second voice utterance recorded on smartphones. Sonde's mission is to empower billions of people with mobile phones to do a voice-powered health check instantly. To help bring voice-powered health checks to billions of people, Sonde has optimized their solution on Snapdragon processors to harness voice on-device - something never offered up until now.

Sonde engineered a cloud-based platform that has already collected over 1 million voice samples to build its vocal biomarker technology. It offers to its partners a cloud-based API for a health insights measure.

Click to see a larger image

Here is how Sonde's solution works:

1. The user captures 6 to 30 seconds of voice response to a prompt;
2. The audio samples are collected on the device, encrypted, and analyzed to extract 500 features from the voice sample;
3. Sonde's algorithms check the voice sample for audio quality, correct utterance, and background noise;
4. When the audio passes these tests, the samples are sent to the scoring engine, where the vocal biomarkers signals are extracted from voice by both Sonde proprietary algorithms and specific machine learning based models.

Sonde's APIs allow their partners to rapidly integrate Sonde's solution into their telehealth, back to work, remote patient monitoring, and clinical applications. While a cloud-based implementation delivers profound objective health insight from speech, the edge embedding on Snapdragon allows the Sonde platform to improve scoring latency, security and privacy, and scalability. Furthermore, it reduces the burden on users to have to remember to complete a health check if they opt-in for periodic passive sampling to detect changes across a host of health conditions. In collaboration with Qualcomm Technologies, Sonde is achieving these user experience milestones by optimizing its technology for use with Snapdragon 888 and 788G processors.

Scoring Latency - For Sonde's mental fitness health check, current cloud-based scoring has a latency of 40 seconds. This includes the transmission of 30 seconds of captured audio (appx. 2.7 MB of data) from the device to the backend. In the backend, sample quality is checked, and then features are extracted and scored. The scores are then sent back to the smartphone.

Such a latency for the score is a poor user experience. As shown in the block diagram above, Sonde ported its vocal biomarker signal processing library from the cloud to a Qualcomm Hexagon DSP processor with cDSP (Compute-DSP) and Qualcomm Hexagon Vector eXtensions (HVX).This port reduced the latency for the Mental Fitness check from 30 seconds to 7 seconds and the Respiratory check from 10 seconds to 1.3 seconds.

Passive health check modeSonde offers a passive health check option where users do not have to give a voice sample proactively. Only with the user's consent, when a user is talking to or calling from their edge device, Sonde can perform a health screen on that audio. For example, when a user is saying a wake word and then saying a command or in a call, Sonde can acquire the half-duplex audio (only user side audio and not far side) without impacting the call application. This audio then follows the same path in the processor as an explicit audio sample, scored. The score can be aggregated and shared with users daily, weekly, or upon detection of a change in health.

Sonde's current implementation includes a passive health check in a VoIP/data call - enabling an entirely new and "hands-free" way for users to keep tabs of their health. Sonde's passive mode implementation can also be enabled through Qualcomm Technologies' Low Power Audio Subsystem Hardware Architecture implemented on aDSP (Audio DSP). The aDSP, which is always on, can do a preliminary check on the audio to detect valid audio (e.g., volume, energy, audio length, acceptable signal-to-noise ratio) and pass the audio to cDSP/HLOS for health scoring.

Machine learning model acceleration

Sonde's vocal biomarker-based health check scores are based on both signal processing algorithms and machine learning models. Sonde has ported its models on an ONNX framework that is executed in the HLOS. Currently, the models are population-based, which will adapt over time as the population model improves and user longitudinal samples are acquired. There are plans to improve the score accuracy through personalized models for individuals.

Security and Privacy - Health scores and the audio samples captured to assess them require strong security. In an edge implementation, the recorded audio does not leave the device for feature extraction and scoring. Captured audio files are immediately deleted after scoring. A user's scores remain private on the phone and can be securely shared with other secure health and wellness applications.

Learn how the idea to use voice as a biomarker came about...and how Snapdragon plays a critical role.

Check it out

Scaling - Billions of voice-based health checks in the cloud would consume energy and will be expensive. Moving Sonde's health check technology to the edge requires very minimal cloud infrastructure and offers significant cost savings.

Availability - In many parts of the world, Internet access remains sporadic and expensive. With the Sonde application and APIs running locally on the device, health checks can be done in areas of the world where network connectivity is limited.

By collaborating with Qualcomm Technologies, Sonde's opitmization of its solution with Snapdragon processors can enable voice-based health scoring at a very low price and a heightened level of security for many individuals.

Learn more about the Qualcomm Platform Solutions Ecosystem Program

Learn more about Sonde Health

Snapdragon and Qualcomm Hexagon are products of Qualcomm Technologies, Inc. and/or its subsidiaries. Qualcomm Platform Solutions Ecosystem Program is a program of Qualcomm Technologies, Inc. and/or its subsidiaries.