smarty4covid: Intelligent Multimodal Framework for COVID-19 Risk Assessment and Monitoring based on Explainable Deep Learning

The project aims at the development of an Intelligent Multimodal Framework for COVID-19 Risk Assessment and Monitoring based on Explainable Deep Learning. Multimodal deep learning techniques will be applied on heterogeneous data, consisting of self-reported risk factors, along with audio recordings of breathing, voice and speech, towards the discovery of novel biomarkers of infection and disease progression with the ultimate goal to (i) increase the sensitivity of the screening procedure by detecting cases with high risk of COVID-19 infection, (ii) enable the remote monitoring of patients with COVID-19, and (iii) provide alert signals in case of emergency.

The project advances the current state of the art by initiating innovative clinical pilots involving hospitalized patients with COVID-19 through a strategic partnership with the AHEPA Hospital. It is expected that the application of AI on the heterogeneous data (e.g. medical history, laboratory exams, audio recordings of breath, speech and cough) that will be collected within the framework of the clinical pilots will produce new biomarkers of disease progression.

Besides the version used within the clinical study under the supervision of healthcare professionals, smarty4covid has released a web-based application smarty4covid to facilitate crowd sourcing data collection. This parallel study aims at the development of models able to predict the risk of COVID-19 infection and provide biomarkers for disease progression in infected non-hospitalized individuals.