AngioDynamics : Using AI to Predict Clinical Trial Success

by Juan Carlos Serna, SVP, Scientific and Clinical Affairs, AngioDynamics

Running a clinical trial for novel therapeutic discovery is expensive - they run from $12.2 million - $33.1 million per trial.[1] When a trial fails to complete, there are significant financial repercussions, resource wasting, and, more importantly, potential human cost. Terminated or inefficient clinical trials substantially slow our ability to get life-changing treatments into the hands of the people who need them. Artificial Intelligence (AI) is an untapped resource that can help us more accurately predict whether or not a clinical trial will be successful.

Machine learning (ML) tools in the medical field have mostly focused on designing clinical trials. They are employed in target validation, identifying prognostic biomarkers, and analyzing digital pathology data in clinical trials[2]. Yet, they have left unaddressed the issue that clinical trials have a significant rate of abandonment. A predictive model of successful trial completion will enable financial resources, study volunteers, and talent to focus on more promising projects.

Using New AI Tools

At AngioDynamics, using AI to predict clinical trial abandonment was something we've wanted to do for a while. But the tools available in ML require complex computer architecture, data manipulation, and multidisciplinary biostatistical and data science support. We turned to a new platform developed by Akkio (www.akkio.com) that allows non-machine learning experts to build and deploy predictive models without the commonly associated barriers with ML.

A massive trove of data on clinical trials can be found at clinicaltrials.gov, a US National Library of Medicine resource. We downloaded data on just over two hundred thousand clinical trials conducted over the last few years. For each study, there is data on 80 attributes of the trial - everything from title, location, the number of participants to detailed elements of the trial design. This data is a mix of numbers, categories, and free form text. Akkio uses natural language processing to weigh that free form text alongside the quantitative data. Most importantly, this data contains each trial's status - whether it was completed or was terminated. (Side note: the data we originally downloaded included some additional fields that described why a study was terminated - we excluded those fields in the creation of the predictive model, as that would be information not known during a trial).

Creating a Predictive Model

After uploading the data, we created a predictive model without writing any code, entirely through Akkio's interface in a web browser. Akkio automatically detected the type of data in each of the 80 fields. We selected the field 'overall_status' in the user interface, and Akkio created a predictive model in a couple of minutes.

The model correctly predicts the outcome of trials in a held-out test dataset 90% of the time. But what we care most about is predicting terminated trials - and that is a relatively infrequent outcome. In the 'test' set (separated from the model training dataset), the model correctly predicted 63 of 208 terminated studies. The software also delivers a percent likelihood for each result. You can see that the trials that did terminate were far more likely to have been terminated than the average trial. The model doesn't catch all the trials that will fail, but it substantially helps you focus on the most at risk.

Akkio also shows the fields with the most predictive power in the dataset. In this instance, they were Enrollment, Number of Facilities, Official Title, Interventions, and Intervention Model. Together they account for just over 22% of the predictive power, though, so they are far from the only factors that matter.

What's Next?

In the hopes of driving trial efficiency across the industry, Akkio has made the predictive model available on a public web page. We have taken the data from the ongoing trials we have at AngioDynamics and viewed their predicted results - and we're using that information to refine our clinical operation plans.

The medical community has high regard for traditional scientific methods and statistical analysis. Using ML to predict clinical trial completion is new - and what we've done here is just a starting point. But it's an exciting one that opens powerful possibilities for us to increase the efficiency of new treatment development.

Juan Carlos Serna leads AngioDynamics' global Clinical Affairs and Services, Medical Affairs, Physician Education, and Healthcare Economics teams as they work to advance scientific research and patient access to AngioDynamics' technologies and devices. To learn more about AngioDynamics, visit www.angiodynamics.com.

[1] https://pubmed.ncbi.nlm.nih.gov/30264133/

[2] https://www.nature.com/articles/s41573-019-0024-5

AngioDynamics is a registered trademark of AngioDynamics, Inc., an affiliate or subsidiary. All other trademarks are property of their respective owners. © 2021 AngioDynamics, Inc. GL/NA/CL/643 Rev 01 01/2021