CAPS Methods Core Seminar: Automating Tasks with the Redcap API

Statisticians and data analysts may find themselves repeatedly performing the same task with Redcap data, such as manually exporting data to a CSV file, processing the data, exporting to CSV, then uploading the transformed data back to Redcap or emailing it as a report.  Or research assistants may be burdened with a recurring task of manually transcribing data from an external source or another Redcap form.  Studies with as few as several dozen participants that require frequent communications can consume significant staff time when manually performed.  Repetitive scenarios such as these may be handled more efficiently by leveraging the Redcap API, which permits deployment of fully automated, customized solutions that can solve complex business problems while reducing or eliminating staff/analyst effort and promoting consistent and accurate results. 

This will be a practical, technical presentation using Python on Linux/Unix.  Many of the basic concepts can be applied to different languages and operating systems, so R and SAS Windows users can adapt ideas to their own work.  Investigators can benefit by observing how workflows can be automated, even when using other data sources such as Qualtrics.  The discussion will span five main areas:

1. Lightning intro to Python 3 and Jupyter notebooks
2. Redcap API setup and interaction using the Redcapy open source library
3. Communication using the SMTP (email) protocol and SMS via the Twilio API
4. Real world use cases (architecture and code walkthroughs)
5. Production server environment setup, job scheduling, and related tasks

Our use cases will show, for example, auto-distributed Python-generated reports using exported Redcap data, and scheduled copying of data from one project to another.  More complex scenarios will show how we automated the transmission of select participant enrollment data from Redcap to a third party (Agile Health) server using both APIs for several hundred participants, and how we routinely used the Twilio API to communicate reminders, performance feedback, and earned rewards directly with study participants.

While incorporating API access into laptop-hosted code can improve an analyst's efficiency, scheduled tasks ideally are hosted on a production server to ensure availability and reliability.  To properly use and manage one, we will review important topics such as network architecture, UCSF data center server hosting, remote access, shell scripts, environment variables, version control, logging, and batch job scheduling.

Bio: William Santo is a data engineer/scientist in the Preventive and Restorative Dental Sciences department of the School of Dentistry.  As part of its Data Coordinating Center, he currently supports data integration and development efforts for several institutions collaborating in NIH/NIDCR-sponsored UH3 projects as part of the Multidisciplinary Collaborative Research Consortium to Reduce Oral Health Disparities in Children (MCRC OHDC).  Since 2004, responsibilities have included report writing, systems design, statistical/deep learning analyses, server administration, end-user support, data management, and web/desktop/mobile development using multiple programming languages.  CTMS platform experience includes Redcap, Qualtrics, OnCore, and a custom SAS/AppDev Studio (Java-based) Web app.

Title: Automating Tasks with the Redcap API

Presenter: William Santo, BA , Data Engineer/Scientist @ Center to Address Disparities in Children's Oral Health (CAN DO), Department of Preventive and Restorative Dental Sciences

Room: AmfAR Conference room MH-3700

Please let Estie Hudes know if you are planning to attend the seminar and, if you are coming from outside of Mission Hall, whether you need to be put on the building security list.