Product, Design & Research Leader
The Starling v5.0 platform delivers personalized, adaptive Cognitive Behavioral Therapy education. As Head of Product, I directed the Design team inventing novel solutions for authoring adaptive content and user experiences.
Project Title: Starling v5.0 Platform
Company: Starling Minds, Vancouver, BC
Years: 2019/20/21
Role: Head of Product, Design & Research, Chief UX Architect
Design Methodologies: design thinking, rapid prototyping
Research Methodologies: card sorting, lab usability testing, user interviews, survey design
Early scribblings.
Project Summary
The promise of Starling Mind's new v5.0 platform was to deliver psycho-educational experiences tailored to the unique needs and conditions of users. To achieve this, we developed novel tools and processes for building unique experiences that would guide the user through Cognitive Behavioral Therapy (CBT) scenarios that would adapt to their data phenotype and behavior. This resulted in the development of an acyclic graph traversal authoring system that could put the complete cycle of design, authoring, coding, upload, testing, and debugging in the hands of designers without the need for dedicated engineering resources, increasing time-to-market of new and updated psycho-educational experiences and reducing associated costs equal to two full-time software engineers and one additional full-time QA engineer.
The Challenge
While the Starling Minds Therapy team had a general conception of the CBT approach they wanted to embody in the StarlingCBT v5.0, and while the previous technology team had left us with a basic architecture upon which to build an adaptive content model and a raw prototype mobile app, there was not yet any way for the Design team to build the adaptive psycho-educational (eLearning) experiences that would react and change based on the current needs of the user.
Early versions of the Starling platform required dedicated engineering and QA resources for the development of new CBT experiences. One of the requirements of this project was to provide tools that would empower the Design team to be in full control of the content design and development cycle, increasing time-to-market of new content and reducing costs.
The Solution
To achieve these requirements, we built the content framework as an acyclic graph traversal model, whereby the user would be taken down a branched content path determined by their data phenotype and user choice. The structure of the pathways and the overall learning experience was based on a heuristic model of CBT, designed to mimic CBT treatment as delivered by a professional practitioner. This heuristic model was created and validated by our in-house team of clinical psychologists.
The data phenotype was made up of demographic data, responses to standard Psychiatric assessment instruments (GAD-7 for anxiety, PHQ-9 for depression, DASS for stress) at intake and throughout the experience, answers to a multitude of questions presented in the interactive learning content, and unstructured data provided by the user in the form of journaling and community discussions. About 70% of the traversal path decisions were made by the expert system based on the user data phenotype and about 30% were made based on user choice.
To enable the system to respond to the data phenotype, we developed a proprietary scripting language of rules that the designers could incorporate into the pathways at any point that the system needed to make a navigational choice to direct the user, or to present content compiled to suit the user's unique needs
Early versions of the pathways and content were designed in a mind-mapping tool called Xmind and printed on a plotter for discussions and editing by the team of designers and psychologists.
While this process was great for early collaboration and team bonding, it would be too unwieldy for production of the massive amount of content authoring needed to drive the CBT experiences. Eventually, the Technology team built an integration of the Xmind maps that would import and compile the branch structures, content, and therapeutic rules scripting language invented by the team into the automated learning experiences.
You can read more about that process in this case study.
An example of a traversal map and rules built in Xmind.
File taxonomy devised by the team for content organization and parsing into the platform.
Early Results
Our early content trees tended to be much more complex than they needed to be. As we conducted our iterative testing, research, and data analysis throughout the design process, we found that decreases in user stress, anxiety, and depression were not affected by over 70% of our content branches. It turned out that the efficacy of the CBT treatment did not require significant adaptations for people who were suffering from anxiety, as opposed to generalized stress, or depression. People responded positively to the adaptive content when it provided support specific to their experience and when it gave them opportunities to build and practice CBT skills, such as mood tracking and though balancing. More significant factors for the branching and adaptation were age, demographic data, personality type, and level of confidence. This resulted in a much more ubiquitous user experience that was simpler for users and much more cost effective to develop and maintain.
In order to create these content branches and the rules to traverse users down the most appropriate, individualized paths, we had to construct an intermediary system between the platform and the presentation layer for authoring and managing an acyclic graph traversal matrix. This is a fancy way of saying that we could author anything from very general, ubiquitous user experiences, to extremely personalized experiences that would adapt based on the user's demographic data, occupation, employment status, symptom intensity, assessment scores, and answers and choices made by the user in the course of the experience.
This authoring system would allow non-programmer designers to design, build, upload, and test incredibly complex adaptable experiences without the need for engineers to hard-code content courses, modules, or sessions. This architecture and feature set proved to be faster, cheaper, and much more flexible than maintaining an engineering team dedicated to coding each content experience. The content designers became what I liked to call "scientist poets," capable of designing technically complex, scientifically validated psycho-educational experiences with flair, panache, and artistry.
The Admin environment we designed and constructed for content authoring also gave the Starling team the ability to manage customer programs, manage users, manage media, handle translations, moderate the user community, and monitor and respond to users at risk of suicide or self harm. You can read the StarlingAdmin system case study here.
User testing early content wireframes.
User testing early content wireframes.
Results
The invention of the content authoring process enabled the Therapy and Design teams to deliver adaptive psycho-therapeutic experiences that improved user outcomes and increased user engagement. The methods we developed empowered the Design team to take control of the entire content release cycle, reducing costs and increasing flexibility and rapid iteration and testing.
Since the release of the authoring system, we achieved:
-
Increased speed-to-market of new psycho-educational content: new courses could be released in 4-6 weeks, rather than 6-8 months as was previously the case,
-
Reduced cost due to reduced engineering overhead: the new authoring system eliminated the need for two software engineers and one QA engineer, a savings of close to $400,000 per year in salaries and associated costs,
-
77% of users reported lower anxiety, stress, and depression,
-
91% of users say Starling helped them improve their mental health,
-
4 in 5 employee absences were reduced by 6.5 days per year,
-
Customers had 90% reduction in related disability claim costs compared to human-guided support.