CASE STUDY 1: Personalized CBT App >

StarlingCBT delivers personalized, adaptive Cognitive Behavioral Therapy education for employees suffering from mental health issues. As Head of Product, I delivered the company's first true mobile experience to made available to over 5,000,000 Canadians.

Project Title: StarlingCBT v5.0

Company: Starling Minds, Vancouver, BC

Years: 2019/20/21

Role: Head of Product, Design & Research, Chief UX Architect

Design Methodologies: design thinking, rapid prototyping

Research Methodologies: card sorting, lab usability testing, user interviews, survey design

Project Summary

With its next-generation digital mental health platform development project over 8 months behind and vastly over budget, I was brought into Starling Minds to build a professional Product team and launch a world-class mobile, digital therapeutic experience. This included building the company's first formal product management processes, rebuilding the engineering team, building design systems and a new design-to-development pipeline, and instituting Agile project management and formal data analysis and research.

Results over 18 months included over $5m in combined funding raised from SR&ED, IRAP, and Canada's Digital Technology Supercluster, a 3x increase in team size, the launch of a new next-generation, expert system mental health platform with mobile app, an acyclic-graph-traversal content management system, a patient monitoring portal, 40% increase in user engagement, and release of two new product lines serving the mental health needs of over 5 million healthcare workers, educators, and general workforce across Canada—all in the middle of a global pandemic.

The Challenge

Mental health challenges are at a global all-time high. Therapy is effective for many people, but therapists are in short supply and only a small percentage of people ever get the professional help they need. The goal of Starling Minds was to provide mental health support at scale through the use of automated Cognitive Behavioral Therapy (CBT) education delivered digitally, without human intervention.

Previous product attempts by the company had been successful enough at gaining user adoption and increasing positive outcomes to encourage investors to keep the company evolving. At the urging of investors, the company set out to develop a new version of the digital CBT (or iCBT) product, including an expert-system-based platform for personalizing the user experience and a mobile app. R&D had been done to create the architecture upon which the expert system could be built and prototypes of the platform and mobile app had been created, but the project was months behind schedule, over-budget, and none of the components were close to ready for release to general availability. The expert system had not yet been devised to deliver a personalized user experience, adaptive content based on specific user needs had not yet been developed, and the mobile app was only a medium fidelity functional prototype delivering years-old static content from the previous, out-of-date platform. Pressure from investors to release was extremely high. The company had recently taken on a new CEO and let go most of the engineering, design, and data teams.

There was no product, no team, no time, and no plan.

Early architecture and UI diagram

The Solution

At the invitation of the new CEO, I was brought in as a consultant in the Spring of 2019 to assess the market readiness of the new platform and mobile app. After two weeks of sessions with the team and analyzing the incomplete mobile app, I had to deliver my unfortunate news that the product would not be ready for general release for 14 to 18 months. After using my research to reset expectations with investors and the Board of Directors, the CEO offered me a new position as VP Product, where I would oversee product management, product marketing, UX and content design, engineering, devops, and data analytics.

People & Process

My first priority was to build a new team and the processes that would set them up for success. I built a standout team of product managers, designers, researchers, data analysts, data scientists, and engineers. Then, the first six months of my time at Starling was spent managing the stabilization of the aging platform that was still in production, while directing the team to invent and build upon the prototype architecture of the v5 platform. I brought the v4 production platform into HIPAA compliance and raised funding through SR&ED and IRAP grants to support the team expansion, while we also began devising the architecture that would drive the personalized, adaptive therapeutic experience. To facilitate all of this, I spearheaded the implementation of more formal Agile project management practices, implemented a stable design-to-dev pipeline (Figma > Zepplin > GitHub > AWS) and worked with the CEO to recruit a Head of Technology, my partner who would free me up to focus on product management, design, and research.

User Types & Occupationally Aware Experiences

The user experience of the platform required UIs for three separate user groups:

StarlingCBT for members receiving the CBT education,
StarlingHQ for customer users—disability case managers, primary care physicians—to refer the CBT app to users and track their progress (read the case study here),
StarlingAdmin for our internal Starling team to manage programs, products, users, customers, and content (read the case study here).

The primary user group for the StarlingCBT app were employees struggling with mental health issues. There were two main product lines providing tailored educational experiences to two more specific user types:

Starling Mental Fitness was designed for employees still at work but suffering from absenteeism or presenteesim due to mental health issues,
Starling Return to Health was designed for employees on a short-term or long-term disability claim.

Further, experience with earlier versions of the product showed that user engagement was higher with more occupationally-specific content. Accordingly, both product lines would have multiple sub-categories of users in verticals, such as education, healthcare, and general workforce. Within those sub-categories, specific experiences could be provided for occupation types, such as nurses, physicians, paramedicals within the healthcare vertical, and teachers and education administrators within education.

Target customers for the Starling Mental Fitness solution were enterprise employers, self-insured enterprises, and employee benefits providers. The target customers for Starling Return to Health were employee benefits providers and other organizations managing disability claims.

Persona guidance from our design guidelines.

User Research & Testing

Our user research methodologies consisted of several Design Thinking techniques including card sorting, user interviews, ethnographic study, questionnaires, and stakeholder workshops. Internal stakeholders who contributed came from our Therapy, Customer Success, Sales ,Marketing, Design, and Technology teams. The Design Thinking structure was new for Starling Minds and the team and product benefitted from it greatly.

Once design and development sprints started, we employed a rapid prototyping cadence which included user testing rounds on paper prototypes, clickable wireframes, and functional prototypes. User testing sessions were conducted in our offices in Gastown in Vancouver, but once the pandemic hit we transitioned all user testing to Zoom sessions. As I was the most experienced user researcher on the team, I did the bulk of early test sessions, and then trained multiple team members to prepare for and conduct user testing studies, which allowed me to gradually delegate more and more of those responsibilities.

Over the course of the design phase, iterative user testing reduced usability issues by 87%.

As an added dimension to the user research, I also started the company's first data analysis team with productized customer reporting to accompany the CBT app. Quantitative data from these user behavioral analyses were layered on qualitative user testing results to provide a deep, multi-faceted view of user progress and reaction to the therapeutic approaches.

User persona workshop board.

UX Design & Evolution

Once our initial research phase was complete, we began the UX design phase. This included the UI design and art direction for the app as well as the educational design of the therapeutic content. An early decision I made was to use Learning Designers with backgrounds in developing e-learning technology to design our educational and therapeutic content. This proved to be a wise decision as these designers brought a level of complexity and empathy to the design that lacked in competitors' solutions.

We learned many lessons during the design and invention phases. One of the main challenges in the design was to figure out how to merge static components of the app that would provide a ubiquitous user experience with very complex, adaptive parts of the educational and therapeutic branches that would personalize the user experience depending on changes in user need, symptoms, assessment scores, and choices.

You can learn more about the results of our content development methods in this case study.

An example of a high-level UX flow combining user interaction and nodes for automated branch traversal.

Design Themes

One of the most important aspects of the UX design was the consideration of user state of mind. This is always a consideration in UX design to varying degrees, but when providing support and education to people suffering from mental health issues, care needed to be taken to not inflict any undue stress on the user from either the app usability, language, or aesthetics.

One design decision we made that was vastly different for our competitors was to use nature imagery, rather than human imagery. Our early testing showed that many users actually experienced increased levels of stress caused by images of people who they did not relate to or images of stressful emotional situations. In order to be supportive and non-triggering, concepts such as anger or frustration could be symbolized by stormy weather, rather than an image of someone raging.

Other important design concepts were empathy, unconditional support, and encouraging rather than challenging language.

Emotional design guidance from our design guidelines.

Psychiatric Assessments & CBT Education

The educational direction of the product was based on standard Congitive Behanvioral Therapy concepts. Upon registration and intake into the app, users would be tested with standard psychiatric assessment instruments, including the GAD-7 for anxiety, the PHQ-9 for depression, and the DASS for stress. The app was not a diagnostic tool, however, the results of the assessments directly influenced the direction of the personalized educational paths. The paths would also be influenced by other data collected during the course of learning, which resulted in a very interactive and dynamic experience for users.

The structure of the learning modules took the form of individual sessions that could be completed within 5 to 8 minutes. Each session explored specific concepts and training in themed groupings. Each of these groupings successively built to give the user basic skills in identifying their mood patterns and triggers, changing moods, balancing and changing distorted thought patterns, setting realistic goals, and building mindfulness skills. The result for users who were still at work would be developing a set of skills to help themselves be more emotionally resilient at work and in the rest of their lives. For people on a disability claim, the learning was focused very specifically on overcoming emotional hurdles and preparing to rejoin the workforce.

Examples of input screens.

Supercluster Funding & Beta Release

When the global COVID-19 pandemic lockdown began in March of 2020, we were six months into design and three months into development of StarlingCBT v5.0. Realizing that we were at risk of missing the chance to provide our members support with the new platform at the early stages of the pandemic, we knew we needed to amp up production of the new platform. In the meantime, we released a product specifically designed for COVID-19 Anxiety on our v4 platform at the beginning of April 2020. That program was released free to the public and gained over 10,000 new members in just over two weeks.

Building on the success of that release, we secured $2.5m in funding from Canada's Digital Technology Supercluster to ramp up production of the new platform and get it into market more quickly. We quickly added new members to the Starling team and increased the pace of production. StarlingCBT v5.0 was released as a Beta in the early Summer of 2020, 12 months after I joined the Starling team and 4 months into the global COVID-19 pandemic. Initial Beta testing was done with employees at the Fraser Health Authority in BC. Several rounds of user testing were conducted during this time.

General Release & Results

StarlingCBT v5.0 was released to general availability in the Fall of 2020, within the original 18 month release estimate I made as the result of my initial research and consultation. The new platform and mobile app were well received by customers, users, and investors.

Since the release of the new version:

User engagement increased by 40% over the old platform,
Usability issues decreased by 87%,
77% of users reported lower anxiety, stress, and depression,
91% of users say Starling helped them improve their mental health,
4 in 5 employee absences were reduced by 6.5 days per year,
Customers had 90% reduction in related disability claim costs compared to human-guided support.

The development we achieved in this period set Starling Minds up as a standout mental health technology and put the company years forward than its original product roadmap.

“We use Starling because we believe that the total health of our employees is important. At the end of the day, the healthier people are, the more productive they’ll be. Ultimately, Starling lets our employees know we care about their well-being.” - A customer VP of Human Resources

"Starling Minds is the only digital mental health solution that fits exactly into our approach for addressing workforce mental health. Our workplace health and safety strategy touches employee experience, early intervention and prevention, and absence and disability management. Starling’s spectrum of programs couldn’t be a better fit for our people leaders and the employees they serve." - Executive Director of Workplace Health & Safety, Healthcare Authority