Introduction

Project Overview
Chewy customers were receiving limited information on the status of their package deliveries. This led to confusion as to where they should look to find more information or whether they needed to take any further action to receive their goods, such as contacting customer service. We hypothesized that showing customers more information in the right place could reduce customer anxiety and reduce their need to contact customer service. The resulting solution saves Chewy $1.05M yearly in customer service contacts.

Role
Staff Product Designer for the Delivery Experience team, responsible for all design efforts.

Team

2 Product Managers, Delivery Experience
Product Manager, Account
Technical product managers for Delivery Experience, Account, Storefront Web and Native Mobile Applications
Software Engineer Managers for Delivery Experience Technology, Account, Storefront Web and Native Mobile Applications

Problem Statement

The Challenge
Customers were seeing limited status messages to communicate information about their deliveries on their account pages and in email communication. In order to get more granular information, they must click through those experiences to reach a tracking page, leading many of them to contact customer service instead. Questions related to order status amounted to an estimated 754k annualized contacts or 3.6% of total contacts ($5.2M entitlement).

Context and Constraints
Status messages are derived from a series of 140 event codes that are communicated to Chewy via API (Application Programming Interface) from their third party carriers (such as FedEx or OnTrac). These event codes communicate information about the entire lifecycle of a delivery, and need to work for any possible circumstance. Translating these events directly to the customer would result in a noisy or confusing experience. Event codes needed to be broken down categorically and mapped to new status messages, with accompanying changes to the user interface to accommodate them.

Research

User Research
Initial discovery for this project began using quantitative analysis of Chewy’s Voice of the Customer (VOC) data. Voice of the customer is a tool and team that analyzes transcription of customer calls using natural language processing and understanding. Call data is aggregated and summarized in custom reports to allow team members to dig into specific problems. I worked with the VOC to develop a new dashboard to help uncover gaps in our customer experience.
Estimates from our call data indicated that of 754K customer calls, with calls related to order status satisfaction levels were 5.8% lower and customers experiencing this gap were .75% more likely to churn (leave Chewy as customers).

Findings
Digging further into the data, it became clear that customers were 9.52% more likely to call about “unhappy path” scenarios in which a delivery was having some kind of issue, such as shipping delays, or damaged or lost packages. Despite these kinds of issues occurring for 11.7% of orders, this information was not being displayed in an easily accessible place for customers. The team prioritized our work using pareto analysis to focus on the events that were causing the majority of customer contacts.

Design

Hypothesis
I believed that by showing customers more granular information about their package status, and making it both visible, accessible and understandable, that we would reduce the need for customers to contact customer service.

Process: Content
Following our initial discovery, we needed to determine how to provide more information to our customers. I worked with my product management partners to examine the 140 priority tracking event codes that we were receiving from our third party carriers. I broke down these event codes into whether they were relevant to customers or not, and then further into whether or not these codes applied to the expected “happy path” of a delivery or not. I grouped these codes by their statuses and wrote 24 unique contextual messages for each group that could be conveyed directly to the customer. The statuses were left out of scope for the pilot, and marked for improvement in the future. Additionally I worked with our partners in Customer Service to provide new messaging to our agents that would help explain to customers what they would see in the Chewy app. This messaging was added to backend service that processed the incoming event codes and returned the appropriate and contextual response for our customers and agents.

Each new message was documented along with the four following pieces of information to provide instruction to our internal API team:

Documentation showing example for one of the 24 new messaging use cases.

Event Codes 140 Use cases were identified as candidates for the pilot based on these top level event codes and their associated sub event codes. They were mapped to 24 unique combinations of tracking statuses and contextual messages.
Current State As noted previously, this is the only information that was currently provided to the customer. As part of the pilot, these statuses remained unchanged until all Chewy platforms were displaying contextual messages.
Launch Experience Customers will see this kind of message in conjunction with the tracking status.
Customer Service Message Customer service team members will also see internally-focused messaging providing guidance on how to handle a customer contact related to this particular tracking event status and message.

Process: User-Interface
The Chewy app needed to be modified to accommodate this new information. This gave me an opportunity to align our messaging to our new Chirp design system. The design system provided an alert system that covered four types of messages, but did not provide a solution for in-line status as part of a component. The solution shown here was aligned to the design system’s alerts using the same categorical color and icon usage.

iOS example of the changes made to accommodate new messaging.

The user interface made the five following changes to make space for the new contextual messages:

The truck icon was removed from the placed on date to remove any confusion with the estimated delivery date.
Package statuses were aligned across all Chewy platforms (app/web) to use the new message API as the source of truth
Contextual messages were added to further explain the package status
Estimated delivery dates were added so that customers did not have to go to the track package page to see this information
Package numbers were moved to the right to increase the density of information and reduce scrolling.

The new messages contained the following parts:

Anatomy of a contextual message, and supporting content

High level status that maps to the icon and corresponds to the priority of the message displayed
Color coded icons to draw the eye in and quickly provide a visual cue to the status message
Contextual Message that is written in one or two parts:
- The new context that explains what is happening with the customer’s delivery
- An additional textual Call-To-Action to either provide self-service options or a suggestion to contact customer service
Estimated Delivery Date to provide an updated time to expect a package
Package number to provide information about how many packages are in the order.

Testing and Validation

Testing
Contextual Messaging was piloted in the first quarter of ‘23 through a progressive A/B experiment. As the pilot progressed, the percentage of users seeing the test vs the control was increased. The team established guardrail metrics around customer contacts to ensure that we were not seeing an increase in contacts due to the test cohort. A/B testing was chosen due to the fact that this change in information did not include any specific trackable interaction metrics outside of page views that could be used to determine customer behavior, and the impact of the change would be best validated through quantitative testing.

Metrics and Success
The result of the experiment was that the overall contact rate related to questions of order status dropped by 14.4 basis points from .69% to .55% with statistical significance (meaning that the change was not due to random noise in the data) . Of that group, the contact rate specifically for deliveries where issues occurred, such as delay, damage or lost, dropped by 39 basis points from 2.06% to 1.67%, also statistically significant.

Results

Outcomes
The launch of Contextual Messaging at Chewy reduced the contact rate as a percent of total orders shipped by .116%. Based on 2023 projected orders of 140M, the result was a reduction of 162K customer contacts per year at $6.48 per contact, resulting in savings of $1.05M yearly for the company.

Impact
The significant financial results are just the outcome, by they tell a story of the power of communication to reduce customer anxiety, and that better clarity can avoid confusion. By providing customers with more important details about what was happening with their order, they were able to make decisions about how to proceed, rather than feeling lost and that they needed to speak to a person in order to get more information.

Contextual Messaging