Post-Canvas Rollout Research
Post-rollout friction, ranked and fixable
A mixed-methods CSAT/PMF study that turned scattered post-rollout complaints into a ranked, fixable list — led by a lack of AI conversation memory.
Situation
After the AI Canvas rollout, early signals suggested real product-market fit for integrated AI workflows — but a handful of recurring complaints kept surfacing in scattered support tickets and sales calls, with nobody having measured how widespread they actually were.
Task
Quantify adoption, retention, and satisfaction for AI Canvas post-rollout, and determine which recurring complaints were genuine blockers versus noise.
Approach
A mixed-methods study combined a survey — fielded to 1,200 active AI-feature users, with 56 responses in a single day — measuring CSAT and PMF, with ten 30-minute follow-up interviews going deeper on awareness, initial reactions, Flows and Sidekicks usage, and AI tool switching.
Result
Satisfaction landed in "fair" territory by industry benchmark (52% top-two-box CSAT), and product-market fit read as moderate-to-borderline (36% "very disappointed" if the product disappeared — above the weak-PMF line, below strong). The interviews explained the middle-of-the-road numbers: the single most consistently cited friction, across every user segment, was the lack of conversation memory — AI sessions resetting mid-project forced people to rebuild context repeatedly. Close behind were trust issues (platform reliability undermining confidence when demonstrating AI live to clients or executives), output control (AI-generated content routinely exceeding scope, forcing manual scoping-down before anyone could present it), data-residency questions from regulated-industry users, and domain-specific language gaps for non-profit and community-sector users that generic AI assistants weren’t built to understand.
Methodology
Running CSAT and PMF side by side, rather than either alone, avoided over- or under-reacting to a single number — "fair" satisfaction paired with "borderline" PMF told a more honest story than either metric would have told alone. The ten interviews did the real diagnostic work: without them, "AI reliability" would have stayed a vague complaint instead of a specific, fixable list (session memory, output scoping, data residency, domain language). Quotes were kept anonymized by role — consultant, coach, product manager — rather than by name, consistent with how sensitive, client-facing concerns were being raised.
Skills demonstrated: Survey design, Qualitative synthesis (diary studies, interviews)