AI x Customer Research - November '24

Make more impact with insights personalized for stakeholders, faster usability test planning, an "agentic" process for following up feedback, and more...

Author

Caitlin Sullivan
November 30, 2024

Read time: ≈8 minutes

👋 Welcome to another edition of AI x Customer Research!

I’ve had a month of quite a few experiments since we last saw each other here. This month’s edition will focus on a few tutorials and tactical AI use cases.

Lately, I’ve worked with multiple startup founders who weren’t used to usability testing, and struggled to write usability test protocols to measure the right things. I gave them a few prompts for that, and I’ll share them with you.

Most of us have also delivered insights that didn’t hit the mark - not because they weren’t good insights, but because we didn’t frame them for our audience. I use AI to help me personalize my findings per stakeholder type in Notion.

Plus, “agentic” workflows are hyped, but what is an entry level “agent” we can try out? I have one for you, using Hotjar, Zapier and an email account.

And lastly, I’m finally testing AI moderators! Learn more at the end of this edition.

Let’s dive in -

In this edition:

  1. 🪪 Personalize research findings for stakeholders: Get stakeholders to actually read your work by personalizing findings in seconds.

  2. 💌 Generate personal email replies to customer feedback. Step by step, from Hotjar surveys to custom AI emails via Zapier.

  3. 🧪 Faster Usability Test planning with prompts that write protocols for you.

  4. Plus…I’m prepping the big AI Moderators test for you, coming in December! Here’s what I’m planning, and you can participate…

Let’s do this 👇

WORKFLOW UPGRADES

🪪 Get Stakeholders to actually read your research. Personalize findings with Notion AI.

You’ve done the research, the insights are (hopefully) brilliant, but now comes the tricky part—getting stakeholders to pause in their busy days, and care. Most of the time, they don’t need the full report (and they won’t read it). They need the highlights, tailored to their priorities.

Why this matters:
Getting buy-in from stakeholders can be made easier - by delivering insights in a format each stakeholder will find relevant to them. But that doesn’t need to take more time. Turning one set of insights into multiple versions can happen in seconds with AI.

I use Notion AI to customize insights summaries that get my different stakeholders and client types to pay attention. Here’s how to customize your results for Executives, Product Managers, and Designers—and get more of them acting on research.

〰️

💌 Tutorial: Generate personal email replies to customer feedback (an entry-level Agent)

Ever get that nagging feeling that you forgot to follow up with a customer who gave you valuable feedback? 🫣 It happens all the time. Most weeks, we don’t have enough hours to follow up with customers with the level of personal attention we’d like to.

Imagine this: Every time a valued customer leaves you helpful feedback in-product, they get an equally considered message letting them know (1) that we appreciate their feedback, and (2) maybe that we’re even working on what they mentioned.

I think this is a great use case for AI. Because, against popular opinion, we are not that good at identifying AI-generated content in contexts like this. Good prompting skill goes a long way to generating AI emails that aren’t an embarrassment.

Here’s a tutorial to walk you through creating:

  • Hotjar survey responses that populate a Zapier table

  • …that generates custom email text based on the survey response

  • …that lets you check the AI-generated text before sending

The step-by-step tutorial videos are below -

Enjoy!

PROMPTING PLUS

🧪 Faster Usability Test Planning

Usability tests are vital, but planning them can be time consuming—especially if it’s been a while since you planned your last test. Writing scripts and finding ways of measuring key UX elements like "trust" or "delight" can also be hard to do spontaneously.

These AI prompts can help you come up with ways to measure tricky UX elements and build complete test protocols in minutes.

For best results, add details like your project goals, examples of "good" outcomes, and industry context. Expect to tweak the output slightly, but these prompts have recently gotten my clients at least 85% of the way to finished protocols.

Steal these prompts -

1. Create a complete test protocol.

Why use this: It builds a ready-to-run test script, saving your team many minutes (or maybe even hours) of planning time while focusing on key outcomes.

Prompt: "You're a usability testing wizard for [specific domain]. Write a test plan for [feature or tool], including 5 tasks and follow-up questions that uncover insights about [outcome, e.g., ease of navigation or user confidence]."

2. Measuring UX Metrics like confidence, trust or delight

Why use this: Confidence, trust and delight are critical but hard to measure. This prompt gets you actionable metrics your team can track over time.

"I need 3 ways to measure how [confident/delighted/other feeling] users feel when using [specific feature] in a user test. Keep them quick, actionable, easy to analyze and compare in many tests over time."

3. Identifying pain points in the design

Why use this: You know this. 😄 Knowing where users struggle is the first step toward prioritizing fixes that matter most to them.

"Help me create a test to uncover the most frustrating parts of [feature/use case/product]. Include realistic tasks that trigger the user to walk through a use case they have actually experienced before. Add questions I can ask to reveal their pain points throughout the test."

4. Comparing versions to drive improvements

Why use this: Direct comparisons give you clear data to back design decisions, especially when iterating on a feature.

"Write a script to compare how users experience [feature v1] vs. [feature v2]. Include tasks to measure [metric, e.g.: time to complete, satisfaction] and follow-up questions to dig into what drove the user to take actions and respond in specific ways in the test."

5. Uncovering high-impact features

Why use this: This helps you pinpoint the features that deliver real value, so you can focus your team’s efforts on amplifying them.

"Draft a protocol to figure out which parts of [feature, product] provide the most value. Include tasks to measure [metrics, e.g., speed, accuracy, or satisfaction]. Craft follow-up questions that get far beyond surface-level responses from the user about what is valuable and why."

PARTICIPATE

🧑‍🔬 AI Moderators Uncovered - See what it’s like to be on the customer’s side

I promised I would dig into AI Moderators for you - and that’s finally coming to you in December.

I’ve tested 11 AI moderation tools on my own. But in order to run the best test I can, I need real participants - no synthetic users here.

This week, I started testing 6 AI moderation tools with participants on LinkedIn. I’m already collecting results. 🏃‍♀️

I’d like to invite YOU to participate if you’re interested.

What’s in it for you? (Since you already get the results here next month!)

  • You’ll see what it’s like from your customers’ perspective

  • The more participants I have, the better the report results will be for you

While I can’t guarantee that I’ll still need participants by the time this reaches you, I’ll hopefully still have a test link to send you.

To participate, just reply to this email with “AI Moderators”

As always, thanks for being here. I hope this edition serves you well going into the final stretch of the year.

See you in December,
Caitlin